Calculus, finite differences Interpolation, Splines,...

24
Calculus, finite differences Interpolation, Splines, NURBS CMSC 828 D

Transcript of Calculus, finite differences Interpolation, Splines,...

Page 1: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Calculus, finite differencesInterpolation, Splines, NURBS

CMSC 828 D

Page 2: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Least Squares, SVD, Pseudoinverse

• Ax=b A is m×n, x is n×1 and b is m×1.• A=USVt where U is m×m, S is m×n and V is n×n• USVt x=b. So SVt x=Utb• If A has rank r, then r singular values are significant

Vtx= diag(σ1-1,…,σr

-1 ,0, …, 0)Utbx= Vdiag(σ1

-1,…,σr-1 ,0, …, 0)Utb

11

,tri

r i r ri i

σ ε σ εσ +

=

= > ≤∑ u bx v

•Pseudoinverse A+=V diag(σ1-1,…,σr

-1 ,0, …, 0) Ut

–A+ is a n×m matrix. –If rank (A) =n then A+=(AtA)-1A

–If A is square A+=A-1

Page 3: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Well Posed problems• Hadamard postulated that for a problem to be “well

posed”1. Solution must exist2. It must be unique3. Small changes to the input data should cause small changes to the

solution

• Many problems in science and computer vision result in “ill-posed” problems.

– Numerically it is common to have condition 3 violated.

• Recall from the SVD 11

,tni

i r ri i

σ ε σ εσ +

=

= > ≤∑ u bx v

•If σs are close to zero small changes in the “data” vector b cause big changes in x.•Converting ill-posed problem to well-posed one is called regularization.

Page 4: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Regularization• Pseudoinverse provides one means of

regularization

• Another is to solve (A+εI)x=b 21

( )n

tii i

i i

σε σ=

=+∑x u b v

•Solution of the regular problem requires minimizing of ||Ax-b||2

•This corresponds to minimizing

||Ax-b||2 + ε||x||2

–Philosophy – pay a “penalty” of O(ε) to ensure solution does not blow up.–In practice we may know that the data has an uncertainty of a certain magnitude … so it makes sense to optimize with this constraint.

•Ill-posed problems are also called “ill-conditioned”

Page 5: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Outline• Gradients/derivatives

– needed in detecting features in images• Derivatives are large where changes occur

– essential for optimization

• Interpolation– Calculating values of a function at a given point

based on known values at other points– Determine error of approximation– Polynomials, splines

• Multiple dimensions

Page 6: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Derivative

• In 1-D

• Taylor series: for a continuous function

0

( ) ( )limh

df f x h f x

dx h→+ −=

( )

2 2

2

2 2

2

( ) ( )2 !

( ) ( ) 12 !

n n

nx x x

n nn

nx x x

df h d f h d ff x h f x h

dx dx n dx

df h d f h d ff x h f x h

dx dx n dx

+ = + + + + +

− = − + + + − +

" "

" "

f(x)

x

•Geometric interpretation –Approximate smooth curveby values of tangent, curvature, etc.

Page 7: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Remarks• Mean value theorem:

– f(b)-f(a)=(b-a)df/dx|c a<c<b

– There is at least one point betweena and b on the curve where the slope matches that of the straight line joiningthe two points x

f(x)

•df/dx=0–represents a minimum, maximum or saddle point of the curve y=f(x)

–d2f/dx2 > 0 minimum, d2f/dx2 < 0 maximum

–d2f/dx2 = 0 saddle point

Page 8: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Finite differences

• Approximate derivatives at points by using values of a function known at certain neighboring points

• Truncate Taylor series and obtain an expression for the derivatives

• Forward differences: use value at the point and forward x x x x

• Backward differences

( ) ( )

( ) ( )

21 2

2

21 2

2

( ) ( )2

( ) ( )2

x x

x x

df h d fh f x h f x O h

dx dx

df h d fh f x f x h O h

dx dx

= + − − +

= − − + +

Page 9: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Finite Differences• Central differences

– Higher order approximation

( )

( )

2 22

2 2

2

( ) ( ) ( ) ( )2

2 2

( ) ( )2

x x x

x

df f x h f x h d f f x f x h h d fO h

dx h dx h dx

df f x h f x hO h

dx h

+ − − −= − + + +

+ − −= +

–However we need data on both sides

–Not possible for data on the edge of an image

–Not possible in time dependent problems (we have data at current time and previous one)

Page 10: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Approximation• Order of the approximation O(h), O(h2)

• Sidedness, one sided, central etc.

• Points around point where derivative is calculated that are involved are called the “stencil” of the approximation.

• Second derivative

( )2 2

22 2

2

2 2

( ) ( ) ( ) ( )0

2 2

( ) 2 ( ) ( )( )

x x

f x h f x h d f f x f x h h d fO h

h dx h dx

d f f x h f x f x hO h

dx h

+ − − −= − − + +

+ − + −= +

23 ( ) 4 ( ) ( 2 )( )

2df f x f x h f x h

O hdx h

− + + − += +

•One sided difference of O(h2)

Page 11: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Polynomial interpolation• Instead of playing with Taylor series we can obtain fits

using polynomial expansions.– 3 points fit a quadratic ax2+bx+c

• Can calculate the 1st and 2nd derivatives

– 4 points fit a cubic, etc.

• Given x1, x2, x3, x4 and values f1, f2, f3, f412 3 2 3

0 01 11 1 1 1 1 12 3 2 3

1 12 22 2 2 2 2 22 3 2 3

2 23 33 3 3 3 3 32 3 2 3

3 34 44 4 4 4 4 4

1 1

1 1

1 1

1 1

a af fx x x x x x

a af fx x x x x x

a af fx x x x x x

a af fx x x x x x

− = =

•Vandermonde system – fast algorithms for solution.•If more data than degree .. Can get a least squares solution.•Matlab functions polyfit, polyval

Page 12: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Remarks• Can use the fitted polynomial to calculate derivatives

• If equation is solved analytically this provides expressions for the derivatives.

• Equation can become quite ill conditioned – especially if equations are not normalized.

ax2+bx+c can also be written as a* (x-x0)2+b* (x-x0) + c*

– Find the polynomial through x0-h, x0, x0+h

20 1

1 02

2 1

1

1 0 0

1

h h a f

a f

h h a f

− − =

–a0=f0, a1=,(f1-f-1)/2h a2=(f-1-2f0+f1)/2h2

–Gives the expected values of the derivatives.

Page 13: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Polynomial interpolation• Results from Algebra

– Polynomial of degree n through n+1 points is unique – Polynomials of degree less than xn is an n dimensional space.– 1,x,x2, …,xn-1 form a basis.

• Any other polynomial can be represented as a combination of these basis elements.

– Other sets of independent polynomials can also form bases.

• To fit a polynomial through x0,…,xn with values f0, …,fn– Use Lagrangian basis lk.

0,

, 0,...,n

ik

i k ii k

x xl k n

x x=≠

−= =−∏

–p(x)=a0l0+a1l1+…+anln.–Then ai=fi

–Many polynomial bases: Chebyshev, Legendre, Laguerre …–Bernstein, Bookstein …

Page 14: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Increasing n• As n increases we can

increase the polynomial degree.

• However the function in between is very poorly interpolated.

• Becomes ill-posed.

• For large n interpolant blows up.

•Idea: –Taylor series provides good local approximations

–Use local approximations

•Splines

Page 15: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Spline interpolation• Piecewise polynomial approximation

– E.g. interpolation in a table– Given xk ,xk+1, fk and fk+1 evaluate f at a point x such that

xk<x<xk+1

11 1

1 1

,( )

0 , otherwise

k kk k k k

k k k k

x x x xf f x x x

x x x xf x+

+ ++ +

− − + ≤ ≤ − −=

•Construct approximations of this type on each subintervalThis method uses Lagrangian interpolants

•Endpoints are called breakpoints

•For higher polynomial degree we need more conditions

• e.g. specify values at points inside the interval [xk<x<xk+1]•Specifying function and derivative values at the end points xk,xk+1 leads to cubic Hermite interpolation

Page 16: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Cubic Spline• Splines – name given to a flexible piece of wood used by

draftsmen to draw curves through points.– Bend wood piece so that it passes through known points and draw a line

through it.– Most commonly used interpolant used is the cubic spline– Provides continuity of the function, 1st and 2nd derivatives at the

breakpoints.– Given n+1 points we have n intervals– Each polynomial has four

unknown coefficients• Specifying function values

provides 2 equations• Two derivative continuity

equations provides two more

{ }

( ) ( )( ) ( )

'' ''1

' '1

, , 1,..., 1

( ) 1, , 1

2, ,

2, ,

i i

i i

i i

i i

x f i n

P x f i n

P x P x i n

P x P x i n

= += = +

= =

= =

"

"

"

•Left with two free conditions. Usually chosen so that second derivatives are zero at ends

Page 17: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Interpolating along a curve

• Curve can be givenas x(s) and y(s)

• Given xi,yi,si

• Can fit splines for x and y

• Can compute tangents, curvature and normal based on this fit

• Things like intensity van vary along the curve. Can also fit I(s)

*

*

**

**

* **

Page 18: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Two and more dimensions• Gradient

• Directional derivative in the direction of a vector n

1 2 ii

f f ff

x y x

∂ ∂ ∂∇ = + =∂ ∂ ∂

e e e

1 2 ii

f f ff n

x y x

∂ ∂ ∂∇ = + =∂ ∂ ∂

n e n e n< < <

•Geometric interpretation–∇f is normal to the surface f(x)=c

– n = ∇f/|∇ f|

•Taylor series( ) 3

3

1( ) ( ) ( ) ( ) (| | )

21

( ) ( ) (| | )2i i j

i i j

f f f f O

f ff f h h h O

x x x

+ = + ∇ + ∇∇ +

∂ ∂ ∂+ = + + +∂ ∂ ∂

x h x h x hh : x h

x h x h

<

Page 19: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Finite differences• Follows a similar pattern. One dimensional

partial derivatives are calculated the same way.

• Multiple dimensional operators are computed using multidimensional stencils.

2 21, , 1 , , 1 1,2

2 2 2

4i j i j i j i j i jf f f f ff ff

x y h+ + − −+ − + +∂ ∂∇ = + =

∂ ∂

Page 20: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Interpolation

• Polynomial interpolation in multiple dimensions

• Pascals triangle

• Least squares

• Move to a local coordinate system

Page 21: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Tensor product splines

• Splines form a local basis.

• Take products of one dimensional basis functions to make a basis in the higher dimension.

Page 22: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

NURBS

• Used for precisely specifying n-d data.

• October 3 Tapas Kanungo, NURBS: Non-Uniform Rational B-Splines

Page 23: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Derivative of a matrix

Page 24: Calculus, finite differences Interpolation, Splines, NURBSusers.umiacs.umd.edu/~ramani/cmsc828d/lecture7.pdf · Polynomial interpolation • Results from Algebra – Polynomial of

Jacobian and Hessian