Download - MIT Numerical PDE

Transcript
Page 1: MIT Numerical PDE

18.336 spring 2009 lecture 1 02/03/09

18.336 Numerical Methods for Partial Differential Equations

Fundamental Concepts

Domain Ω ⊂ Rn with boundary ∂Ω � � PDE in Ω b.c. on Γ ⊂ ∂Ω

PDE = “partial differential equation” b.c. = “boundary conditions”

(if time involved, also i.c. = “initial conditions”)

Def.: An expression of the form

F (Dk u(x), Dk−1 u(x), ..., Du(x), u(x), x) = 0, x ∈ Ω ⊂ Rn (1)

is called kth order PDE,where F : Rnk × Rnk−1 × ... × Rn × R × Ω → R is given,and u : Ω R is the unknown.→A function u satisfying (1) is called solution of the PDE.

⎞ ⎟⎠⎛ ⎜⎝

Du = (ux1 , ..., uxn ) gradient (vector) ux1x1 . . . ux1xn

.D2u = ... . . ..

. Hessian (matrix)

uxnx1 uxnxn· · · . . . . . .

etc.

1

Page 2: MIT Numerical PDE

� � � � � �

� �

Def.: The PDE (1) is called...

(i) linear, if � aα(x)Dα u = f(x)

|α|≤k

homogeneous, if f = 0

(ii) semilinear, if

aα(x)Dα u + F0(Dk−1u, ..., Du, u, x) = 0

|α|≤k

(iii) quasilinear, if

aα(Dk−1u, ..., Du, u, x) Dα u + F0(Dk−1u, ..., Du, u, x) = 0 ·

|α|≤k

(iv) fully nonlinear, if neither (i), (ii) nor (iii).

Def.: An expression of the form

F (Dk u(x), Dk−1 u(x), ..., Du(x), u(x), x) = 0, x ∈ Ω ⊂ Rn

is called kth order system of PDE,

where F : Rmnk × Rmnk−1 × ... × Rmn × Rm × Ω Rm

and u : Ω → Rm , u = (u1, ..., um). →

Typically: # equations = # unknowns , i.e. n = m.

Some examples:

ut + ux = 0 linear advection equation

ut = uxx heat equation

uxx = f(x) Poisson equation (1D)

�2u = f Poisson equation (nD)

ut + cux = Duxx convection diffusion equation

ut + (21 u2)x = 0 ⇔ ut + u ux = 0 Burgers’ equation (quasilinear)

2 2� u = u a semilinear PDE

utt = uxx wave equation (1D)

u 0 1 u wave equation, written as a system =

v 1 0 ·

v utt = vxt = vtx = uxxt x

ut + uux = �uxxx Korteweg-de-Vries equation

�ut + (�u · �)�u = −�p + ν�2�u incompressible Navier-Stokes equation

�� · �u = 0 �

[dynamic-algebraic system]

ht + (uh)x = 0 shallow water equations ut + uux = −ghx [system of hyberbolic conservation laws]

|�u| = 1 Eikonal equation (nonlinear)

2

Page 3: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 4: MIT Numerical PDE

� �

18.336 spring 2009 lecture 2 02/05/09

Well-Posedness

Def.: A PDE is called well-posed (in the sense of Hadamard), if

(1) a solution exists (2) the solution is unique (3) the solution depends continuously on the data

(initial conditions, boundary conditions, right hand side)

Careful: Existence and uniqueness involves boundary conditions

Ex.: uxx + u = 0

a) u(0) = 0, u(π 2 ) = 1 unique solution u(x) = sin(x)⇒

b) u(0) = 0, u(π) = 1 no solution ⇒c) u(0) = 0, u(π) = 0 infinitely many solutions: u(x) = A sin(x)⇒

Continuous dependence depends on considered metric/norm.We typically consider || · ||L∞ , || · ||L2 , || · ||L1 .

Ex.: ⎧⎨⎫⎬ut = uxx heat equation

u(0, t) = u(1, t) = 0 boundary conditions well-posed⎩ ⎭u(x, 0) = u0(x) initial conditions

ut = −uxx backwards heat equation ⎧⎨

⎫⎬no continuous dependence

u(0, t) = u(1, t)u(x, 0) = u0(x) initial conditions

boundary conditions⎩ ⎭ on initial data [later]

Notions of Solutions

Classical solution

kth order PDE ⇒ u ∈ Ck

Ex.: �2 u ∈ C∞u = 0 ⇒

ut + ux = 0 u(x, t) ∈ C1

u(x, 0) ∈ C1 ⇒

Weak solution kth order PDE, but u /∈ Ck .

1

Page 5: MIT Numerical PDE

� �

� �

Ex.: Discontinuous coefficients⎧⎪⎪⎪⎪⎨(b(x)ux)x = 0 u(0) = 0

⎫⎪⎪⎪⎪⎬ 4 1 x x <3 2 ⎪⎪⎪⎪⎩

u(1) = 1

b(x) =

u(x) =⎪⎪⎪⎪⎭

⇒ 2 1 1 x + x ≥1 3 3 21 x <2

2 x ≥ 21

Ex.: Conservation laws

ut + (12 u

2)x = 0 Burgers’ equation

Fourier Methods for Linear IVP

IVP = initial value problem

ut = ux advection equation ut = uxx heat equationut = uxxx Airy’s equationut = uxxxx

w=+∞

e iwx u(w, t)dwa) on whole real axis: u(x, t) = Fourier transformw=−∞

+�∞

+∞

b) periodic case x ∈ [−π, π[: u(x, t) = uk(t)e ikx Fourier series (FS) k=−∞

Here case b). ∂u ∂nu

PDE: (x, t) − ∂xn

(x, t) = 0∂t

duk (t) − (ik)n uk(t) e ikx = 0

dt insert FS:

k=−∞

Since (eikx)k∈Z linearly independent: duk

= (ik)n uk(t) ODE for each Fourier coefficient dt

2

Image by MIT OpenCourseWare.

Page 6: MIT Numerical PDE

� �� �

Solution: uk(t) = e(ik)nt uk(0) � π u0(x)e−ikxdxFourier coefficient of initial conditions: uk(0)= 21 π −π

+∞

u(x, t) = uk(0)e ikx e(ik)nt ⇒ k=−∞

n = 1: u(x, t) = uk(0)e ik(x+t) all waves travel to left with velocity 1 k

n = 2: u(x, t) = uk(0)e ikx e−k2t frequency k decays with e−k2t

k

n = 3: u(x, t) = uk(0)e ik(x−k2t) frequency k travels to rightk with velocity k2 dispersion� →

n = 4: u(x, t) = uk(0)e ikx e k4t all frequencies are amplified

k unstable→

Message:

For linear PDE IVP, study behavior of waves eikx .The ansatz u(x, t) = e−iwteikx yields a dispersion relation of w to k.

The wave eikx is transformed by the growth factor e−iw(k)t .

Ex.:

wave equation: utt = c2uxx w = ±ck conservative |e±ickt| = 1 heat equation: ut = duxx w = −idk2 dissipative e−dk2t 0 conv.-diffusion: ut = cux +duxx w = −ck−idk2 dissipative

|eickte−

|dk

→2t 0|

ik2t

| → Schrodinger: iut = uxx w = −k2 dispersive |e

e−ik3

|t

= 1 Airy equation: ut = uxxx w = k3 dispersive | | = 1

3

Page 7: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 8: MIT Numerical PDE

����

18.336 spring 2009 lecture 3 02/10/09

Four Important Linear PDE

Laplace/Poisson equation

−�2u = f in Ω u = g on Γ1 Dirichlet boundary condition←

∂u ∂n

= h on Γ2 ← Neumann boundary condition

Γ1 ∪Γ2 = ∂Ω

⎧ ⎪⎪⎨ ⎪⎪⎩

f ≡ 0 Laplace equation→ �2u = 0u = “harmonic function”

Physical example:

Heat equation: ut −�2u = f

source

stationary (t →∞) : ut = 0 ⇒ −�2u = f

Dirichlet: prescribe u = g ∂u

Neumann: prescribe flux = h ∂n

1

Image by MIT OpenCourseWare.

Page 9: MIT Numerical PDE

��

� � �

Fundamental solution of Laplace equation:

Ω = Rn

no boundary conditions

Radially symmetric solution in Rn \{0} : ∂r 2xi xi

⎡ ⎤� 1 = =n

2 r = |x| = xi

2 ⎢⎢⎣⎥⎥⎦

∂xi 2|x| r 1 ∂r · r − xi ∂xi =

1∂2 2r xii=1

r −=

2 r2 r3∂xi u(x) = v(r)

∂r ⇒ uxi = v�(r)∂xi �2

∂2r xi 2 2∂r 1 xi

+ v�(r) ·= v��(r) + v�(r) 2

= v��(r) r −⇒ uxixi r2 r3∂xi ∂xi

n

⇒ �2u = uxixi = v��(r) + v�(r) · n − 1r

i=1

Hence:

�2u = 0 ⇐⇒ v��(r) + n −

r 1 v�(r) = 0

v�=0�(log v�(r))� =

v��(r)=

1 − n ⇐⇒ v�(r) r

⇐⇒ log v�(r) = (1 − n) log r + log b

v�(r) = b ·⎧ ⎪⎨

1−nr⇐⇒ ⎫ ⎪⎬br + c n = 1b log r + c n = 2⇐⇒ v(r) =⎪⎩ b rn−2

+ c n ≥ 3 ⎪⎭

Def.: The function⎧⎨⎫⎬−1 |x| n = 1

2 1−

1 log |

2π = 0, α(n) = volume of unit ball in Rn)n = 2Φ(x) = (xx|⎩ ⎭|x|n

1 −2 n ≥ 3

n(n−2)α(n) ·

is called fundamental solution of the Laplace equation.

Rem.: In the sense of distributions, Φ is the solution to 2Φ(x) = δ(x)−� ����

Dirac delta

2

Page 10: MIT Numerical PDE

� �

� �

� �

Poisson equation:

Given f : Rn R,� →

u(x) = Φ(x − y)f(y) dy (convolution)Rn

solves −�2u(x) = f(x).

Motivation:

�2u(x) = R2

−�x 2Φ(x − y)f(y) dy =

R2

δ(x − y)f(y) dy = f(x).

Φ is a Green’s function for the Poisson equation on Rn .

Properties of harmonic functions:

Mean value property

average average �↓ �↓ u harmonic u(x) = −∂B(x,r) u ds u(x) = −B(x,r) u dy ⇐⇒ ⇐⇒ for any ball B(x, r) = {y : ||y − x|| ≤ r}.Implication: u � harmonic ⇒ u ∈ C∞

Proof: u(x) = Rn χB(0,r)(x − y)u(y) dy

u ∈ Ck convolution u ∈ Ck+1 �= ⇒

Maximum principle

Domain Ω ⊂ Rn bounded.

(i) u harmonic max u = max u (weak MP) ⇒ Ω ∂Ω

(ii) Ω connected; u harmonic If ∃ x0 ∈ Ω : u(x0) = max u, then u ≡ constant (strong MP)

Ω

Implications

• u → −u ⇒ max → min • uniqueness of solution of Poisson equation with Dirichlet

boundary conditions2−� u = f in Ωu = g on ∂Ω

Proof: Let u1, u2 be two solutions.

Then w = u1 − u2 satisfies2w = 0 in Ω max principle �

= w ≡ 0 u1 ≡ u2 � w = 0 on ∂Ω

⇒ ⇒

3

Page 11: MIT Numerical PDE

� �

� � � � � �

Pure Neumann Boundary Condition: 2−�

∂u u = f in Ω

= h on ∂Ω∂h � �

has infinitely many solutions (u u + c), if − fdx = − h dS. → Ω ∂ΩOtherwise no solution.

Compatibility Condition:

∂f − Ω f dx =

Ω

�2u dx = Ω div�f dx =

∂Ω

�f ·n dS = ∂Ω ∂n

dS = ∂Ω

h dS.

4

Page 12: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 13: MIT Numerical PDE

� �

� �

18.336 spring 2009 lecture 4 02/12/09

Heat equation 2ut = � u

Physics:

Fick’s law: flux F� = −a�u � � d

mass balance: u dx = −b F n dS = −b divF dx dt V ∂V

· V

⇒ ut = −b div(−a�u) = c �2u↑

simple: c = 1

Fundamental Solution

4tΦ(x, t) = 1

e− |x|2

(4πt)n/2

ut = �2u in Rn × ] 0, ∞ [solves

u(x, 0) = δ(x) t = 0

Superposition: Φ ∈ C∞

u(x, t) = Rn

Φ(x − y, t)u0(y) dy ⇓

solvesut = �2u in Rn × ] 0, ∞ [

u ∈ C∞

u(x, 0) = u0(x)

Maximum Principle

Ω ∈ Rn bounded

ΩT := Ω × [0, T ], ∂ΩT = (Ω × {0}) ∪ (∂Ω × ] 0, T [ )

1

Image by MIT OpenCourseWare.

Page 14: MIT Numerical PDE

� �

� �

� �

� �

If u is the solution to ⎧ ⎫ ⎨ ut = �2u in ΩT ⎬⎩ u = u0 on Ω × {0} ⎭u = g on ∂Ω × ] 0, T [

then

(i) max u = max u (weak) ΩT ∂ΩT

(ii) For Ω is connected: If ∃(x0, t0) ∈ ΩT : u(x0, t0) = max u, then u = constant (strong)

ΩT

Implications:

max min• → • uniqueness (see Poisson equation)

infinite speed of propagation:⎧• ⎫ ⎨ ut = �2u in ΩT ⎬⎩ u = 0 on ∂Ω × ] 0, T [ ⎭u = g on Ω × {0}strong max principle

g ≥ 0 =↓⇒ u > 0 in ΩT .

Inhomogenous Case:

ut −�2u = f in Rn × ] 0, ∞ [u = 0 on Rn× {0}

t

solution: u(x, t) = Φ(x − y, t − s)f(y, s) dy ds 0 Rn

“Duhamel’s Principle” (variation of constants):superposition of solutions starting at s with initial conditions f(s).

Transport equation

ut + b · �u = 0 in Rn × ] 0, ∞ [ u = u0 on Rn b = direction vector (field) × {0}

solution: u(x, t) = u0(x − tb).check: ut = −b · �u0(x − tb) = −b · �u�

Inhomogenous Case:

ut + b · �u = f in Rn × ] 0, ∞ [u = u0 on Rn × {0}

Duhamel’s principle yields the solution: t

u(x, t) = u0(x − tb) + f(x + (s − t)b, s) ds 0

2

Page 15: MIT Numerical PDE

� �

Wave equation 2u = futt −� ����� source �

utt − uxx = 0 in Rn × ] 0, ∞ [1D

u = g, ut = h on Rn × {0}

0 = utt − uxx = (∂t + ∂x)(∂t − ∂x)uDefine: v(x, t) := (∂t − ∂x)u(x, t)

transport ⇒ vt + vx = 0 ⇒ v(x, t) = a(x − t)Thus: ut − ux = a(x − t)(inhomogenous transport) [b = −1, f(x, t) = a(x − t)]

t

⇒ u(x, t) = 0�

a(x + (t − s) − s) ds + b(x + t)

x+t

= 1 a(y) dy + b(x + t)2

x−t

initial conditions: b = g, a = h − gx

x+t

u(x, t) = 1 (g(x+t)+g(x−t))+ 1 h(y) dy2 2

⇒ x−t

d’Alembert’s formula. Ex.:

Higher space dimensions

utt −�2u = 0 in Rn × ] 0, ∞ [ u = g, ut �

= h on Rn × {0}

u(x, t) = th(y) + g(y) + �g(y) (y − x) dS(y)3D · ∂B(x,t)

Kirchhoff’s formula.

Obtain from 3D solution by projecting to 2Du(x1, x2, x3, t) := u(x1, x2, t)

2D

yields

u(x, t) = 1 tg(y) + t2h(y) + t�g(y) · (y − x) dy

2 B(x,t) (t2 − |y − x|2)1/2

Poisson’s formula.

3

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 16: MIT Numerical PDE

Ex.: g ≡ 0, h = δ. � � 1 �

u(x, t) = t δ(y) dy = 4πt |x| = t

∂B(x,t) 0 else3D

sharp front

t2 1 t2

2D u(x, t) = 12 δ(y) dy = ⎧ B(x,t) (t

2 − |y − x|2)1/2 · ⎫ 2πt2

· (t2 − |x|2)1/2 ⎨ 1 ⎬ x ≤ t

= ⎩ 2π(t2 − |x|2)1/2 ⎭0 else

signal never vanishes

4

Page 17: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 18: MIT Numerical PDE

� � �

� � �

18.336 spring 2009 lecture 5 02/19/09

Finite Difference (FD) Approximation

Consider u ∈ C l .Goal: Approximate derivative by finitely many function values:

∂ku m�

∂xk (x0) ≈ aiu(xi) (k ≤ l)

i=0

Vector of coefficients a = (a0, a1, , am) is called FD stencil.· · · How to get stencil?

Taylor expansion

In 1D: u(x) = u(x0) + ux(x0) (x − x0) + 1 (x0) (x − x0)2 ·

2 uxx · +1

6 uxxx(x0) · (x − x0)3 + O(|x − x0|4)

Name xi = xi − x0

= ⇒ u(xi) = u(x0) + ux(x0�) · xi + � 12 uxx(x0) · x2

i� + 1

6 uxxx(� x0) · x3

i + O(|xi|4) m m m

= aiu(xi) = u(x0) ai + ux(x0) aixi⇒ · · i=0 i=0 i=0

m

+uxx(x0) 1 aixi 2 + O(h3) where xi ≤ h ∀i.·

2 i=0

Match coefficients: m �� � � ��

aiu(xi) ≈ ux(x0) ai = 0, aixi = 1 aixi 2 small⇒

i=0 i im� � � � �� �

aiu(xi) ≈ uxx(x0) ai = 0, aixi = 0, aixi 2 = 2 aixi

3 small⇒ i=0 i i i

etc.

1

Image by MIT OpenCourseWare.

Page 19: MIT Numerical PDE

� � � � �

� � � � �

� � � � � �

1

Vandermonde matrix ⎤⎡ ⎤⎡1 1· · · · · · 1 1 · · · 0 x1 xm· · · x0 xm· · · · · ·

2 2¯x0 xm· · · · · · =V = V (x0, x1, , xm) = · · ·

⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎦

⎢⎢⎢⎣ . . . .. . . ... .. . . . . . .0 xk xk

k k¯ 1 · · · m¯x0 xm· · · · · · Constraints for stencil:

V a = b· linear system ⎤⎡

a0 1 1 0

1⎢⎣

⎥⎦...· · ·

k = 1 : aiu(xi) ≈ ux(x0) : =·x0 ¯· · · xm

am ⎤⎡⎤⎡⎤⎡ 1 1 a0 0· · · ⎢⎣ ...

⎥⎦=⎣ 0⎣ ⎦ ⎦x0 ¯k = 2 : aiu(xi) ≈ uxx(x0) : · · · xm · x2

0 x2 m am 2· · ·

If m = k = In general one unique stencil a⇒If m > k = Multiple stencils ⇒

Can add additional criteria, e.g. require higher order.

Ex.: k = 1, m = 1

= x0 x1

· a1 1

x0 = 0, x1 = h = a01 , a1 = 1

� 1 1 0a0

−= ⇒ h h

x0 = 0, x1 = −h = ⇒ a0 = h 1 , a1 = −

h 1

Ex.: k = 1, m = 2 ⎤ x = (x0, x0 + h, x0 − h)

a0

⎡1

1 1 1 0 = = a1 − a2

h⎣ ⎦=a1· ⇒0 h −h 1 a0 = −a1 − a2a2

One-parameter family of stencils Additional criterion: second order accuracy

2

⎥⎥⎥⎦

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 20: MIT Numerical PDE

⎤⎡⎤⎡⎤⎡ 1 1 1 a0 0 ⎣ ⎣⎦· a1

⎦=⎣ 1⎦ 1 10 h −h h2

a0 = 0, a1 = −= = , a22h⇒

2h h20 a2 0

Ex.: k = 2, m = 2a0 = −a1 − a2⎤⎡⎤⎡⎤⎡

1 1 1 a0 0 a1 =

x2 1

(x1 − x2)· ⎣ 0 x1 x2⎣⎦· a1

⎦=⎣ 0⎦=⇒0 x2

1 x22 a2 2

a2 = x2 2

(x2 − x1)· Equidistant: x = (x0, x0 + h, x0 − h)

2 1 =a0 = −h2 , a1 = a2 h2

Higher space dimensions

2D x�i = (xi, yi)

⎤⎡⎤⎡ a01 1 1· · · · · · 0

·

⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎥⎥⎦

a1 . . ...... .

⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎦

⎥⎥⎥⎥⎥⎥⎦

←stencil �a for uxx(x0)

0 x1 xm 002

· · · · · ·0 y1 ym· · · · · ·

=2 2x0 x1 · · · · · · m

0 x1y1 xmym· · · · · · 00 y2 y2 01 · · · · · · m am

3

⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩

⎤⎡⎢⎢⎢⎢⎢⎢⎣

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 21: MIT Numerical PDE

� � � � � �

0 0

3D ⎤⎡ a01 1 1 0· · · · · · · · · · · ·a10 xi· · · · · · · · · · · · 0

⎤⎡ ⎤⎡⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

·

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

. .0 yi· · · · · · · · · · · ·0 zi· · · · · · · · · · · ·

0.... 0

20 x 2 stencil for �.· · · · · · i · · · · · · ←.. =y2 2 uxx + uyy + uzz i· · · · · · · · · · · · . . z2

i 2.· · · · · · · · · · · ·..0 · · · · · · xi yi

0 · · · · · · xi zi

0 · · · · · · yi zi

0· · · · · · .0.· · · · · · . . 0· · · · · ·

am

2Ex.: u(x0)2D

11211 h2 , 0, h2 , 0

h2 , 0, h2 , 0, 4 ,2h 1111 2�a = +− − −=

h2 , h2 , h2 , h2 ,h2 h2

�a = ( exercise )· · · · · ·

2 ( ) = u x0

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

4

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 22: MIT Numerical PDE

• � � � �

� � � �

1

Poisson Equation ⎧ ⎫ ⎨ −uxx = f(x) in ]0, 1[ ⎬1D ⎩

u(0) = a ⎭ ← Dirichlet boundary condition

ux(1) = c Neumann boundary condition ←

Discretize on regular grid �x = (0, h, 2h, , nh, 1), where h = · · ·n+1

(xi) = −u(xi−1) + 2u(xi) − u(xi+1)

Interior: f(xi) = −uxx + O(h2)h2⎛ ⎞ � � ui−1

= −h1 2 , h

2 2 , −h

1 2 · ⎝ ui ⎠ + O(h2)

ui+1

Dirichlet boundary condition: u0 = u(x0) = a (exact)

Neumann boundary condition:

Naive choice: c = ux(1) un+1 − un 1 1 un = + O(h) = , +O(h)

h −

h h ·

un+1

O(h) on a single cell = Could preserve O(h2) globally, or drop ⇒ accuracy to O(h). Here the bad event happens.

Second order approximation:•

c = ux(1) = u(xn+2) − u(xn)

+ O(h2)2h

Obtain un+2 by −un + 2un+1 − un+2

= f(1)h2

1 1 un h = ⇒ −

h, h ·

un+1 = c + �2

f�� (1) � right hand side correction yields 2nd order

• Alternative: ⎛ ⎞� � un−1

−21 h , h

2 , −23 h · ⎝ un ⎠ = c

un+1

2nd order one-sided stencil (check by V a = b).·

5

Image by MIT OpenCourseWare.

Page 23: MIT Numerical PDE

� �� � � �� � � �� �

� � � �

Discretization generates linear system:

1 u0 a ⎤⎡

1 2 1 f(x1)− − u1h2 h2 h2

..

⎤⎡⎤⎡ ⎢⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎥⎦

·

⎢⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎥⎦

=

⎢⎢⎢⎢⎢⎢⎢⎣

..

...f(xn)

⎥⎥⎥⎥⎥⎥⎥⎦

. . .. . . .....

. . .(∗). . .. . .. . .

1 2 1−h2 h2 −h2 un

1 1 h f(1)c +−h h un+1 2

A �u �b

Second order approximation (try it yourself!)

Big Question:

How to solve sparse linear systems A �u = �b?· lecture 11. →

Rem.: (∗) (∗∗)⇔⎡ ⎤⎡⎤ 1 h2 a2 1

h2 f(x1) + −h2 u1

=

⎢⎢⎢⎢⎢⎢⎢⎢⎣

1 h2 h2

2 1⎢⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎥⎦

·

⎢⎢⎢⎢⎢⎢⎣

f(x2) ...

− − ⎥⎥⎥⎥⎥⎥⎦

h2 . . . .... .

. . .. . .. . .(∗∗).. . . ... . .. . .

1 2 h2 h2

1 h2

. f(xn−1)− −f(1)1 1 un f(xn)+

hc +h

1 ↑

from Neumann boundary conditions h2

h 2

un+1 = un + h c + h 2 f(1)

=⇒ − h1 2 un+1 =

h1 2 un + 1 c + h f(1)

h 2

Advantages: fewer equations • • matrix symmetric

⎤⎡ ⎥⎥⎥⎥⎥⎥⎥⎥⎦

6

Page 24: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 25: MIT Numerical PDE

� �� � � �� � � �� � ����

� �

� �

18.336 spring 2009 lecture 6 02/24/09

General Linear Second Order Equation⎧⎪⎪⎪⎨⎪⎪⎪⎩

⎫⎪⎪⎪⎬a(x)uxx(x) + b(x)ux(x) c(x)u(x)

diffusion advection growth/decay source

u(0) = α u(1) = β

= f(x) x ∈ ]0, 1[+

⎪⎪⎪⎭

Approximation: ui−1 − 2ui + ui+1 ui+1 − ui−1

ai + bi + ciui = fih2 2h

where ai = a(xi), bi = b(xi), ci = c(xi).

Linear system: A �u = f�·⎡ ⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤h2 a1 + hb1c1 − 2a1 2 a2 − hb

2 2 h2c2 − 2a2 a2 + hb

2 2

. . .. . .. . .

⎥⎥⎥⎥⎥⎥⎥⎥⎦

A = h1 2 . . .. . .. . .

an−1 − hbn

2 −1 h2cn−1 − 2an−1 an−1 + hbn

2 −1

hbn h2cn − 2anan − 2 ⎤⎡

αf1 − ha12 −

2bh 1

f2 .. .

⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎦f�=

fn−1

+ βfn − ah

n 2 2

bn h

Potential Problems: A non-symmetric•

• If |a(x)| � |b(x)|, instabilities possible due to central differences. Often better approximations possible.

1

Image by MIT OpenCourseWare.

Page 26: MIT Numerical PDE

Ex.: Heat equation in rod with variable conductivity

(κ(x)ux)x = f(x) (∗)

⇔ κ(x)uxx + κx(x)ux = f(x) (∗∗) Can discretize (∗∗) as before. Suboptimal results. Better: discretize (∗) directly (in line with physics)

Use cell centers

1 2)ux(xi+

2) ≈ κi+�

ui+1 − uiκ(xi+ 1 1 ·

h2

ui+1 − ui ui − ui−1(κux)x(xi) ≈ 1 κi+h − κi−1 1⇒ · ·

h h2 2

h 1 (κi− 1

2ui−1 − (κi− κ+ i+

21 )ui + κi+ ui+1)= 1 1

2 2

−(κ +κ1 3) κ322 2

1h2

⎢⎢⎢⎢⎢⎢⎣

κ −(κ )25κ+

23 κ3 5

2 2. . .A = . . ..

3

. .

2κn− −(κ 1

2+κn− ) κ

32

32n−

κn− 12

12

1

n−+κn+−(κ )

2n−

Symmetric matrix, −A positive definite.Great for linear solvers −→ CG (conjugate gradient method).

2D/3D

� · (κ�u) = f||

2D: (κ(x)ux)x + (κ(x)uy)y ←− 2× 1D

⎤⎡ ⎥⎥⎥⎥⎥⎥⎦

2

Image by MIT OpenCourseWare.

Page 27: MIT Numerical PDE

� �

Errors, Consistency, Stability

Presentation for Poisson equation, but results transfer to any linear finite difference scheme for linear PDE.

u��(x) = f(x) � A U = F·↑

vector of approximate function values Ui ⎡ ⎤ u(x1)

true solution values: U = ⎣⎢ ... ⎦⎥

u(xn)

Local Truncation Error (LTE)

Plug true solution u(x) into FD scheme:

τi = 1 (u(xi−1) − 2u(xi) + u(xi+1)) − f(xi)h2

= u��(xi) + 1 u����(xi)h2 + O(h4) − f(xi)12

1= u����(xi)h2 + O(h4)

12 ⎡ ⎤ τ1

τ = ⎢⎣ ...

⎥⎦ = A U − F· τn

AU = F + τ⇒

Global Truncation Error (GTE)

Error vector: E : U − UAU = F AU = F + τ

⇒ AE = −τ and E = 0 at boundaries

−e��(x) = −τ(x) ]0, 1[Discretization of

e(0) = 0 = e(1)

T (x) ≈ 1 u����(x)h2 12

⇒ e(x) ≈ − 1 u��(x)h2 + 1 h2(u��(0) + x(u��(1) − u��(0))12 12

Message: Global error order = local error order if method stable.

Stability

Mesh size h : Ah Eh = −τh · ⇒ Eh = −(Ah)−1 · τ h

τh⇒ ||Eh|| = ||(Ah)−1 · || ≤ ||(Ah)−1|| · ||�

τ�� h||�

O(h2)(LTE)

3

Page 28: MIT Numerical PDE

Stability: ||(Ah)−1|| ≤ C ∀ h < h0

Inverse FD operators uniformly bounded.

|| ∀ h < h0.⇒ ||Eh|| ≤ C · ||τ h

Consistency

0||τh|| → 0 as h →LTE goes to 0 with mesh size

Convergence

0||Eh|| → 0 as h →GTE goes to 0 with mesh size

Lax Equivalence Theorem

consistency + stability convergence ⇐⇒

Proof: (only “= ” here) ⇒

||Eh|| ≤ ||(Ah)−1|| · ||τh|| ≤ C · ||τh|| −→ 0 as h → 0↑ ↑

stability consistency

Also: O(hP ) LTE + stability = O(hP ) GTE ⇒

Stability for Poisson Equation

Consider 2-norm �� � 1

i||U ||2 = Ui 2 2

||A||2 = ρ(A) = maxp|λp| largest eigenvalue

⇒ ||A−1||2 = ρ(A−1) = maxp|λp−1| = (minp|λp|)−1

Stable, if eigenvalues of Ah bounded away from 0 as h 0→In general, difficult to show. But for Poisson equation with Dirichlet boundary conditions, it is known that

λp = 2 (cos(pπh) − 1)h2

⇒ λ1 = h2 2 (−2

1 π2h2 + O(h4)) = −π2 + O(h2) Stable �

Hence: ||Eh 1||2 ≤ ||(Ah)−1||2 · ||τh||2 ≈ π2 ||τ h||2.

4

Page 29: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 30: MIT Numerical PDE

18.336 spring 2009 lecture 7 02/26/09

Spectral Methods

[ Reference: Trefethen, Spectral Methods in MATLAB, SIAM 2000 ]

Classical Methods: error = O(hp), p = 1, 2, 3... fixedSpectral: error = O(hp) ∀ pError decays (with h) faster than any polynomial order

1 e.g. error = O(hh ) exponential decayOnly true if solution smooth: u ∈ C∞

Otherwise: u ∈ Cp, u /∈ Cp+1 error = O(hp)⇒

Message 1:

Spectral methods have a restricted area of application: Linear problems on simple domains with simple boundary conditions and smooth solution. [often times subproblems] But for those, they are awesome.

Two Cases:

1. Periodic: Ω = [0, 2π], where “0 = 2π” u(x + 2π) = u(x)

Use trigonometric functions: u(x) = cke ikx �

k

2. Non-Periodic: Ω =] − 1, 1[

Use polynomials on Chebyshev points (non-equidistant).

Message 2:

FD/FE/FV methods are local. Spectral methods are global.

1

Page 31: MIT Numerical PDE

� � � �

Periodic Domains

Ω = [0, 2π], “0 = 2π”, u(x) = u(x + 2π) Uniform grid

Task: Approximate u�(xi) ≈ αijuj

j

O(h) : u�(xi) ≈ ui+1 − ui

h

O(h2) : u�(xi) ≈ ui+1 − ui−1

(3 point)2h

O(h4) : u�(xi) ≈−ui+2 + 8ui+1 − 8ui−1 + ui−2

(5 point) 12h

O(h6) : u�(xi) ≈ ui+3 − 9ui+2 + 45ui+1 − 45ui−1 + 9ui−2 − ui−3

(7 point) 60h

Limit: use all points; expect O(hN ) = O(hh 1 )

1 h 1 2h u�(xi) ≈

2 cot

2 · (ui+1 − ui−1) −

2 cot

2 · (ui+2 − ui−2)

1�

3h+ cot (ui+3 − ui−3) − . . .

2 2 ·

Limit N →∞ “infinite grid”1 1 1

u�(xi) ≈ h · (ui+1 − ui−1) −

2h · (ui+2 − ui−2) +

3h · (ui+3 − ui−3) − . . .

Matrix Notation:

�u = (u1, u2, . . . , uN )T

! w� = (w1, w2, . . . , wN )

T ≈ (u�(x1), . . . , u�(xN ))

T

w� = D �u·

⎡5 point stencil:

0 2 1 1 2 3 −

12 12 −3

−2 . . . . . . 1 . . . 3 12 1 . . . . . . . .. . . . 12

. . . . .

⎤⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

D = banded sparse matrix. . . . .. . . . .. . . .. . . . 1 . . . .

12−

1 . . . . . . 2 . . . 12 3

− 2 1 1 2 03 −

12 12 −3

2

Page 32: MIT Numerical PDE

� �

� �

⎤Spectral N = 6 :

0 α1 −α2 α3 −α4 α5

−α1 0 α1 −α2 α3 −α4

α2 −α1 0 α1 −α2 α3

⎡ ⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎦

full matrixD6 = −α3 α2 −α1 0 α1 −α2

α4 −α3 α2 −α1 0 α1

−α5 α4 −α3 α2 −α1 0 � �jh αj =

21 cot

2

How to obtain spectral differentiation matrices?

Fourier Basis:

u periodic on [0, 2π] = Ω

u(x) = uke ikx (Fourier series, Ω bounded) ⇒

u only known at grid points xj = jh

Zk∈

2π π N where h = = [here: N even]

N ⇒

h 2

2N

1uke ikxj discrete Fourier series uj = ⇒

2πk=−

2N +1 ⎡ ⎤

physical space : discrete, bounded

Fourier space : bounded, discrete

N

⎣ ⎦

N N uj e

−ikxj ∀k = − + 1, . . . , uk = h 2 2

j=1

Discrete Fourier Transform (DFT) −→ Use FFT to do in O(N log N)! NN

1 2

uj = uke 21ikxj = � uke ikxj

2π 2πN 2

N 2

k=− +1 k=−

2N

1 1where � ck , u−N =uN 22

+ c + + c += c cN N N N+1 −1· · ·2 2− −

2 2 2 2

k=−2 N

3

Page 33: MIT Numerical PDE

Finite Fourier sum due to grid aliasing:

sin(x) = sin(5x) on the grid xj = 2π j N

Nyquist sampling theorem

At grid points:N

2

k −=

1 � uke ikxju(xj ) =

2 N

Define interpolant:N

2

k −=

1 � uke ikx p(x) = Band-Limited Interpolant (BLI)

2 N

N�

j=1

N

2

Trigonometric polynomial of degree ≤ 2

Basis function:� � 1 j = 0 (mod N)

BLI for δj 0 j = 0 (mod N)

DFT uj = δj −→ uk = h δj e

−ikxj = h ∀k

N

sin(πx )hh � e ikx p(x) = =: SN (x)= =⇒ · · · 2

hπ tan(x

2 )2πk=−

2 N

“periodic sinc function”

4

Page 34: MIT Numerical PDE

Arbitrary grid values:

(u1, u2, . . . , uN ) N N

uj = umδj−m ⇒ p(x) = umSN (x − xm)

Differentiation:

=1 =1m m

Differentiate BLI p(x) −→ p�(x)

S � (xj ) = 0 j = 0 (mod N)

N 1 (−1)j cot( jh 2 2 ) j = 0 (mod N) ⎡. . . .. . . .. . . . ⎢⎢⎢⎣

1 1 1 2 cot(h

2 ) 0 2 cot(h

2 ) −2 cot(22 hDN ) . . .. . . −=⇒

. . . .. . . .. . . . ⎤⎡ . . . .. . . . . −1 csc(2h )

2 2 . .. . 1

2 csc2(h

2 ) . .

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

(2) 2. . π2 1 .DN = DN = .. .−3h −

6 . .. . 1

2 csc2(h

2 ) . .

−1 csc2(2h ) . . .

2 2 . .. .. .

5

⎤⎥⎥⎥⎦

Page 35: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 36: MIT Numerical PDE

� �

18.336 spring 2009 lecture 9 03/05/09

Non-periodic Domains

So use algebraic polynomials p(x) = a0 + a1x + + aN xN · · ·

Problem: Runge phenomenon on equidistant grids

p(x)���−→u(x) as N →∞

jRemedy: Chebyshev points xj = cos π

N

PN (x) u(x) as N →∞→

1

Page 37: MIT Numerical PDE

� � �

��

Spectral Differentiation:

Given (u0, u1, . . . , uN ) � interpolating polynomial N

k=� j (x − xk) p(x) = uj

k=� j (xj − xk)j=0 ⎧ ⎪⎪⎨

⎫ ⎪⎪⎬ai

i = j aj (xi − xj)

�↓p�(x) → p�(xj ) → Dij 1=⎪⎪⎩

⎪⎪⎭i = j xj − xk

k=j

aj = k=� j (xj − xk)

Chebyshev Differentiation Matrix:

DN =

2N2 + 1 6

2 (−1)j

1 − xj

1 2 (−1)N

(−1)i+j

xi − xj

−1 2

(−1)i

1 − xi

(−1)i+j

xi − xj

−xj

2(1 − x2 j )

1 2

(−1)N+i

1 + xi

−1 2 (−1)N −2

(−1)N+j

1 + xj −

2N2 + 1 6

where xj = cos( jπ ), j = 0, . . . , NN

w� = DN u(�x) ≈ u�(�x) with spectral accuracy.· w� = DN

2 u(�x) ≈ u��(�x) with spectral accuracy.· etc.

2

Page 38: MIT Numerical PDE

Chebyshev Differentiation using FFT:

1. Given u0, . . . , uN at xj = cos( jπ ).N

Extend: U� = (u0, u1, . . . , uN , uN−1, . . . , u1) 2N

π � 2. FFT: Uk = e−ikθj Uj , k = −N + 1, . . . , N

N j=1

3. Wk = ikUk, WN = 0 (first derivative) N

1 � 4. Inverse FFT: Wj = e ikθj Wk, j = 1, . . . , 2N

2π ⎧ k=−N+1⎪⎪⎪⎪ Wj⎨ wj �

1 − xj 2 , j = 1, . . . , N − 1= −

5. N N⎪⎪⎪⎪ 1 � 1 � ⎩ w0 =2π

� n 2 un, wN =2π

� (−1)n+1 n 2 un

n=0 n=0

p(x) = P (θ), x = cos θ N

p(x) = αnTn(x) Visualization for N = 10: n=0

N

P (θ) = αn cos(nθ) n=0

Tn(x) = Chebyshev polynomial

Tn+1(x) = 2xTn(x) − Tn−1(x) N

− nαn sin(nθ)P �(θ) n=0p�(x) = = dx − sin θdθN

nαn sin(nθ)n=0= √

1 − x2

3

Page 39: MIT Numerical PDE

� �

� �� �

� �

� �

� �

Boundary value problems

uxx = e4x, x ∈] − 1, 1[ Poisson equation with homogeneous Ex.:

u(±1) = 0 Dirichlet boundary conditions

Chebyshev differentiation matrix DN .

Remove boundary points: ⎤ ⎥⎥⎥⎥⎥⎦

⎡⎤⎡⎤ ⎥⎥⎥⎥⎥⎦

⎡ v0[= 0] w0

w1 ⎫⎬⎭

⎢⎢⎢⎢⎢⎣

⎢⎢⎢⎢⎢⎣

⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎦

v1

D2. . p13.mInterior points. .= N ·. .wN−1 vN−1

vN [= 0] wN

D2 N

Linear system:

D2 �u = f�n ·

where �u = (u1, . . . , uN−1), f� = (e4x1 , . . . , e4xN −1 )

Nonlinear Problem

uxx = eu, x ∈] − 1, 1[Ex.:

u(±1) = 0

initial guess �

(2)Need to iterate: �u(0) = 0, �u(1), �u , . . .

D2 �u(k+1) = exp(�u(k)) fixed point iterationN ←

Can also use Newton iteration...

Eigenvalue Problem

p14.m

uxx = λu, x ∈] − 1, 1[Ex.:

u(±1) = 0

Find eigenvalues and eigenvectors of matrix D2 N

Matlab: >> [V,L] = eig(D2)

Higher Space Dimensions

p15.m

Ex.: uxx + uyy = f(x, y) Ω =] − 1, 1[2

f(x, y) = 10 sin(8x(y − 1))u = 0 ∂Ω

Tensor product grid: (xi, yj ) = (cos( iπ ), cos( jπ ))N N

4

Page 40: MIT Numerical PDE

p16.m

Matrix approach:

Matlab: kron, ⊗

LN = I ⊗ D2 D2 N +

˜N ⊗ I

>> L = kron(I,D20)+kron(d2,I)

Linear system: LN �u = f�·

spectral classical FD

Helmholtz Equation

Wave equation with source

−vtt + vxx + vyy = eiktf(x, y)

Ansatz: v(x, y, t) = eiktu(x, y)

Helmholtz equation: ⇒

uxx + uyy + k2u = f(x, y) Ω =] − 1, 1[2

u = 0 ∂Ω

5

p17.m

Image by MIT OpenCourseWare.

Page 41: MIT Numerical PDE

Fourier Methods

So far spectral on grids (pseudospectral). Can also work with Fourier coefficients directly.

Ex.: Poisson equation

−(uxx + uyy) = f(x, y) Ω = [0, 2π]2

periodic boundary conditions

f(x, y) = f kle i(kx+ly)

k,l∈Z

u(x, y) = ukle i(kx+ly)

k,l∈Z

uxx(x, y) = ukle i(kx+ly) (−k2)· k,l

−�2u(x, y) = ukl(k2 + l2)e i(kx+ly) =

! f(x, y)

k,l

f kl

ukl = k2 + l2

∀ (k, l) = (0� , 0)⇒

u00 arbitrary constant, condition on f : f 00 = 0

In Fourier basis, differential operators are diagonal.

6

Image by MIT OpenCourseWare.

Page 42: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 43: MIT Numerical PDE

� �

� �

18.336 spring 2009 lecture 10 03/10/09

Elliptic Equations and Linear Systems

In rectangular geometries, construct 2D/3D from 1D by Tensor product.

−(uxx + uyy) = f (x, y) ∈ ] − 1, 1[2

Ex.: u = 0 on ∂Ω

Assume have 1D discretization of

−uxx = f ] − 1, 1[u = 0 {−1, 1}

1

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 44: MIT Numerical PDE

� � � �

e.g.: finite difference •

K1D = sparse

2D discretization: I ⊗ K1D K1D ⊗ I

K2D = = +

• Spectral (cheb.m (interior))

K1D = full

2D discretization: I ⊗ K1D K1D ⊗ I

K2D = = +

Kronecker product: ⎤⎡

1 2 a b=

3 4 ⊗

c d

⎢⎢⎣

a b 2a 2b c d 2c 2d 3a 3b 4a 4b

⎥⎥⎦

3c 3d 4c 4d

Matlab: K2D = kron(I1D, K1D) + kron(K1D, I1D)

2

Page 45: MIT Numerical PDE

Lexicographic ordering:

Also possible: red-black ordering

→ K2D = h1 2

(advantageous for elimination)

3D:

K3D = kron(K2D, I1D) + kron(I2D, K1D)

Alternative to Tensor product: directly by Taylor expansion

3

Image by MIT OpenCourseWare.

Images by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 46: MIT Numerical PDE

Ex.: 2D Poisson equation:

−(uxx + uyy) = f

1 h2

1 6h2

5-point stencil 9-point stencil

h2

�52 u = �2u + (uxxxx + uyyyy) + O(h4)

12h2

�92 u = �2u + (uxxxx + 2uxxyy + uyyyy) +O(h4)

12 � �� � =�2(�2u)=�2f

Advantage of 9-point stencil: deferred correction

h2 2−�9 uij = f(xi, yi) −

12�2f(xi, yi).

Fourth order scheme.

4

Page 47: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 48: MIT Numerical PDE

� �

18.336 spring 2009 lecture 11 03/12/09

Efficient Methods for Sparse Linear Systems

If spectral FFT ! ⇒

If non-spectral (FD, FE): elimination (direct) •

• iterative → multigridKrylov methods (e.g. conjugate gradients)•

Elimination Methods

Solve A x = b·Matlab: x = A\b

If A square, regular: can use elimination methods

A symmetric positive definite: Cholesky factorizationOtherwise: LU factorization

Fill-in

Adding rows creates nonzero entries and may thus destroy sparsity.

Matlab:

[L,U] = lu(A); spy(L) spy(U)

Minimum degree algorithms:

Reduce fill-in by reordering of rows and columns

Ex.: Red-black ordering for K2D

1

Image by MIT OpenCourseWare.

Page 49: MIT Numerical PDE

� �

� �

Matlab:

A non-symmetric A symmetric

p = colamd(A) p = symamd(A) [L,U] = lu(A(p,:)) [L,U] = lu(A(p,p))

or [L,U] = chol(A(p,p))

Strategy: Choose remaining Strategy: Choose remaining column with fewest nonzeros meshpoint with fewest neighbors

Further: • Graph separators

Nested dissection•

Elimination is great for small matrices whose entries are directly accessible.

Preconditioning

A x = b·Condition number: κ = cond(A) = ||A|| · ||A−1||

A symmetric: cond2(A) = |λmax||λmin|

cond(A) � 1 small error in b can yield large error in x⇒

Formulate equivalent system which is better conditioned.

Left preconditioning: solve (P −1A) x = P −1b· Right preconditioning: 1. solve (AP −1) y = b·

2. solve P x = b·

1 1 Ex.: A = λ ∈ {0.999, 1000.001} ⇒ cond(A) ≈ 1000

1 1000

1 0 P = diag(A) =

0 1000

cond2(P −1A) = cond(AP −1) ≈ 2.65

2

Image by MIT OpenCourseWare.

Page 50: MIT Numerical PDE

� �

Ex.:

P = D• A =

P = D + L•

• P = Lapp · Uapp (ILU = Incomplete LU factorization) [Matlab: luinc]

Iterative Methods

A x = b· ⇔�

x = (I − A) · x + b � splitting

x(k+1) = (I − A) · x(k) + b iteration(0)x = x0

(AP −1)y = b Apply to precondition system:

Px = y

y(k+1) = (I − AP −1)y(k) + bPx(k+1) = (P − A)x(k) + b⇔

(k+1) = (I − P −1A) (k) + P −1bx x⇔ � �� � =M

⇔ P (� x(k+1) ��− x(k)�) = b� − A��· x(k� )

(k) =r(k)=z

update residual

Error:

x = A−1b e(k) = x − x(k)

e(k+1) = M e(k) [independent of b ]⇒ · Iteration converges if ρ(M) < 1Spectral radius ρ(M) = max |λ(M)|

Popular Preconditioners:

A =

P = D P = D + L P = D + wL

Jacobi Gauß-Seidel SOR (Successive

M = I − D−1A M = I − (D + L)−1A (overwrite entries

as computed) OverRelaxation) [better: SSOR]

3

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 51: MIT Numerical PDE

� �

� �

Theorem:

• If A diagonal dominant |aii| > |aij | ⇒ Jacobi converges j=1

• If Jacobi converges ⇒ Gauß-Seidel converges (× 2 faster) • If 0 < w < 2 ⇒ SOR converges

2 wopt =

1 + 1 − µ2 , µ = ρ(I − (D + I)−1A).

Multigrid

Heat Equation Iterative Scheme 1

2u = f p≈

Δt I P (u(k+1) (k)) = −A u(k) + fut −� −→ − u ·

↑Poisson matrix

Iterative schemes behave like heat equation.Slow convergence, fast smoothing of error.

Smoothers:

P = 23 D Weighted Jacobi

P = D + L Gauß Seidel (popular)P = D + wL SOR (costly)

Smoother reduces high frequency error components fast. Smooth error is rough on coarser grid.

−uxx = 1 Ex.:

u(0) = 0 = u(1)

2 −1. .−1 . .. . . .. .. . −1

−1 2

⎢⎢⎢⎣fine grid (h =

81 ) Ah = 64

coarse grid A2h = . . .

⎤⎡ ⎥⎥⎥⎦

4

Page 52: MIT Numerical PDE

Interpolation: Linear

⎤⎡ 1 2 1 1

2

⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎥⎥⎥⎥⎥⎥⎥⎥⎦

1 ∈ R7×3I =2

1 1 2 1

Restriction: Full Weighting

⎤⎡ 1 2 1

1 ⎣ 1 2 1R =4

1 2 1

R = 1 IT 2

Coarse Grid Matrix: ⎤⎡ 2 −1 −1 2 −1

2

⎣ ⎦Galerkin: A2h = R Ah · I = 16 · −1

5

Page 53: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 54: MIT Numerical PDE

18.336 spring 2009 lecture 12 03/17/09

Twogrid Method:

Fine: Ah · u = bh

R2h Ihh 2h↓ ↑

Coarse: A2h · v = b2h

(1) Iterate Ah · u = bh (ν1 × GS) � uh

(2) Restrict residual rh = bh − Ahuh by r2h = Rh 2h rh·

(3) Solve for coarse error: A2h · e2h = r2h

(4) Interpolate error: eh = I2hh · e2h

Update uh = uh + eh

(5) Iterate Ah · u = bh (ν2 × GS) starting with uh

v-cycle:

Multigrid:

Use twogrid recursively in (3) •

V-Cycle:

1

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 55: MIT Numerical PDE

���� ����

W-Cycle: Apply (3) twice

FMG (full multigrid):

Optimal: Cost = O(n).

Krylov Methods

Consider A x = b already preconditioned. · Iterative scheme: x(k+1) = x(k) + (b − Ax(k)) x(0) = 0 x(1) = b x(2) = 2b − Ab x(3) = 3b − 3Ab + A2b

(k)Observe: x

where

∈ Kk

Kk = span{b, Ab, . . . , Ak−1b}Krylov subspace

K1 ⊂ K2 ⊂ K3

∈ ∈ ∈

. . . ⊂

(1) (2) (3)x x x

Find sequence x(k) ∈ Kk which converges fast to x = A−1 b.· ⊕ Only requirement: Apply A (can be blackbox).

Examples of Krylov Methods:

Choose x(k) ∈ Kk, such that (1) rk = b − Axk ⊥ Kk → conjugate gradients (CG) (2) ||rk||2 minimal GRMES & MINRES (3) rk ⊥ Kk(A

T ) →→BiCG

(4) ||ek||2 minimal SYMMLQ→

2

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 56: MIT Numerical PDE

Conjugate Gradient Method

A symmetric positive definiteEnforce orthogonal residuals: rk ⊥ Kk

Know xk ∈ Kk ⇒ rk = b − Axk ∈ Kk+1 ⇒ rk = γk qk+1 (γk ∈ R)where q1, q2, q3, . . . orthonormal, and qk ∈ Kk.⇒ ri

T · rk = 0 ∀i < k. �Δrk = rk − rk−1 ⊥ Ki T

Δxi = ⇒ Δxi · Δrk = 0 ∀i < k

xi − xi−1 ∈ Ki

Also: Δrk = (b − Axk) − (b − Axk−1) = −A Δxk·

ΔxiT A Δxk = 0 ∀i < k ⇒ · ·

Updates (= directions) are “A-orthogonal” or “conjugate”;Scalar product (Δxi, Δxk)A := Δxi

T A Δxk.· · Search direction: dk−1

Update solution: xk = xk−1 + αkdk−1

New direction: dk = rk + βkdk−12

αk = dT

||rk−

A 1||2

dk−1 , so that error in direction dk−1 minimal

k−1 · · 2

βk = ||rk||2 , so that (dk, dk−1)A = 0 2 ||rk−1||2

CG finds unique minimizer of E(x) = 1 xT A x−xT b (xmin = A−1 b)2 · · · ·

using conjugate directions after at most n steps.

In practice much faster than n steps.

3

Image by MIT OpenCourseWare.

Page 57: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 58: MIT Numerical PDE

� �

� � � � � �

18.336 spring 2009 lecture 13 03/19/09

Initial Value Problems (IVP)⎧⎨⎩

⎫⎬ut = Lu in Ω× ]0, T [ PDE← u = u0 on Ω × {0} initial condition← u = g on ∂Ω× ]0, T [ boundary condition←

where L differential operator. 2

⎫⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪

Ex.: • L = �Poisson equation •Lu = b · �u advection equation

heat equation→

u)•Lu = −�2(�2 not done yet.biharmonic equation •Lu = F |�u|Eikonal equation

beam equation→

nonlinearlevel set equation→ ⎭

etc.• Stationary solution of IVP: Lu = 0 in Ω(if it exists) u = g on ∂Ω

Later: second order problems systems

⎡ ⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎦

∂ u 0 1 ∂ u utt = uxx

∂t v =

1 0 · ∂x v

(wave equation)

Semi-Discretization

In space (method of lines):• Approximate u( , t) by �u(t)·Approximate Lu by A �u (for linear problems) [FD, FE, spectral]·

d → system of ODE: dt

�u = A · �u

In time:• Approximate time derivative by step: d

u(x, t) ≈ u(x, t + Δt) − u(x, t)

[explicit Euler]dt Δt

Stationary problem:→ unew(x) = u(x) + ΔtLu(x) = (I + ΔtL)u(x)

Need to know about ODE solvers.

1

Page 59: MIT Numerical PDE

� �

Numerical Methods for ODE

y(t) = f(y(t)) y(t) ∈ Rd

y(0) = y

yn ≈ y(t), yn+1 ≈ y(t + Δt)n+1 n

Linear approximation: y ≈ y

Δ

− t

y

Explicit Euler (EE): yn+1 = yn + Δt f(yn)· Implicit Euler (IE): yn+1 − Δt f(yn+1) = yn ·

nonlinear system Newton iteration. ←

Local truncation error (LTE):EE: τn = y(t + Δt) − (y(t) + Δtf(y(t))) = 1 y(t)Δt2 = O(Δt2)

2 IE: τn = O(Δt2)

Global truncation error (GTE): T

Over N = time steps. Δt

En = yn − y(tn)En+1 = En + Δt(f(yn) − f(y(tn))) + τn

= En+1 |ET

n + ΔtL|En + τn⇒ |EN

| ≤LT |

τn

| | |= ⇒ | | ≤ e

Δt maxn | | = O(Δt)

Time Stepping:

GTE = one order less than LTE.

Higher Order Time Stepping

• Taylor Series Methods:

Start with EE, add terms to eliminate leading order error terms. PDE Lax-Wendroff →

• Runge-Kutta Methods:

Each step = multiple stages

k1 = f(yn + Δt aij kj )j

. . .

kr = f(yn + Δt arj kj )�jn+1y = yn + Δt bj kj

j

Butcher tableau: (cl = alj )j

2

Page 60: MIT Numerical PDE

c1 . . . cr

a11 . . . a1r . . . ar1

. . .

. . .

. . . arr

= c A

bT

b1 . . . br

0 EE:

IE: 1

Explicit midpoint: 0 1 2

Heun’s: 0 1

RK4:

0 1 2 1 2 1

Implicit trapezoidal: 1 2

• Multistep Methods: r� r�

1

1 1

k1 = f(yn) yn+1 = yn + Δtk1

k1 = f(

=yn+1 � �� � y n + Δt · k1)

yn+1 = yn + Δt · k1

1 2 0 1

1 1 2

1 2

1 2

1 6 1 2 1

1 2

1 3

1 1 3

1 6

PDE → Crank-Nicolson

αj y n+j = Δt βj f(y n+j ) j=0 j=0

Explicit Adams-Bashforth:

yn+1 = yn + Δtf(yn) = EE O(Δt) yn+2 = yn+1 + Δt [3 f(yn+1) − 1 f(yn)] O(Δt2)·

2 2 . . .

Implicit Adams-Moultion:

yn+1 = yn + Δt (21 f(yn) + 1

2 (yn+1)) = trapezoidal O(Δt2)

yn+2 = yn+1 + Δ· t ( 5 f(yn+2) + 8 f(yn+1) − 1 f(yn)) O(Δt3)·

12 12 12 . . .

BDF (backward differentiation):

yn+1 = yn + Δtf(yn+1) = IE O(Δt) 3yn+2 − 4yn+1 + yn = 2Δtf(yn+2) O(Δt2)

. . .

3

Page 61: MIT Numerical PDE

� �� �

� �

���� ����

� �� �

� � � �

� �

� �� �

� �

� �� �

� ��

� � � �

Linear ODE Systems

y = A y· y(0) = y

solution: y(t) = exp(tA) y· solution stable, if Re(λi(A)) < 0 ∀i.

= MEE

EE: yn+1 = (I + Δt A) yn · ·IE: yn+1 = A)−1 yn ��·(I − Δt ·

=MIE

Iteration yn+1 = M · yn stable, if |λi(M)| < 1 ∀i λi eigenvalue of A⎧⎨

⎫⎬1 + Δt λi eigenvalue of MEE· |1 + Δt λi < 1 if Δt <|

1 − Δt λi·

·

EE conditionally stable: Δt < ρ(A)

IE unconditionally stable

Message: One step implicit is more costly than one step explicit.

|λi|1 1⇒⎩ ⎭eigenvalue of MIE < 1 always1 − Δt λi· 2

But: If ρ(A) large

stiffness

, then implicit pays!

Ex.: Different time scales

A = −50 49

, y =2

49 −50 0

Solution: y(t) = e−t 1+ e−99t −1 ·

1 ·

1

behavior restricts Δt for EE ⎤⎡Ex.: −2 1 . .⎢⎢⎢⎣

⎥⎥⎥⎦

1A =

(Δx)2 ·

. .1 heat equation

4

. .. .. . . . 1

−21 ρ(A) <⇒(Δx)2

−4<λi<0

(Δx)2

EE stable if Δt < . 2

OTOH: Crank-Nicolson Δt Δt

I − 2

A · yn+1 = I +2

A · yn

Unconditionally stable.

4

2

Page 62: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 63: MIT Numerical PDE

� �

18.336 spring 2009 lecture 14 03/31/09

Von Neumann Stability Analysis

Lax-equivalence theorem (linear PDE):

Consistency and stability convergence ⇐⇒↑ ↑

(Taylor expansion) (property of numerical scheme)

Idea in von Neumann stability analysis:Study growth of waves eikx .(Similar to Fourier methods)

Ex.: Heat equation

ut = D uxx·

Solution: e−Dk2t ikx u(x, t) = � �� � e·

=G(k) growth factor

no growth if |G(k)| ≤ 1 ∀k

FD Scheme:

Un+1j − Uj

n Ujn +1 − 2Uj

n + Ujn −1

= D Δt

· (Δx)2

(Explicit Euler) (Central) � � DΔt ⇒ Ujn+1 = Uj

n + r · Ujn +1 − 2Uj

n + Un , r = (Δx)2j−1

Insert u(x, tn) = eikx into FD scheme:

Ujn+1 = eikΔx·j + r eikΔx·(j+1) − 2eikΔx·j + eikΔx·(j−1)

= (1 + r(eikΔx + e−ikΔx − 2))eikΔx·j

= G(k) eikΔx·j ·

Growth factor: G(k) = 1 − 2r (1 − cos(kΔx))· FD scheme stable, if |G(k)| ≤ 1 ∀kHere: worst case: kΔx = π G(k) = 1 − 4r⇒ Hence FD scheme conditionally stable:

1 (seen before) r ≤ 2

1

Page 64: MIT Numerical PDE

� �

Fast version:

G − 1 eiθ − 2 + e−iθ 2DΔt

= D (Δx)2

= (Δx)2

· (cos(θ) − 1)

⇒ G = 1 − 2r · (1 − cos θ), θ = kΔx

Ex.: Crank-Nicolson

Un+1 − Un 1 Un+1 − 2Un+1 + Un+1 + Un j j j+1 j j−1 Uj

n +1 − 2Uj

n j−1

= D + Δt

· 2 ·

(Δx)2 (Δx)2

G − 1= D

1(G + 1)

eiθ − 2 + e−iθ

Δt · 2 · ·

(Δx)2

G =1 − r · (1 − cos θ)⇒ 1 + r (1 − cos θ)·

Always |G| ≤ 1 ⇒ unconditionally stable.

Ex.: 2D heat equation

ut = uxx + uyy

Forward Euler

Un+1 − Un Uin +1,j − 2Un

i−1,j Un i,j + Un

1j ij = i,j + Un

+ i,j+1 − 2Un i,j−1

Δt (Δx)2 (Δy)2

u(x, y, tn) = e i(k,l)·(xy ) = eikx eily·

G − 1=

eikΔx − 2 + e−ikΔx

+ eilΔy − 2 + e−ilΔy

Δt (Δx)2 (Δy)2

Δt Δt⇒ G = 1 − 2(Δx)2

· (1 − cos(kΔx)) − 2(Δy)2

· (1 − cos(lΔy))

Δt Δt Worst case: kΔx = π = lΔy ⇒ G = 1 − 4

(Δx)2 − 4

(Δy)2

Stability condition: Δt Δt 1 1

� 1 1

�−1 h2

+ + = (Δx)2 (Δy)2

≤ 2 ⇔ Δt ≤

2 ·

(Δx)2 (Δy)2 ���� 4 if Δx=h=Δy

2

Page 65: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 66: MIT Numerical PDE

� �� �

18.336 spring 2009 lecture 15 02/13/08

Finite Difference Methods for the One-Way Wave Equation

ut = cux

u(x, 0) = u0(x)

Solution: u(x, t) = u0(x + ct)

Information travels to the left with velocity c.

Three Approximations: ⎧ ⎪⎪⎪⎪⎪⎪⎪⎪ cUj

n +1 − Uj

n

upwind Δx

Un+1 ⎨j − Uj

n

= Un − Uj

n −1j

c downwindΔt ⎪⎪⎪⎪⎪⎪⎪⎪

Δx

Un ⎩ c j+1 − Ujn −1

centered 2Δx

Accuracy:

Taylor expansion of solution u

u(x, t + Δt) − u(x, t) 1 1 = ut + uttΔt + uttt(Δt)2 + O((Δt)3)

Δt 2 6u(x + Δx, t) − u(x, t) 1 1

= ux + uxxΔx + uxxx(Δx)2 + O((Δx)3)Δx 2 6

u(x + Δx, t) − u(x − Δt, t) 1= ux + uxxx(Δx)2 + O((Δx)4)

2Δx 6

Substitute into FD scheme:

Upwind: ut +1 uttΔt −cux − 1 cuxxΔx + O(Δt2) + O(Δx2)���� 2 2

=cux= 1 c2uxxΔt

2

Leading order error: 1 1 1 2 1 1 uttΔt − cuxxΔx = c uxxΔt − cuxxΔx = cuxxΔx(r − 1)2 2 2 2 2

cΔt = 0 if r = 1 r =

ΔxFirst order if r = 1 Courant number

1

Image by MIT OpenCourseWare.

Page 67: MIT Numerical PDE

Downwind: Analogous: first order

Centered: ut + 1 uttΔt − cux − 1 cuxxxΔx2 + O(Δt2) + O(Δx4)2 6

Δt First order in time 2

→Δx Second order in space →

Stability:

eUpwind:

G − 1= c

ikΔx − 1 Δt Δx

⇒ G = 1 − r + reikΔx

⇒ |G| ≤ |1 − r| + |reikΔx| = 1, if 0� ≤��r ≤ 1� CFL-Condition

conditionally stable

Downwind: G = 1 + r − re−ikΔx

unstable

eikΔx − e−ikΔx

2 = 1 + ir sin(kΔx)Centered: G = 1 + r

unstable

2

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 68: MIT Numerical PDE

� �

Messages:

1. Upwind works (CFL-condition on stability) 2. Centered needs a fix

Add Diffusion:

UnUjn+1 − Uj

n Ujn +1 − Uj

n −1 j+1 − 2Uj

n + Ujn −1

= c + θ Δt 2Δx (Δx)2

Replace Ujn by average:

Ujn+1 − λ

2 Ujn +1 + (1 − λ)Uj

n + λ 2 Uj

n −1

= cUj

n +1 − Uj

n −1

Δt 2Δx Δt

where λ = 2 θ (Δx)2

How much diffusion?

Lax-Friedrichs:

(Δx)2

Eliminate Ujn by λ = 1 θ = ⇒

2Δt

Un+1 1 + r 1 − r cΔt j = Uj

n +1 + Uj

n −1 r =

Δx� ��2 � � ��2 � ≥0(for |r|≤1) ≥0(for |r|≤1)

Monotone scheme

Accuracy: First in time, Second in space (exercise)Stability: G = ir sin(kΔx) + cos(kΔx)� �� � � �� �

central difference diffusion

conditionally stable r| | ≤ 1

Lax-Wendroff: Δt

Choose θ to get second order in time: θ = c 2

2 Un+1

j − Ujn Uj

n +1 − Uj

n −1 Δt 2 Uj

n +1 − 2Uj

n + Ujn −1

= c + c Δt 2Δx 2 (Δx)2

3

Image by MIT OpenCourseWare.

Page 69: MIT Numerical PDE

Accuracy: ut+1 uttΔt+1 utttΔt2−cux−1 cuxxxΔx2 Δt c2uxx−Δt c2uxxxxΔx2 2 6 6 −

2 24

= 61 utttΔt2 −

61 cuxxxΔx2 = O(Δt2) + O(Δx2)

ut − cux = 0 1 Δt 2uttΔt − c uxx = 0 2 2

Stability: λ = r2

G = r2 + r

e ikΔx + (1 − r 2) + r2 − r

e−ik(Δx)

2 2= (1 − r 2) + r 2 cos(kΔx) + ir sin(kΔx)

Worst case: kΔx = π ⇒ G = 1 − 2r2

Stable if |r| ≤ 1

4

Image by MIT OpenCourseWare.

Page 70: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 71: MIT Numerical PDE

18.336 spring 2009 lecture 16 02/19/08

Modified Equation

Idea: Given FD approximation to PDE

Find another PDE which is approximated better by FD scheme. Learn from new PDE about FD scheme.

Ex.: ut = cux Un+1

j − Ujn Uj

n +1 − Un Uj

n +1 − 2Uj

n + Un

Lax-Friedrichs: Δt

− c 2Δx

j−1 − 2Δt

j−1 = 0

1 1 1 1 Taylor: ut+

2 uttΔt−cux −

6 cuxxxΔx 2 −

2ΔtuxxΔx 2 −

24ΔtuxxxxΔx 4+. . . � 2 �1 Δx

= (ut − cux) + utt Δt − uxx + . . . 2 ���� Δt

2=c uxx� 2 �1 Δx= (ut − cux) + c 2Δt − uxx

2 ΔtModified equation:

Δx2 cΔt ut − cux = (1 − r 2)uxx r =

2Δt ΔxAdvection-diffusion equation with diffusion constant

Δx22D = ( 1 r )���� ����2Δt

− added diffusion antidiffusion by central differencing

Ex.: Upwind: ut − cux =

21 cΔx(1 − r)uxx (exercise)

Compare:

For c = 1, r = 21 DLF =

43 Δx , DUW =

41 Δx−→

Upwind less diffusive than LF.

Ex.: Lax-Wendroff ut − cux = 1 cΔx2(r2 − 1)uxxx (uxx cancels by construction)

6

Advection-dispersion equation with dissipation constant

µ = −61 cΔx2(1 − r2)

Disturbances behave like Airy’s equation

Message:

First order methods behave diffusive. Second order methods behave dispersive.

1

Page 72: MIT Numerical PDE

� �� �

More on Advection Equation

ut + cux = 0

So far:

1. Upwind: ⎧⎪⎪⎪⎨

⎫⎪⎪⎪⎬Un

j − Ujn −1 −c

Δx c > 0

Ujn+1 − Uj

n

e = O(Δt) + O(Δx)= →⎪⎪⎪⎩⎪⎪⎪⎭

Δt Un

−c j+1

Δ

x

Ujn

c < 0

2. Lax-Friedrichs/Lax-Wendroff:

Ujn+1 − Uj

n

= Uj

n +1 − Uj

n −1

+ θUj

n +1 − 2Uj

n + Ujn −1

Δt 2Δx (Δx)2

LF: θ = (Δx)2

e = O(Δt) + O(Δx 2)2Δt

LW: θ =Δt

c 2 e = O(Δt2) + O(Δx 2)2

Semidiscretization:

Central: ux = Uj+1 − Uj−1

+ O(Δx 2)2Δx ⎤⎡⎤⎡ ⎤⎡0 1 −1

(ux)1 u1⎢⎢⎢⎣

⎥⎥⎥⎦

. .1 . .−1⎢⎣

⎥⎦⎢⎣

⎥⎦. ....

...Matrix = ·2Δx . .. . . . 1(ux)k uk

1 −1 0

=A

AT = −A ⇒ eigenvalues purely imaginary

Need time discretization that is stable for u = λu with λ = iµ, µ ∈ R

Linear Stability for ODE:

Region of absolute stability = {λ ∈ C : method stable for u = λu}

2

Page 73: MIT Numerical PDE

Ex.:

Forward Euler Backward Euler Trapezoidal un+1 = un + λΔtun

n+1 1 n n+1 1 + 21 λΔt

u = u u = = (1 + λΔt)un 1 − λΔt 1 −

21 λΔt

Stable if |1 + λΔt| ≤ 1

RK4 RK2 Adams-Bashforth 3

Can also use higher order discretization of ux

(up to spectral). If central need ODE solver for timestep ⇒that is stable for u = iµu.

Spurious Oscillations

Stable does not imply “no oscillations.”

Ex.: Lax-Wendroff

Overshoots remain bounded stable.⇒Still bad (e.g. density can become negative)

3

Image by MIT OpenCourseWare.

Page 74: MIT Numerical PDE

� � Total Variation:

TV (u) = |uj+1 − uj | ≈ |ux(x)|dx “total up and down” j

Method total variation diminishing (TVD), ifTV (un+1) ≤ TV (un).

Bad News: Any linear method for advection that is TVD,is at most first order accurate.

[i.e.: high order spurious oscillations]→

Remedy: Nonlinear Methods:

1. Flux-/Slope- Limiters

� conservation laws; limit flux TVD→

2. ENO/WENO

(weighted) essentially non-oscillatory(essentially TVD; no noticeable spurious oscillations)

ENO/WENO

Approximate ux by interpolation.

ENO: At each point consider multiple interpolating polynomials (through various choices of neighbors). Select the most “stable” one to define ux.

WENO: Define ux as weighted average of multiple interpolants. Higher order when u smooth, no overshoots when u non-smooth.

4

Page 75: MIT Numerical PDE

� �

Ex.: Fifth order WENO

s1 = 13 (v1 − 2v2 + v3)2 + 1 (v1 − 4v2 + 3v3)

2 12 4

s2 = 13 (v2 − 2v3 + v4)2 + 1 (v2 − v4)

2 12 4

= 13 + 1 (3v3 − 4v4 + v5)2 =

Uj+1 − Ujs3 12 (v3 − 2v4 + v5)

24 vj

Δx a1 = 1 /(� + s1)

2 10 6 2)a2 = 10 /(� + s2)

2 � = 10−6 max(vj· j

3 a3 = 10 /(� + s3)

2

sa = a1 + a2 + a3 a1w1 = sa

a2w2 = sa

a3w3 = sa

w = 1 (2v1 − 7v2 + 11v3) + w2 · (−v2 + 5v3 + 2v4) + w3 · (2v3 + 5v4 − v5))6 (w1 ·

Left sided approximation to ux at x4

Right sided approximation to ux at x3

ut + cux = 0

Upwind WENO with FE:

Ujn+1 − Uj

n � −c WENOleft Uj

n > 0 �

= ·

Δt −c WENOright Ujn ≤ 0·

TVD time stepping

Consider method that is TVD with FE. Is it also TVD with high order time stepping? In general: “no.” But for special class of ODE schemes: “yes.” Strong Stability Preserving (SSP) methods

5

Image by MIT OpenCourseWare.

Page 76: MIT Numerical PDE

Ex.: FE (un) = un + Δtf(un)

RK3-TVD 1 2 3 1n+1 un + un + FE(FE(un)))FE (u =3 3 4 4

Convex combination of FE steps ⇒ Preserves TVD property

Compare: Classical RK4 cannot by written this way. It is not SSP.

Popular approach for linear advection:ut + cux = 0

RK3-TVD in time, upwinded WENO5 in space.

2D/3D: Tensor product in space.

WENO5-stencil

6

Image by MIT OpenCourseWare.

Page 77: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 78: MIT Numerical PDE

� �

18.336 spring 2009 lecture 17 04/09/09

Conservation Laws

ut + (f(u))x = 0 if u∈C1

ut + f �(u) ux = 0 ⇐⇒ � �� � =c(u)

Conservation form Differential form � b �

d u(x, t)dx = f(u(a, t)) − f(u(b, t)) f = flux function

dt a

Integral form

Ex.: Transport equation f(u) = cu c(u) = f �(u) = c⇒

Ex.: Burgers’ equation

f(u) = 1 u2 c(u) = f �(u) = u2 ⇒

ut + uux = 0Model for fluid flow

Du 1DMaterial derivative: = ut + (u · �)u = ut + uux

Dt

Ex.: Traffic flow

ρ = 0 empty ρ(x, t) = vehicle density

ρ = 1 packed

� b

m(t) = ρ(x, t)dx = number of vehicles in [a, b] a

d m(t) = f(ρ(a, t)) − f(ρ(b, t))

dt Influx Outflux

1

Image by MIT OpenCourseWare.

Page 79: MIT Numerical PDE

����

Equation: ρt + (f(ρ))x = 0 where f(ρ) = v ρ·vehicle velocity

Velocity function Flux function Velocity of information v = v(ρ) = 1 − ρ ⇒ f(ρ) = ρ(1 − ρ) c(ρ) = f �(ρ) = 1 − 2ρ

Method of Characteristics

ut + f �(u)ux = 0u(x, 0) = u0(x)

Follow solution along line x0 + ct, where c = f �(u0(x0)).d

u(x + ct, t) = cux(x + ct, t) + ut(x + ct, t)dt

= (c − f �(u(x + ct, t))) ux(x + ct, t) = 0 � �� � ·

=0

⇒ u(x + ct, t) = constant = u(x0, 0) = u0(x0).

Ex.: Transport Burgers’ Traffic

Characteristic lines intersect shocks ⇒

2

Page 80: MIT Numerical PDE

� � �

� �

Weak Solutions

ut + (f(u))x = 0 (∗)

u(x, 0) = u0(x)

Solution for u0 ∈ C1 smooth until characteristics cross.

x1 + f �(u0(x1)) t = x2 + f �(u0(x2)) t· · x2 − x1 1 ⇒ t = −

f �(u0(x2)) − f �(u0(x1)) = −

(f � ◦ u0)�(x) x ∈ [x1, x2]

1 = −

f ��(u0(x))u0�(x)

1 ⇒ ts = −infxf ��(u0(x))u0

�(x)

Solution for t > ts:

Weak formulation ∞ +∞ +∞

(∗∗) uϕt + f(u)ϕx dx dt = − [uϕ]t=0 dx ∀ � ϕ ∈��

C� 1 0

0 −∞ −∞ Test function, C1 with compact support

If u ∈ C1 (“classical solution”), then (∗) ⇔ (∗∗)Proof: integration by parts.In addition, (∗∗) admits discontinuous solutions.

Riemann Problem

uL x < 0 u0(x) =

uR x ≥ 0� bd (uL − uR) s = u(x, t) dx·

dt a

= f(uL) − f(uR)

f(uR) − f(uL) s = ⇒

uR − uL

Rankine-Hugoniot Condition for shocks

3

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 81: MIT Numerical PDE

Ex.: Burgers’

R − 2u uL 1 2

212s =

uL + uR =

2uR − uL

Replace breaking wave by shock

Rarefactions

Ex.: Burgers’ Many weak solutions...

This is what physics yields (stable w.r.t. small perturbations)

Entropy Condition to single out unique weak solution:

• Characteristics go into shock:

f �(uL) > s > f �(uR)

• Solution to ut + (f(u))x = 0 is limit of ut + (f(u))x = νuxx as ν → 0

“Vanishing viscosity”

• Many more...

All equivalent.

4

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 82: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 83: MIT Numerical PDE

18.336 spring 2009 lecture 18 04/14/09

Intermezzo:Boundary Conditions for Advection

Linear advection

ut + ux = 0 x ∈ [0, 1]

Upwind (first order) treats boundary conditions naturally correctly.

LF, LW: Need artificial/numerical boundary conditions at x = 1,that acts as close to “do nothing” as possible.

WENO: Need information outside of domain.

Which values?

Extrapolate solution; works if u smooth.E.g., constant/linear/etc. extrapolation.

1

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 84: MIT Numerical PDE

� �� �

Numerical Methods for Conservation Laws

Naive Schemes

Ex.: Burgers’

ut + uux = 0 with u > 0 upwind

Un+1 Un j − Uj

n

= −Un j − Ujn −1

Δt j Δx Non-conservative; fine if u ∈ C1 . But: Wrong if u has shocks (see pset 5).

Conservative FD Methods

ut + (f(x))x = 0

Discretize in conservation form ⇒ conservation ⇒ correct shock speeds ⎧ ⎪⎪⎨

⎫ ⎪⎪⎬

Upwind:f(Uj

n) − f(Ujn −1)

f �(Ujn) ≥ 0

ΔxUn+1 j

−− Ujn

if=⎪⎪⎩f(Uj

n +1) − f(Uj

n)

Δx

⎪⎪⎭Δt

f �(Ujn) < 0−

Lax-Friedrichs:

Ujn+1 − 1

2 (Ujn +1 + Uj

n −1)

+ f(Uj

n +1) − f(Uj

n −1)

= 0 Δt 2Δx

(no straightforward LW, since based on linear Taylor expansion)

Numerical Flux Function

Ujn+1 − Uj

n Fjn − Fj−1

n

+ = 0 Δt Δx

)−f (Un j )

� f(Uj

n) ≥ 0nj

njf(U +1

U +1−UUpwind: Fj n = if n

j < 0f(Uj+1 n)

Δx LF : Fj

n = 21 (f(Uj

n) + f(Ujn +1)) − (Uj

n +1 − Uj

n)2Δt

2

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 85: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 86: MIT Numerical PDE

18.336 spring 2009 lecture 19 04/10/09

Finite Volume Methods (FVM)

FD: Un ≈ function value u(jΔx, nΔt)j � (j+

Δx (j−

1 2)Δx

)Δx

1FV: Un ≈ cell average j u(x, nΔt)dx

1 2

Fluxes through cell boundaries

Un+1 j

Fj+n − Fj−

n 1 2

1 2

− Ujn

= 0

Godunov Method

REA = Reconstruct-Evolve-Average Burgers’ equation

+Δt Δx

CFL Condition: Δt ≤ C Δx· Local RP do not interact

1

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 87: MIT Numerical PDE

� �� �

If f ��(u) > 0 (convex flux function)

f(Ujn −1) Uj

n −1 > us, s > 0

⎧ ⎪⎨ ⎫ ⎪⎬ (1)

F n f(Ujn) if Uj

n < us, s < 0 (2)1 = 2

j− ⎪⎩ ⎪⎭f(Us) Uj

n −1 < us < Uj

n (3)

f(ujn) − f(uj

n −1)

s = n n Shock speed

uj − uj−1

f �(us) = 0 Sonic point [Burgers’: us = 0 ]

Transsonic rarefaction

If no transsonics occur, we recover exactly upwind. Close Relation

FV FD.⇒ ←→

High Order Methods

Linear case: Taylor series approach LW → FD: Larger stencilsFV: Reconstruct with linear, quadratic, etc. functions in each cell.

σjn = 0 Godunov’s method ⇒

σjn =

ujn +1 − uj

n

Δx Lax-Wendroff ⇒

un(x,tn)=Ujn+σi

n (x−xj )·

2

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 88: MIT Numerical PDE

Riemann Problem

High order not TVD need limiters. ⇒

Nonlinear Stability

Property Conservation law Numerical scheme Monotone

Initial conditions v0(x) ≥ u0(x) ∀x

⇒ v(x, t) ≥ u(x, t) ∀x, t.

Vj n ≥ Un

j ∀j ⇒ Vj

n+1 ≥ Un+1 j ∀j

L1-contracting

||u(·, t2)||L1 ≤ ||u(·, t1)||L1

∀t2 ≥ t1

||Un+1 − V n+1||1 ≤ ||Un − V n||1

[||U ||1 = Δx �

j

|uj |]

TVD

TV(u(·, t2)) ≤ TV(u(·, t1)) ∀t2 ≥ t1

[TV(u) = � |u(x)|dx]

TV(Un+1) ≤ TV(Un)

[TV(U) = �

j

|Uj+1 − Uj |]

Monotonicity preserving

ux(·, t1) ≥ 0 ⇒ ux(·, t2) ≥ 0 if t2 > t1

Un j ≥ Un

j+1 ∀j ⇒ Un+1

j ≥ Un+1 j+1 ∀j

TVB (bounded)

TV(Un+1) ≤ (1+αΔt)· TV (Un) α independent of Δt

Remark: Discontinuous solution ⇒ L1−norm is appropriate

||uh(x) − u(x)||L∞ = 1 ∀h

h 0→||uh(x) − u(x)||L1 −→ 0

Theorem (Godunov):

A linear, monoticity preserving method is at most first order accurate. Need nonlinear schemes. ⇒

3

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 89: MIT Numerical PDE

� �

High Resolution Methods

A. Flux Limiters

ut + (f(u))x = 0

Ujn+1 − Uj

n Fjn − Fj

n −1

+ = 0 Δt Δx

Use two fluxes:

TVD-flux (e.g. upwind) F• High order flux F•

Smoothless indicator:

θj = uj − uj−1 ≈ 1 where smooth uj+1 − uj away from 1 near shocks

Flux: Fj = Fj + ( Fj − Fj ) Φ(θj )·

Ex.: ut + cux = 0

+ c ΔtFj = cUj 2 1 − cΔx (Uj+1 − Uj ) Φ(θj )���� � · ��

· � Fupwind FLW−Fupwind

Conditions for Φ(θ):

• TVD: 0 ≤ Φ(θ) ≤ 2θ0 ≤ Φ(θ) ≤ 2

• Second order: Φ(1) = 1Φ continuous

Two Popular Limiters:

Superbee van Leer

Φ(θ)=max(0, min(1, 2θ), min(θ, 2)) Φ(θ) = |θ| + θ 1 + |θ|

4

Images by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 90: MIT Numerical PDE

� �

B. Slope Limiters

Upwind: σj = 0

LW: σj = Uj+1 − Uj

Δx Minmod-limiter:

σj = minmod Uj − Uj−1

,Uj+1 − Uj

Δx Δx ⎧⎨minmod(a, b)=

a |a| < |b| & ab > 0b if |a| > |b| & ab > 0⎩0 ab < 0

Many more... relation

Slope limiters Flux limiters ←→

⎫⎬⎭

5

Image by MIT OpenCourseWare.

Page 91: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 92: MIT Numerical PDE

18.336 spring 2009 lecture 20 04/23/09

Operator Splitting

IVP: ut = Au + Bu where A, B differential operators.

Most accurate: Discretize Au + Bu, and time step with high order.But: Sometimes not possible, or too costly.

Alternative: fractional steps⎧ ⎨ Time step t t + Δt :→(1) Solve ut = Au ⎩ (2) Solve ut = Bu

Special case: A,B linearSolution operators:etAu0 : solution of ut = Au, u(0) = u0

etBu0 : solution of ut = Bu, u(0) = u0

et(A+B)u0 : solution of ut = Au + Bu, u(0) = u0

True solution: u(t + Δt) = eΔt(A+B)u(t)Lie splitting: uL(t + Δt) = eΔtAeΔtBu(t)

2 2Strang splitting: uS (t + Δt) = e 1 ΔtAeΔtBe

1 ΔtAu(t) SWSS splitting: uSW (t + Δt) =

21 (eΔtAeΔtB + eΔtB eΔtA)u(t)

Local Truncation Errors:

Lie: uL(t + Δt) − u(t + Δt) = Δt2 [A, B]u(t) + O(Δt3)

2 Strang: us(t + Δt) − u(t + Δt) = Δt3 ( 1 [B, [B, A]] − 1 [A, [A, B]])u(t) + O(Δt4)·

12 24 SWSS: usw(t + Δt) − u(t + Δt) = O(Δt3)

Commutator: [A, B] = AB − BA

If operators A and B commute, then all splittings are exact; Otherwise:

Lie (globally) first order accurate, • Strang and SWSS (globally) second order accurate. •

Ex.: Convection-Diffusion equation ut + cux = duxx

Solution: u(x, t) = h(x − ct, t), where h solves ht = dhxx

Au = −cux and Bu = duxx

ABu = −c(duxx)x = −cduxxx = BAu [A, B] = 0 ⇒ ⇒ splitting exact ⇒ use Lie splitting

1

Page 93: MIT Numerical PDE

Ex.: Convection-Reaction equation

ut + cux = a − bu Au = −cux and Bu = a − bu ABu = −c(a − bu)x = bcux

BAu = a − b(−cux) = a + bcux

[A, B] = 0 use Strang splitting to be second order accurate � ⇒

Ex.: Dimensional splitting

2D Advection ut + aux + buy = 0 (1) ut + aux = 0 for Δt (2) ut + buy = 0 for ΔtAu = −aux, Bu = −buy, [A, B] = 0

Remark: No error due to splitting, if ut = Au and ut = Bu solved exactly.

If discretized in time, results will in general differ.

Uijn+1 − Uij

n Uijn − Ui

n −1,j Uij

n − Un

Ex.: FE unsplit: Δt

= −a Δx

− b Δy

i,j−1 [a, b > 0]

Lie-splitting: (1) Uij ∗ − Uij

n

= −aUij

n − Uin −1,j

Δt Δx

(2) ij i,j−1Uijn+1 − U∗ Uij

∗ − U∗

Δt = −b

Δy Hence:

aΔt bΔt Uij

n+1 =Uijn −

Δx (Un

i−1,j ) − Δy

(Un i,j−1)ij − Un

ij − Un

ab(Δt)2

i,j−1 + Uij)+ΔxΔy

(Uijn − Ui

n −1,j − Un

2

Page 94: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 95: MIT Numerical PDE

� �

� � � �

� � � � � �

18.336 spring 2009 lecture 21 04/28/09

Systems of IVP ⎡ ⎤ u1(x, t) ⎢ . ⎥

Solution has multiple components: �u(x, t) = ⎣ .. ⎦m equations um(x, t)

Uncoupled (trivial) ut = uxx

vt = vxx

Solve independently

Triangular (easy)

(1) ut + uux = 0 velocity field (2) ρt + uρx = dρxx density of pollutantSolve first (1), then (2)

Fully Coupled (hard)

ht + (uh)x = 0 shallow water equations

ut + uux + ghx = 0

h uh ⇔ u

+ 1 u2 + gh = 0

t 2 x

hyperbolic conservation law

Linear Hyperbolic Systems

Linearize SW equations around base flow ¯ u:h, ¯¯h u h h

+ = 0 u g u

· u

t x

Linear system:

(∗) �ut + A�· �ux = 0 � A ∈ Rm×m

u h � ¯SW: A = λ = u± gh

g u

(∗) is called hyperbolic, if A is diagonalizable with real eigenvalues, and strictly hyperbolic, if the eigenvalues are distinct.

A = R D R−1; change of coordinates: �v = R−1 �u· · · ⇒ �vt = D · �vx = 0 Decoupled system(vp)t + λp(vp)x = 0 ∀p = 1, . . . ,m⇒ vp(x, t) = vp(x − λpt, 0) Simple wave Solution is superposition of simple wavesNumerics: Implement simple waves into Godunov’s method.

1

Image by MIT OpenCourseWare.

Page 96: MIT Numerical PDE

� � � � � �

� � � �

� �

� �� �

Wave Equation

21D utt = c uxx

u 0 c u ∂t = ∂x⇔

v c 0 v

Ex.: Maxwell’s Equations

Et = cHx

Ht = cEx

Ett = (Et)t = (cHx)t = c(Ht)x = c(cEx)x = c2Exx ⇒ Htt = = c2Hxx· · ·

Schemes based on hyperbolic systems � � � � � � � � ϕ = u + v ϕ c 0 ϕ ϕt − cϕx = 0

= ψ = u − v

→ ∂t ψ 0 ·∂x ψ

→ ψt + cψx = 0−c

Upwind for ϕ, ψ:⎧ ⎪⎪ ϕjn+1 − ϕj

n ϕj+1 n − ϕj

n⎨ = cΔt Δx⎪⎪ ψj

n+1 − ψjn ψj

n − ψj−1 n⎩ = −c

Δt Δx

u = 21 (ϕ + ψ) and v =

21 (ϕ − ψ)

Un+1 n+1 n n+1 n n n n n j − Uj

n

=1 ϕj − ϕj

+ ψj − ψj

= c ϕj+1 − ϕj

+ ψj − ψj−1

Δt 2 Δt Δt 2 Δx Δx

c Ujn +1 − Uj

n Vj+1 n − Vj

n Ujn − Uj

n −1 Vj

n − Vj−1 n

=2 Δx

+Δx

− Δx

+Δx

Vj+1 n − Vj−1

n cΔx Ujn +1 − 2Uj

n + Ujn −1

= c + 2Δx 2

· Δx2

Vjn+1 − Vj

n Ujn +1 − Uj

n −1 cΔx Vj+1

n − 2Vjn + Vj−1

n

= = c + Δt

· · · Δx 2

· Δx2

Lax-Friedrichs-like Scheme for u, v:�� �n+1 � �n � � � �� �n � �n

� � �1 U U 0 c 1 U U cΔx Uxx

Δt V j

− V

j

= c 0

·2Δx V

j+1

− V

j−1

+2 Vyy

artificial diffusion

Stable (check by von-Neumann stability analysis).

More accurate schemes: Use WENO and SSP-RK for ϕ, ψ.

2

Page 97: MIT Numerical PDE

� � � � � �

Leapfrog Method 2utt = c uxx

Un+1j − 2Uj

n + Ujn−1

2 Uj

n +1 − 2Uj

n + Ujn −1

= c→ Δt2 Δx2

(Two-step method)

Accuracy: 1 2 1 2 2 2)utt + 2 uttttΔt

2 − c uxx − 12 c uxxxxΔx = O(Δt2) + O(Δx

(using utt − c2uxx = 0) second order

Stability:

G2 − 2G + 1 eikΔx − 2 + e−ikΔx 2G= c

Δt2 Δx2

cΔt r = ⇒ G2 − 2G + 1 = 2r2(cos(kΔx) − 1) · G Δx Courant number

G − 2 (1 − r 2(1 − cos(kΔx))) G + 1 = 0 ⇒ � �� � ·2 ⇒ G = a ±

√a − 1

=a

If a > 1 one solution with G > 1 unstable| | ⇒ 2 | |

2 ⇒

If |a| ≤ 1 ⇒ G = a ± i√

1 − a ⇒ |G| = a2 + (1 − a2) = 1 ⇒ stable

Have 1 − cos(kΔx) ∈ [0, 2], thus: a r| | ≤ 1 ⇔ | | ≤ 1 Leapfrog conditionally stable for |r| ≤ 1.

Staggered Grids

u 0 c u ∂t = ∂x v c 0 v

Collocation Grid

Central differencing requires artificial diffusion

Staggered Grid

Central differencing comes naturally

3

Image by MIT OpenCourseWare.

Page 98: MIT Numerical PDE

� �

� � � �

� �

⎧ ⎪⎪ Ujn+1 − Uj

n Vjn − Vj−1

n ⎨ = c Δt Δx ⎩

n+1 − Vjn

= cUn+1 − Un+1⎪⎪ Vj j+1 j

Δt Δx

Both are explicit central differences.

Two step update:

Ujn+1 − 2Uj

n + Ujn+1

1 Ujn+1 − Uj

n Ujn − Uj

n−1

= Δt2 Δt Δt

− Δt

c Vjn − Vj−1

n Vjn−1 − Vj−1

n−1 c Vjn − Vj

n−1 Vj−1 n − Vj−1

n−1

= = Δt Δx

− Δx Δx Δt

− Δt

c2 Ujn +1 − Uj

n Ujn − Uj

n −1 2 Uj

n +1 − 2Uj

n + Ujn −1

= = c 2Δx Δx

− Δx Δx

Equivalent to Leapfrog.

4

Image by MIT OpenCourseWare.

Page 99: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 100: MIT Numerical PDE

� �

18.336 spring 2009 lecture 22 04/30/09

Level Set Method

2D : Moving Curve Orientable, with inside and outside region

3D : Moving Surface

Ex.: Interface between water and oil (surface tension) • Propagating front of bush fire • Deformable elastic solid •

Movement of surface under velocity field �v.Tangential motion does not change surface.Only velocity component normal to surface is important.

Effective velocity field:

Ex.:

�v = Fn

1

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 101: MIT Numerical PDE

• �

Explicit Tracking

I. Lagrangian Markers:

Place markers on surface: �x1, . . . , �xn ∈ Rd

�xk = �v(�xk, t) • Move markers by ODE: �xk(0) = �xk

⊕ Fast, easy to move

⊕ Accurate (high order ODE solvers)

� Uneven marker distribution

� Incorrect entropy solution

� Topology changes

� Numerical instabilities with curvature

� Marker connections in 3D?

2

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 102: MIT Numerical PDE

� �

II. Volume of Fluid:

Regular grid • Store volume/area “inside” surface • Update volume value according to �v•

⊕ Very robust ⊕ Simple in 3D � Not very accurate � Exact surface shape and topology? � Curvature reconstruction?

Implicit Representation

Define function φ(�x), s.t.

φ > 0 outside φ = 0 interface φ < 0 inside

⎫⎬ ⎧⎨

⎩ ⎭

Store φ on regular Eulerian grid • PDE IVP for φ, yielding correct movement •

• Recover n, K from φ �φ

n = |�φ|

κ = � · 2 2�φ

=φxxφy − 2φxφyφxy + φyyφx

2|�φ| (φx 2 + φy

2) 3

Approximate φx, φy, φxx, φxy, φyy by finite differences (e.g. central).

Signed Distance Function:

φ(x, y) = �

x2 + y2 − 1

⎧⎨ ⎩

φ = 0 surface |�φ| = 1 almost everywhere φ < 0 inside

⊕ Surface reconstruction very robust

n = �φ 2φ

⊕ K = �

PDE

Movement under given velocity field �v : φt + �v · �φ = 0 Linear advection

Special case: normal velocity �v = Fn = F �φ |�φ|

φt + F |�φ| = 0 Level set equation ⇒

describes unit circle

3

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 103: MIT Numerical PDE

� �

Numerical Methods

• UpwindWENO•

• (Spectral)

Ex.: Upwind for level set equation (first order)

φn+1 i,ji,j − φn

= max(F, 0)�+ + min(F, 0)�−Δt ij ij

�+ = (max(D−xφ, 0)2+min(D+xφ, 0)2+max(D−yφ, 0)2+min(D+yφ, 0)2)ij

�− = (min(D−xφ, 0)2+max(D+xφ, 0)2+min(D−yφ, 0)2+max(D+yφ, 0)2)ij � �� � all evaluated at φn

i,j

Higher order: WENO and SSP-RK.

Reinitialization

Desirable |�φ| = 1. But in general φt + F |�φ| = 0 does not preserve |�φ| = 1.

Fixes:

Solve IVP • φτ + sign(φ)(|�φ| − 1) = 0In each time step, for 0 ≤ τ ≤ ?

• Solve Eikonal equationGiven φ, find ˆφ, s.t.

φ = 1 |� |{φ = 0} = {φ = 0}

Use fast marching method by Sethian.

• Extension velocity:Change velocity field �v to �v, s.t.

�v = �v at {φ = 0}��v = 0 → |�φ| = 1 preserved

2

1 2

1

4

Page 104: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 105: MIT Numerical PDE

� �

18.336 spring 2009 lecture 23 05/05/08

Navier-Stokes Equations

1 2ut + (u · �)u = −�p + Re � u [+g] Momentum equation

� · u = 0 Incompressibility

Incompressible flow, i.e. density ρ = constant.

Reynolds number:

Re = U · L

= inertial forces

ν viscous forces

U = Characteristic velocity L = Characteristic length scale ν = Kinetic viscosity

u in 2D: u =

v 1(1) ut + uux + vuy = −px +

Re (uxx + uyy) 1(2) vt + uvx + vvy = −py +

Re (vxx + vyy)

(3) ux + vy = 0

Famous Problems:

Lid driven cavity Flow around cylinder

3 unknowns, 3 equations

DAE (Differential Algebraic System), (3) is a constraint

1

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 106: MIT Numerical PDE

� �

Solve by projection approach:

In each time step1

I. Solve ut + (u · �)u = Re�2 u

U∗ − Un

= −(Un +1 2Un

Δt · �)Un

Re�

Note: � · U∗ = 0 �II. Project on divergence-free velocity field

Un+1 − U∗

Δt = −�p

What is p : 0 = ! � · Un+1 = � · U∗ − Δt�2P

1⇒ �2p =Δt� · U∗ Poisson equation for pressure

Discretization:

Solution:u = v = 0,p = constant

But: Central differences on grid allow solution

Uij = Vij = 0,

P1 i + j even Pij = for

P2 i + j odd

Fix: Staggered grid

× pressure p • velocity u ◦ velocity v

2

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 107: MIT Numerical PDE

� �

� �

� �

� � � �

� � � �� � � �

Boundary Conditions:

U13 = uw

U21 + U22 = us U21 + U22 = 2us

2 ⇒

Numerical Method:

I. a) Treat Nonlinear Terms

uux + vuy = (u2)x + (uv)y

uvx + vvy = (uv)x + (v2)y (use ux + vy = 0)

1 2

∂(U2) (Ui+ ,j )2 − (Ui− ,j )

2 1 2 Uij=

∂x Δxij

1 2

1 2

1 2

1 2

1 2

1 2

U 1i,j+2 Vi,j+ − Ui,j− Vi,j−

1 2

∂y ij Δy

Ui+ ,j Vi+ ,j − Ui− ,j Vi− ,j

∂(UV ) =

Ui,j+ 1 2∂(UV )

=

1 2

1 2

∂x Δxij

∂(V 2) (Vi,j+ )2 − (Vi,j− )2

= ∂(UV )∂y ij Δy ∂y ij

Ui,j + Ui+1,j Ui,j + Ui,j+1where Ui+ Ui,j+= =1 1,,j 2 22 2

U∗ i,j ∂(U2) n

∂(UV ) n i,j − Un

Δt = −

∂x i,j

− ∂y i,j

Vi,j ∗ − Vi,j

n ∂(UV ) n ∂(V 2) n

Δt = −

∂x i,j

− ∂y i,j

3

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 108: MIT Numerical PDE

I. b) Implicit Diffusion

U∗∗ − U∗

=1

K2D U∗∗

Δt Re ·

V ∗∗ − V ∗

=1

K2D V ∗∗

Δt Re ·

↑5 point Laplace stencil with Dirichlet boundary conditions

II. Pressure Correction

1 �

∂U∗∗ ∂V ∗∗ �

K2D · P = Δt ∂x

+ ∂y

∂p with Neumann boundary conditions = 0

∂n

U ∂U ∂x

Un+1 − U∗∗ ∂P Δt

= − ∂x

V n+1 − V ∗∗ ∂P Δt

= − ∂y

4

Image by MIT OpenCourseWare.

Page 109: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 110: MIT Numerical PDE

� �� �

� � � �

18.336 spring 2009 lecture 24 05/07/09

Pseudospectral Methods

Vorticity: � uw = �× �1

�ut + (�u · �)�u = −�p + Re�2�u

1 u) = �× (−�p)⇒ �× �ut + �× ((�u · �)� � �� �

+ �× (Re�2�u)

=0 �× and �2 commute

1 w� t +�u·� w� =

Re�2 w�⇒

Navier-Stokes equations in vorticity formulation

How to find �u from w�?

2D w = vx − uy scalar

Stream Function: � �u = ψyψ(�x) s.t. �u = �⊥ψ, i.e. v = −ψx

⇒ �2ψ = ψxx + ψyy = −vx + uy = −w

Thus: w �2ψ=−w

ψ �u=�⊥ψ

�u−→ −→

Remark: Incompressibility guaranteed � · �u = ux + vy = ψyx − ψxy = 0 �

Time Discretization

Semi-implicit:

• �u · �w explicit 2w Crank-Nicolson• �

nwn+1 − w n 1 1 � 2 n 2 n+1

Δt + (�

�u · ��� w n�) =

Re · 2 � w + � w

=Nn

1 1 2 n+1 1 1 2 n ⇒ ΔtI −

2Re� w = −Nn +

ΔtI +

2Re� w

↑If discretized in space linear system →Here: want to use spectral methods

1

Page 111: MIT Numerical PDE

� � � �

� �

� �

Fourier Transform Properties:+∞

F (f) = f(k) = f(x)e−2πikxdx

F −1(f) = f(x) = �−∞

+∞

f(k)e 2πikxdk

F (f �) = 2πikF (f) −∞

F (�2f) = −4π2|�k|2F (f)

F (f ∗ g) = F (f) · F (g)

← 2D FT (|�k|2 = kx 2 + ky

2)

F (f · g) = F (f) ∗ F (g)

Use Here:

1 4π2

+Δt 2Re

4π2

|�k 2 wn+1 = −N� n +1 |�k|2 wn|

Δt −

2Re

1 2π2 �|kNn +− ˆ 2 nw

Δt − |

Rewn+1 = ⇒

1 2π2

Δt +

Re |�k|2

ˆComputation of N :

N = � N = �u · �w ⇒ u ∗ �w

Inefficient in Fourier space.So perform multiplication in physical space.

Numerical Method:

Time step wn wn+1:→

1ˆn1. ψ ˆ−= w

⎧ ⎪⎪⎨ ⎪⎪⎩

⎧ ⎪⎪⎨ ⎪⎪⎩

n

|�k| un = −2πikyψ

n

vn = 2πikxψn

2. wx

n = −2πikxwn

wyn = −2πikyw

n

nunv

3. iFFT →wx

n

nwy

4. Nn = un wxn + vn wy

n · ·

(Nn)∗5. FFT →

6. Truncate kx, ky > 32 kmax

to prevent aliasing

Nn =(N)∗

if kx, ky ≤

32 kmax

0 else

7. Nn+1 +

1 2π2 �k 2 wn− ˆ

Δt −

Re | |

wn+1 = 1 2π2

2

Δt +

Re |�k|

2

Page 112: MIT Numerical PDE

� � �

� �

Particle Methods

Linear advection

ut + cux = 0u(x, 0) = u0(x)

Solution:u(x, t) = u0(x − ct)Need to work hard to get good schemes for advection, on a fixed grid.

Reason: Represent sideways motion by up and down motion.

Alternative: Move computed nodes, rather than having a fixed grid.

Ex.: 1D Advection

ut + cux = 0u(x, 0) = u0(x)

Particle method: 1. Sample initial conditions xj = jΔx, uj = u0(xj ) 2. Move particles along characteristics:

xj = c xj (t) = xj (0) + ct uj = 0

⇒ uj (t) = u0(xj (0))

→ Exact solution on particles.

Ex.: 2D Advection

φt + �v(�x) · �φ = 0φ(�x, 0) = φ0(�x)

Particle method:

1. �xj , φj = φ0(�xj )

�xj = �v(�xj )2. φ j = 0

Solve characteristic ODE, e.g. by RK4; very accurate on particles.

3

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 113: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

Page 114: MIT Numerical PDE

� �

18.336 spring 2009 lecture 25 05/12/09

Ex.: Continuity Equation Traffic on road ρ(t) + (v(x)ρ)x = 0 ⇔ ρt + v(x)ρx = −v�(x)ρ

Characteristic ODE⇒

xj = v(xj) ρj = −v�(xj )ρj

Ex.: Nonlinear conservation law

v = v(x)

ut + (1 2u2)x = 0 Burgers’ equation

xj = uj if solution smooth ⇒ uj = 0

Problem: Characteristic curves can intersect Particles collide ⇔

Particle management required:Merge colliding particles (how?)• Insert new particles into gaps (where/how?) •

An Exactly Conservative Particle Method for 1D Scalar Conservation Laws

[Farjoun, Seibold JCP 2009]

ut + (1 2u2)x = 0

Observe: If we have a piecewise linear function initially, then the exact solution is a piecewise linear function forever (including shocks).

1

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 115: MIT Numerical PDE

� �

Two choices: (A) Move shock particles using Rankine-Hugoniot condition.

(B) Merge shock particles, then proceed in time.

Same for insertion:

Ex.: Shallow water equations ⎧ ⎪⎨Dh Dt

= −uxh ⎫ ⎪⎬

ht + (uh)x = 0 ut + uux + ghx = 0

⇔ ⎪⎩ Du= −ghx

Dt ⎪⎭

⎧⎨

Df Lagrangian derivative = ft + u fx

Dt ·

Particle Method: ⎫⎬xj = uj

hj = −hj (∂xu)(xj )uj = −g(∂xh)(xj)

⎩ ⎭

Required: Approximation to ∂xh, ∂xu at xj

Particles non-equidistant.

2

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 116: MIT Numerical PDE

Meshfree approximation (moving least squares):

Local fit: u(x) = ax2 + bx + c

Weighted LSQ-fit: mina,b,c

� |u(xj ) − u(xj )|2

j:|xj −x0|≤r w(xj − x0)

w(x) = r1 or = e−αr or . . . α

Define (∂xu)(xj ) = u�(x0)

Smoothed Particle Hydrodynamics (SPH)

Quantity f

f(x) = f(x)δ(x − x)dxRd

Sequence of kernels W h

lim W h(x) = δ(x), W h(x)dx = 1 ∀h. h→0 R

Also:

W h(x) = wh(||x||),w(d) = 0 ∀d > h smoothing length←

Approximation I:

fh(x) = f(x)W h(x − x)dxRd

lim fh(x) = f(x).⇒ h 0→

Density ρ � Density measure µρ(A) = ρdx � A

f(x)fh(x) =

Rd ρ(x) W h(x − x) � ρ(x��)dx�

=dµρ(x)

3

Image by MIT OpenCourseWare.

Image by MIT OpenCourseWare.

Page 117: MIT Numerical PDE

Sequence of point clouds {X(n)}h∈N

X(n) = (x1(n), . . . , xn

(n))

Point measure n

δX(m) = mi (n)δ

x(n) −→ µρ

i=1 i n→∞

f(x)Approximation II: fh(x) ≈

ρ(x) W h(x − x)dδX(n)(x)

n

(n)= �

mi f(xi

(n))W h(x − xi

(n)) =: fh,n(x)ρ(xi

(n))i=1

n

⇒ �fh,n(x) = �

mi (n) f(xi

(

(

n

n

)

)

)

) �W h (x − xi (n))

ρ(xi� �� �

i=1 hard-code n

h mi hfk = fiWkiρii=1

n

�fkh =

mi fiWki

h

ρii=1

Apply to Euler equations of compressible gas dynamics: ⎧ ⎫ ⎧ ⎫

density

velocity

⎪⎪⎪⎪⎪⎪⎨ ⎪⎪⎪⎪⎪⎪

∂ρ ∂t D�u Dt

=

=

−� · (ρ�u)

−�p ρ

⎪⎪⎪⎪⎪⎪⎬ ⎪⎪⎪⎪⎪⎪⇔

⎪⎪⎪⎪⎪⎪⎨ ⎪⎪⎪⎪⎪⎪

Dρ Dt D�u Dt

=

=

�u · �ρ − � · (ρ�u)

−�

� p ρ

� p ρ2

�ρ � � � �

⎪⎪⎪⎪⎪⎪⎬ ⎪⎪⎪⎪⎪⎪De p De p p�u energy ⎩

Dt = −

ρ� · �u ⎭ ⎩

Dt = u · �

ρ − � ·

ρ ⎭

⎧ ⎪⎪⎪⎪⎪⎪⎨ → ⎪⎪⎪⎪⎪⎪⎩ ⎧ ⎪⎪⎪⎪⎪⎪⎨ ⇔ ⎪⎪⎪⎪⎪⎪⎩

ρk

uk

ek

ρk

�uk

ek

p = p(s, e)� � ⎫ = �uk mi�Wki − miu�i

h �Wki ⎪⎪⎪⎪⎪⎪�i i �mi pi pk ⎬ = −

ρi ρi �Wki −

(ρk)2 mi�Wki

i i ⎪⎪⎪⎪⎪⎪� mi pi � mi pi�vi

= �uk ρi ρi

�Wki − ρi ρi

�Wki ⎭ i i � ⎫

= mi(�uk − �ui)�Wki ⎪⎪⎪⎪⎪⎪i � �� ρk pi ⎬

= − mi (ρk)2

+(ρi)2

�Wki ⎪⎪⎪⎪⎪⎪� i pi

= mi (ρi)2

(�uk − �ui)�Wki ⎭ i

4

Image by MIT OpenCourseWare.

Page 118: MIT Numerical PDE

� �

� � �

d � �Since ρi = 0 ρk = miWki

dt ⇒

i i

SPH Approximation to Euler Equations:

mk = 0 (or = 0 if adaptive)

�xk = �uk

�uk = − mi (ρ

p

k

k

)2 +

p

i

i

)2 �Wki

i

pi ek = mi

(ρi)2 (�uk − �ui)�Wki

i

ρk = miWki

i

pk = p(ρk, ek)

5

Page 119: MIT Numerical PDE

MIT OpenCourseWarehttp://ocw.mit.edu

18.336 Numerical Methods for Partial Differential Equations Spring 2009

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.