Introduction to Finite Differences U - Isaac Newton … to Finite Differences Consider the heat...

Introduction to Finite Differences

Consider the heat equation on a finite interval subject to Dirichlet boundary conditions and arbitrary (i.e.user specified) initial conditions:

PDE: ∂u

∂t=

∂2u

∂x2, 0 ≤ x ≤ `, t > 0

BC: u(0, t) = γ0 and u(`, t) = γ`, t > 0

IC: u(x, 0) = u0(x), 0 ≤ x ≤ `.

Assume the γ’s are time-independent.

Space-time discretization

We need a numerical representation for the function u(x, t) and the for operators ∂∂t and ∂2

∂x2 .

In the simplest case the dependent variable u is represented by values on a uniform grid or lattice in spaceand time.

• Divide the interval [0, `] into J equally spaced intervals of size 4x or h. Hence h = 4x = `/J .There are a total of J + 1 gridpoints labeled xj . For the uniform grid xj = j4x, j = 0, J

• n labels timesteps, tn denotes discrete time values, and 4t is the timestep. When needed, t0 and tfwill denote the initial and final times and N will denote the total number of time steps. Usually t0 = 0and tn = n4t.

1

2

Also we frequently need N to represent other quantities and only seldom do we need a notation forthe total number of timesteps.

• U denotes the numerical approximation to u. U nj will denote the numerical solution at timestep n and

gridpoint j. If the numerical solution exactly agrees with the true solution, then U nj = u(xj , tn). One

goal, though not the only one, of a numerical scheme is to make

limh,4t→0

|Unj − u(xj , tn)| = 0

Explicit Euler solution

Basic methodology of finite-difference schemes - approximate the derivatives appearing in the partial dif-ferential equation with combinations (differences) of the values on the grid. Here we consider the simplestcase.

Approximate time derivative by the forward difference:

∂u

∂t|j,n =

∂u

∂t(xj, tn) '

Un+1j − Un

j

4t

Approximate space-derivative by the second-order center difference:

∂2u

∂x2|j,n =

∂2u

∂x2(xj, tn) '

Unj−1 − 2Un

j + Unj+1

h2

The heat equation becomes:Un+1

j − Unj

4t=

Unj−1 − 2Un

j + Unj+1

h2

Solving for Un+1j :

Un+1j = Un

j + ν(Unj−1 − 2Un

j + Unj+1)

where ν ≡ 4t/h2.

This is an explicit expression for the U ’s at the n + 1st timestep in terms of the U ’s at the nth timestep.Taking into account the BCs and IC we have the explicit, or forward, Euler scheme for the heat equation:

U0j = u0(xj), 0 ≤ j ≤ J

Un+1j =

γ0, j = 0Un

j + ν(Unj−1 − 2Un

j + Unj+1), 0 < j < J,

γ`, j = J

At each new timestep, the dependent variable at each interior grid point is computed from values at threegridpoints at the preceding timestep.

Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley

3

More on finite-difference formulas

Difference operators

The following difference operators are frequently useful:

• forward difference:

∆+tu(x, t) ≡ u(x, t + 4t) − u(x, t)

∆+xu(x, t) ≡ u(x + 4x, t) − u(x, t)

• backward difference:

∆−tu(x, t) ≡ u(x, t) − u(x, t −4t)

∆−xu(x, t) ≡ u(x, t) − u(x −4x, t)

• central difference:

δtu(x, t) ≡ u(x, t +1

24t) − u(x, t −

1

24t)

δxu(x, t) ≡ u(x +1

24x, t) − u(x −

1

24x, t)

From this follows the second-order central difference:

δ2xu(x, t) ≡ u(x −4x, t) − 2u(x, t) + u(x + 4x, t)

• central difference using double interval:

∆0xu(x, t) ≡1

2(∆+x + ∆−x)u(x, t) =

1

2(u(x + 4x, t) − u(x −4x, t))


4

Derivation of finite-difference formulas

Suppose we want to approximate ∂ru∂xr to some order of accuracy as a weighted sum of values of u on a

(possibly non-uniform) grid . . . , xj−2, xj−1, xj , xj+1, xj+2, . . .:

∂ru

∂xr(xj) '

∑

k

wkuj+k

where uj+k = u(xj+k). How are the weights wk chosen?

The trick is to express all the uj+k = u(xj+k) as Taylor series expansions about point xj:

uj = uj

uj+1 = uj +∂u

∂x|j(xj+1 − xj) +

∂2u

∂x2|j

(xj+1 − xj)2

2+ . . .

uj−1 = uj +∂u

∂x|j(xj−1 − xj) +

∂2u

∂x2|j

(xj−1 − xj)2

2+ . . .

...

where |j means evaluated at xj . Multiplying each equation by corresponding weight and summing we have:

∑

k

wkuj+k = uj

∑

k

wk +∂u

∂x|j

∑

k

wk(xj+k − xj) + . . .∂ru

∂xr|j

∑

k

wk(xj+k − xj)

r

r!+ . . .

The left-hand-side is the sum that we want to approximate ∂ru∂xr . To achieve this for generic functions u, we

must have all terms preceding ∂ru∂xr on the-right-side be zero and the coefficient of ∂ru

∂xr be one. In additionwe would like as many terms as possible following ∂ru

∂xr also to be zero. This is satisfied by choosing theweights such that:

∑

k

wk = 0

∑

k

wk(xj+k − xj) = 0

...∑

k

wk(xj+k − xj)

r

r!= 1

∑

k

wk(xj+k − xj)

r+1

(r + 1)!= 0

...

These equations are linear in the weights and thus can be solved for the wk in terms of the differencesxj+k − xj .

The number of gridpoints necessary to approximate a derivative depends on the order r of the derivative andthe number of high-order terms in the Taylor series that one want to make zero. Frequently on uniform gridssymmetries enter such that with q weights it is possible to satisfy more than q conditions.


Error Analysis

The analysis of errors in numerical schemes is important for the following reasons:

• It tells us where errors come from and where we should concentrate efforts to reduce errors.

• It allows comparison of different schemes.

• It provides a powerful basis for testing programs.

5

6

Roundoff Error

Roundoff errors arise due to finite precision computations. The one rule with regard to roundoff is:

Where possible avoid subtracting nearly equal numbers.

This rule implies, for example, that one should not take tan(x) for large x.

Truncation Error

Truncation error T (x, t) is the error in approximating differential operators and PDEs by discrete represen-tations such as finite differences.

For a PDE written as Fu = 0,

T (x, t) = FAu(x, t)

where FA is the approximation to the differential equation and u is an exact solution to the PDE.

Example: The truncation error for the explicit Euler scheme for the heat equation is

T (x, t) =u(x, t + 4t) − u(x, t)

4t−

u(x − h, t) − 2u(x, t) + u(x + h, t)

h2

=∂u

∂t(x, t) −

∂2u

∂x2(x, t) +

4t

2

∂2u

∂t2(x, t) −

h2

12

∂4u

∂x4(x, t) + . . .

=4t

2

∂2u

∂t2(x, t) −

h2

12

∂4u

∂x4(x, t) + . . .

soTh,4t(x, t) = O(4t) + O(h2)

We assume |∂2u

∂t2 | and |∂4u

∂x4 | are bounded in the space-time domain of interest. The truncation error goes tozero everywhere in the domain as 4t and h go to zero. Hence the explicit Euler approximation is consistentwith the partial differential equation.

Discretization Error

The discretization error enj at a point of the computational grid is the difference between the numerical and

exact solutions:

enj ≡ Un

j − u(xj , tn)


7

where u and Unj satisfy the same initial conditions, i.e. U 0

j = u(xj , 0).

Frequently we are interested in:

Ef ≡ maxj|ej |, for fixed final time tf

also called the discretization error (though usage varies).

For the explicit Euler solution of the heat equation this can be bounded using the truncation error as long asν ≤ 1/2.

Ef ≤ tf T .

where T is a bound on the truncation error: |T nj | ≤ T .

Ef → 0, for 4t, h → 0 such that ν = 4t/h2 ≤ 1/2

The numerical solution is said to converge to the exact solution.

Our main interest is the scaling of discretization error with 4t and h:

Ef = |O(4t) + O(h2)| 1

We can investigate the final error Ef (at a fixed final time) as a function of 4t and h. To make thisprecise, we need a refinement path = (hi,4ti), i = 0, 1, 2, . . . with hi and 4ti going to zero such thatνi = 4ti/h

2i ≤ 1/2. We compute the error Ef at points on the refinement path and verify the scaling of

Ef . In practice for testing it is better to vary only one of 4t and h while holding the other constant.

The scaling of the discretization error with 4t and hprovides a valuable check on the correctness of a program.

1Note: This is slightly subtle in that the discretization error for one-step is 2nd order in 4t, i.e.

E1 ≤ 4t|O(4t) + O(h2)|

For fixed final time tf the number of timesteps N necessary to reach tf increases as 1/4t decreases (N ∼ 1/4t). This cancelsthe leading 4t in the one-step error. Because N ∼ 1/4t we cannot consider E = EN with fixed N but instead we consider afixed final time and hence the notation Ef .


8


Stability and Implicit Methods

Motivation

In the analysis of discretization error in the explicit Euler method for the heat equation, we needed to assumeν = 4t/h2 ≤ 1/2 in order to obtain bounds on the error. Consider now what happens to the numericalsolution using the explicit Euler method when ν = 4t/h2 > 1/2. After a relative small number of timestepsthe solution develops a numerical instability which grows exponentially with the number of timesteps.

This purpose of this chapter is to understand why this happens and to introduce stable (implicit) methods toeliminate this problem.

ODEs

Most of the issues can be understood from the simplest ODE:

u = λu,

9

10

We focus on λ real, but one can also consider complex λ.

Compare the exact solution with a variety of timestepping schemes


11

• Exact solution:

u(t) = u(0)eλt

• The explicit or forward Euler timestepping:

Un+1 − Un

4t= λUn ⇒ Un+1 = (1 + λ4t)Un

• The implicit or backward Euler timestepping:

Un+1 − Un

4t= λUn+1 ⇒ Un+1 =

1

1 − λ4tUn

• The Crank-Nicolson scheme is given by an average of the explicit and implicit schemes:

Un+1 − Un

4t=

λUn + λUn+1

2⇒ Un+1 =

1 + (4t/2)λ

1 − (4t/2)λUn

• All three schemes can be generalized to an arbitrary weighted average:

Un+1 − Un

4t= (1 − θ)λUn + θλUn+1 ⇒ Un+1 =

1 + (1 − θ)λ4t

1 − θλ4tUn

where 0 ≤ θ ≤ 1. This is called the weighted average or θ-method.

Aside on terminology: Implicit timestepping refers to the fact that U n+1 appears on the right-hand-side ofthis equation. Hence it is an implicit equation for U n+1, the value of u at the next timestep, in terms of U n,the known value at the current timestep. For this simple ODE it is trivial to solve for U n+1, but for generalODES this would not be so.

The terms forward and backward come about from a slightly different way of viewing these schemes. Ifthe time appearing on the right-hand-side is taken as the reference point, then the derivative on the left-hand-side is a forward difference or a backward difference.

One-step analysis

For each of the three schemes let us consider the dependence of U n+1/Un on 4t and compare this with theexact expression for u(t + 4t)/u(t). The results are shown graphically for the case λ = −1.


12

We make the following observations.

• All three schemes approximate the exact, exponential, relation for small 4t.

• The forward Euler seems particularly bad for moderate and large 4t. Importantly note that |U n+1| >|Un| for 4t > 2.

• In many ways the backward Euler scheme appear to be the best because is never gets too far from theexact solution and has the correct limit for large 4t. Note, however, that for small 4t the backwardEuler scheme approximates the exact to the same accuracy as forward Euler.

• The Crank-Nicolson scheme is a better approximation to the exact solution for small 4t than eitherof the other two methods. That is the Crank-Nicolson curve is seen to follow the exact solution betterat small 4t. |Un+1| < |Un| for all 4t.

For small 4t expand each form in Taylor’s series to obtain:

Exact:u(t + 4t)

u(t)= eλ4t = 1 + λ4t +

1

2λ24t2 +

1

6λ34t3 + . . .

FE:Un+1

Un= 1 + λ4t = 1 + λ4t

BE:Un+1

Un= (1 − λ4t)−1 = 1 + λ4t + λ24t2 + . . .

CN:Un+1

Un= (1 − λ4t

2 )−1(1 + λ4t2 ) = 1 + λ4t +

1

2λ24t2 +

1

4λ34t3 + . . .

Hence we see that FE and BE agree with Exact to O(4t) and makes an error O(4t2), whereas CN agreeswith Exact to O(4t2) making an error O(4t3). Note that these are the single-timestep errors which are onepower of 4t higher than the final truncation or discretization error.


13

N -step analysis

Consider now the result of repeatedly timestepping each scheme:

Un = αnU0

This issue is,

is |α| ≤ 1 whenever λ ≤ 0, i.e. are numerical solutions bounded whenever exact solutions are?

• For BE and CN methods one can easily establish that when λ < 0, |α| ≤ 1 for 0 ≤ 4t < ∞.BE and CN are unconditionally stable because there is no requirement on 4t for stability.

• For the FE case, however, it is evident that |α| exceeds one and thus |U n| will diverge as n → ∞.This is the essence of the numerical instability illustrated at the beginning of the chapter.The explicit Euler scheme is conditionally stable with stability limit

4t ≤2

−λ

Because instability occurs for α < −1, it is oscillatory in time with U n+1 and Un of opposite signs.This is one of the key signatures of numerical instabilities.


14

PDEs

Notation

• Un will be the vector representing the numerical solution at timestep n:

Un = (Un0 , Un

1 , Un2 , . . . , Un

j , . . . , UnJ−1, U

nJ )T

• L will be the matrix representing the finite-difference representation of a linear operator L. For nowwe consider L = ∂2

∂x2 and

L =1

h2

. . . . . . . . .1 −2 1

. . . . . . . . .

• I will denote the identity matrix.

With this notation we can easily express the 3 timestepping schemes as follows:

FE: Un+1 −Un

4t= L Un ⇒ Un+1 = (I + 4tL) Un

BE: Un+1 −Un

4t= L Un+1 ⇒ Un+1 = (I−4tL)−1 Un

CN: Un+1 −Un

4t=

L Un + L Un+1

2⇒ Un+1 = (I−

4t

2L)−1 (I +

4t

2L) Un

All the above timestepping schemes are of the form:

Un+1 = A−1− A+ Un = A Un.

whereA+ ≡ (I + (1 − θ)4tL) A− ≡ (I− θ4tL).

Stability analysis

All the above timestepping schemes are of the form:

Un+1 = A Un.

The stability analysis is reduced to determining whether the matrix A has any eigenvalues such that |α| > 1.

The von Neumann or Fourier method of stability analysis is based on choosing the eigenvectors of A tobe trigonometric:

φj = cos(βjh)


15

orφj = eiβjh

whichever is easier to work with. β is continuous and can take on any value. In this approach the corre-sponding values of α are simply computed.

The eigenvalues α of A are related to the eigenvalues λ of L:

FE: α = 1 + λ4t

BE: α = (1 − λ4t)−1

CN: α = (1 − λ4t

2)−1(1 + λ

4t

2)

Find the eigenvalue of L:

λφj =1

h2(φj−1 − 2φj + φj+1)

λeiβjh =1

h2(eiβ(j−1)h − 2eiβjh + eiβ(j+1)h)

λ =1

h2(e−iβh − 2 + eiβh)

λ =2

h2(cos βh − 1)

The eigenvalues of A for the forward Euler method are:

α = 1 + 4tλ = 1 +24t

h2(cos βh − 1)

Note that (cos βh − 1) ≤ 0. Hence the only possibility for instability occurs for α < −1. For any 4t, theminimum (most negative) value of α occurs for βh = π giving:

αmin = 1 −44t

h2

For explicit Euler scheme to be stable, αmin must be greater than −1. This is equivalent to the condition

4t ≤h2

2or ν =

4t

h2≤

1

2

• The explicit Euler scheme is conditionally stable.

• The reader can easily show that the BE and CN schemes are unconditionally stable.

Form of the numerical instability. As before the instability occurs for α < −1 and hence when instabilitydevelops it is oscillatory in time with U n+1

j and Unj of opposite signs. However, now there is the additional

dependence on space given by the eigenvector φj = cos(βjh) (now I use the cosine form). The instabilityarises first from the eigenvector with βh = π or φj = cos(πj) = (−1)j . This normal mode changes sign inspace at every other gridpoint. Hence when the instability develops it is oscillatory in space as well as time.Again this is one of the key signatures of a numerical instability.


16


Boundary Conditions

We shall consider the following boundary conditions on the interval [0, `]:

α0(t)u(0, t) + β0(t)∂u

∂x(0, t) = γ0(t), α0, β0 ≥ 0

α`(t)u(l, t) + β`(t)∂u

∂x(l, t) = γ`(t), α`, β` ≥ 0

These boundary conditions are Dirichlet if β = 0 and Neumann if α = 0 and Robin if both α and β arenonzero.

We consider methods for dealing with these cases which have the advantage that the same J + 1 spatialpoints xj with x0 = 0 and xJ = ` are used in each case.

Computationally for (1) and (1) there are two distinct cases: β = 0 and β 6= 0.

Dirichlet case: β = 0

We consider in detail the left boundary only, the right boundary follows similarly.

Without loss of generality we can write the left BC as:

u(0, t) = γ0(t)

since α0 cannot be zero if β0 is.

Eq. () dictates the numerical solution at x = 0 for all time: U n0 = γn

0 ≡ γ0(n4t). Thus Un0 does not need

to be computed from the PDE and and we need not (and cannot) impose the PDE at j = 0.

The following provides a simple implementation. Consider the heat equation and let L be the discretizationof the second derivative. Put zeros in the rows of matrix L at Dirichlet boundary points

L =1

h2

0 01 −2 1

1 −2 1. . .

So [LU]0 = 0, [LU]1 = (U0 − 2U1 + U2)/h2. You may think of this as not imposing the PDE at the

boundary.

17

18

Next define the boundary operator Bn by:

Bn U = Bn

U0

U1

. . .

=

γn0

U1

. . .

Bn sets the boundary values to the corresponding BC at time t = n4t. Note: B is not a linear operator ifγ0 6= 0.

Using this operator the three timestepping schemes can be written:

FE: Un+1 = Bn+1 (I + 4tL) Un

BE: Un+1 = (I−4tL)−1 Bn+1 Un

CN: Un+1 = (I−4t

2L)−1 Bn+1 (I +

4t

2L) Un

Neumann/Robin case: β 6= 0

Without loss of generality we can write the left BC as:

∂u

∂x(0, t) = α0(t)u(0, t) + γ0(t)

since β0 6= 0.

In this case we do not know Un0 directly from the boundary condition, and must find U n

0 by imposing thePDE at the boundary.

However, to use the second-order centered-difference approximation for ∂2u∂x2 at grid point j = 0 would

require “U−1”, i.e.δ2x

h2U0 =

1

h2(U−1 − 2U0 + U1)

and U−1 does not exist. Here we consider a heuristic approach in which () is used where U−1 is obtainedfrom boundary condition ().

To obtain U−1 from the BC use the second-order centered-difference representation for the first derivativeat the boundary:

∂u

∂x(0, t) =

1

2h(U1 − U−1) + O(h2)

Plugging this into the boundary condition () gives:

1

2h(U1 − U−1) = α0U0 + γ0

Solving for U−1:

U−1 = U1 − 2hα0U0 − 2hγ0


19

Then

δ2x

h2U0 =

1

h2(U1 − 2hα0U0 − 2hγ0 − 2U0 + U1)

=1

h2(−2(1 + hαn

0 )Un0 + 2Un

1 ) −2

hγn0

We define Ln by:

Ln =1

h2

−2(1 + hαn0 ) 2

1 −2 11 −2 1

. . .

and define the boundary operator Bn in this case to be:

Bn U = Bn

U0

U1

. . .

=

U0

U1

. . .

+

−24th γn

0

0

. . .

where αn0 ≡ α0(n4t), and as before γn

0 = γ0(n4t). Ln and Bn depend on n (time) if α and γ do.

The three timestepping schemes are then:

FE: Un+1 = Bn (I + 4tLn) Un

BE: Un+1 = (I−4tLn+1)−1 Bn+1 Un

CN: Un+1 = (I−4t

2Ln+1)−1 Bn+1/2 (I +

4t

2Ln) Un

where Bn+1/2 is defined by:

Bn+1/2 U =1

2(Bn+1 + Bn) U =

U0

U1

. . .

+

−4th (γn

0 + γn+10 )

0

. . .

We will frequently write B for the appropriately defined boundary condition operator. The forms of thisoperator, and the timestep n at which it is evaluated depends on the boundary conditions and the methodunder consideration.


20

Periodic BCs

Consider periodic boundary conditions imposed on the grid with x0 = 0 and xJ = `. Then u(0, t) = u(`, t)gives Un

0 = UnJ . The PDE is easily imposed at all grid points. For example,

∂2u

∂x2(0, t) =

1

h2(Un

−1 − 2Un0 + Un

1 ) + O(h2)

=1

h2(Un

J−1 − 2Un0 + Un

1 ) + O(h2)

Thus the matrix L has the structure:

L =1

h2

and is not tridiagonal, nor is it banded. For implicit methods this is a problem because the work required toinvert A− = (I−4tL) is proportional to the square of the bandwidth.

However, A− is sparse – it contains mostly zero entries and it differs in only 6 places from a tridiagonalmatrix. To invert such a matrix, one can turn to the Sherman-Morrison formula.

IfA = A + u⊗ v

thenA−1 = A−1 −

(A−1 · u) ⊗ (v · A−1)

v · A−1 · u

If u and v are unit vectors, then A differs from A in exactly one entry. Hence, A−1− can be written in terms

of the inverse of a tridiagonal matrix together with 6 applications of the Sherman-Morrison formula.


General Linear Parabolic Eq in 1D

Most of the difficulties encountered in going to the general case are in the analysis of the discretizationerror and stability. Implementing the general case can be a simple extension of the constant coefficientheat equation.

Write the PDE as:∂u

∂t(x, t) = L(x, t)u(x, t) + g(x, t)

then discretize all quantities in space and time:

u(x, t) → Un =

Un0

Un1

. . .

UnJ

L(x, t) → Ln g(x, t) → gn =

gn0

gn1

. . .

gnJ

where gnj ≡ g(jh, n4t).

The matrix Ln can be obtained from finite-difference formulas for ∂2u∂x2 and ∂u

∂x .

The three timestepping schemes are then:

FE: Un+1 = B {(I + 4tLn) Un + 4tgn}

BE: Un+1 = (I−4tLn+1)−1 B{

Un + 4tgn+1}

CN: Un+1 = (I−4t

2Ln+1)−1 B

{

(I +4t

2Ln) Un +

4t

2(gn + gn+1)

}

where B represents the appropriate boundary operator.

21

22


Nonlinearity

Difficulties

Some of the numerical difficulties encountered with nonlinear equations are:

23

24

• Implicit methods are much more difficult to apply to nonlinear terms.Consider timestepping

∂u

∂t= F (u)

by implicit Euler method where F () is a nonlinear operator. Then:

Un+1 −Un

4t= F(Un+1) ⇒ (I−4tF)(Un+1) = Un.

Here (I−4tF) is a nonlinear operator. Solving this equation is not simply a matrix inversion.Implicit methods are not often used in treating nonlinearities in PDEs. Generally some sacrifices mustbe made in order of accuracy and/or stability.

• Relatively smooth initial conditions evolve into solutions requiring very fine spatial resolution.For example, solutions to Burgers equation

∂u

∂t= −u

∂u

∂x+ ε

∂2u

∂x2

can become locally very sharp.

• Testing is difficult because exact solutions are rare.

Some possible methods

• Explicit Euler method can always be used:

Un+1 = Un + 4tF(Un).

While not favored because it is low-order accurate (O(4t)), it is nevertheless a good method forgetting results quickly in a few cases.

• Bite the bullet and invert nonlinear operator, iteratively.

• Use Runge-Kutta or a multistep method.

We will consider one of the most common methods for treating nonlinear terms in PDEs. The nonlinearterms are treated explicitly, but more accurately than with the forward Euler method.

Multistep Methods for Nonlinear ODEs

Consider multistep methods for solving the nonlinear ODE:

du

dt= f(u).

These schemes advance solution for time t to time t + 4t using not only f(U n) and (possibly) f(Un+1),but also f(Un−1), f(Un−2), . . .


25

Generally:

Un+1 = Un + 4tk

∑

i=0

βif(Un+1−i)

If β0 = 0 then f(Un+1) is not used and the method is explicit. Of these the Adams-Bashforth schemes arethe most common.

Some cases are:

k = 1, β1 = 1 explicit Euler

k = 2, β1 = 32 , β2 = −1

2 2nd order Adams − Bashforth

k = 4, β1 = 5524 , β2 = −59

24 , β3 = 3724 , β4 = − 9

24 4th order Adams − Bashforth

In detail the second order Adams-Bashforth scheme is:

Un+1 = Un +4t

2

{

3f(Un) − f(Un−1)}

The one-step error is O(4t3) and discretization error is one power of 4t smaller, so Ef = O(4t2)).

Comments:

• The Adams-Bashforth methods give higher-order accuracy in time and yet are explicit.

• Numerical stability is an issue and for large k the timestep restriction is too severe for the schemes tobe practical.

• By saving previous values of f(Un), one need only evaluate f once per timestep. This is significantfor PDEs and is a significant advantage over other high-order methods such as Runge-Kutta in whichmultiple function evaluations are required at each timestep.

• The first timestep(s) cannot be multistep. Some number of steps must be made at lower-order accuracyor using another method, such as Runge-Kutta. When using second-order Adams-Bashforth, it iscommon to take one explicit Euler step at the beginning. This has no importance for the globaldiscretizaton error.

Nonlinear PDEs

Write the PDE as:∂u

∂t= L · u + N (u)

where L and N are linear and nonlinear operators respectively.

One can then treat the linear operator using Crank-Nicolson as before and treat the nonlinear operator using2nd order Adams-Bashforth. This has the advantage of achieving overall O(4t2) accuracy and allowingreasonable size for 4t. In detail:

Un+1 −Un

4t=

1

2

{

LUn+1 + LUn}

+1

2

{

3N(Un) −N(Un−1)}


26

where Nj(Un) is the numerical approximation to N (u) at grid point j.

Solving for Un+1 we obtain:

Un+1 = (I −4t

2L)−1

{

(I +4t

2L)Un +

4t

2

(

3N(Un) −N(Un−1))

}

With boundary conditions properly accounted for and with the possibility of an inhomogeneous term in theequation, this can be written:

Un+1 = (I −4t

2Ln+1)−1 B

{

(I +4t

2Ln) Un +

4t

2

(

3N(Un) −N(Un−1))

+4t

2(gn + gn+1)

}

where B is the boundary condition operator appropriate to the type of BC applied.

The contribution to the global error Ef from time discretization is O(4t2).

The scheme is not unconditionally stable. There is a maximum 4t, determined from N and U. For mostparabolic PDEs this restriction is less severe than the restriction 4t/h2 ≤ 1/2.


Introduction to Spectral Methods

The essential idea is to approximate the solution u(x, t) by an expansion in a finite set of spectral modesφm(x) and their time-dependent amplitudes am(t):

u(x, t) 'M∑

m=0

am(t)φm(x)

Trigonometric functions and orthogonal polynomials (Chebyshev or Legendre polynomials) are the mostfrequently used. The most natural indexing of modes is not always from 0 to M , and so the summationlimits will often not be given, but it is understood to be finite.

Spatial derivatives of u(x, t) are then expressed in terms of spatial derivatives of the φm, e.g.

∂2u

∂x2(x, t) '

∑

m

am(t)φ′′m(x)

So in the trigonometric case φm = eiβmx

∂2u

∂x2(x, t) '

∑

m

−β2am(t)φm(x)

and we have “exactly” differentiated our approximation.

• Orthogonal functions are to spectral methods as Taylor series are to finite-difference methods.

The treatment of time is the generally the same in both methods.

Advantages and Disadvantages

The greatest advantage of spectral over finite difference methods is the greater spatial accuracy. For smoothsolutions, the discretization error of a spectral method decreases faster than any power of the resolution M :

Ef = O(M−k) for every k.

The approximation is said to be exponentially accurate.

Recall, for second-order finite differences we found:

Ef = O(h2)

27

28

For a given number of grid points N = J we have h = O(1/N) so

Ef = O(N−2)

Comments

• The accuracy of spectral methods translates into fewer unknowns and thus greater speed and lessmemory for the same accuracy compared with FD methods.

• Given fast transforms, implicit methods can be efficiently implemented.

• Spectral methods typically produce smaller artificial dissipation and dispersion in comparison to FDmethods.

• Finite-difference and finite-element methods are generally more flexible. They are able to handlevariable coefficients, general boundary conditions, free-boundaries etc. In higher dimensions they areable to handle irregular geometries more easily than spectral methods.

Fourier Pseudospectral Timestepping

Pseudospectral methods can be applied to PDEs with inhomogeneities and nonlinearities. In the pseu-dospectral approach we have both a spatial grid (collocation points) and a spectral expansion. The numericalsolution can either be represented by the amplitudes am or by the values on the grid Uj . Operations such asdifferentiation are best carried out in spectral space (because the modes φm(x) can be differentiated exactlyand the derivative operator often local in spectral space). Complications arising from inhomogeneities andnonlinearities are best handled in physical space where they are local.

We consider in detail how to implement pseudospectral timestepping for problems with periodic boundaryconditions for which Fourier modes are the appropriate choice.

Amplitudes and Grid values

Let U denote the vector of values on a uniform grid xj = jh where h = `/J . Let a denote the vector ofamplitudes. The spectral expansion relates these

Uj =

J−1∑

m=0

amei2πmj/J j = 0, 1, . . . , J − 1,

Hence

am =1

J

J−1∑

j=0

Uje−i2πmj/J , m = 0, 1, . . . , J − 1,

Thus the amplitudes and grid values are simply related by the Discrete Fourier Transform (DFT) which shallbe written:


29

a = DFT U

U = DFT−1 a

There are a variety of conventions for ordering the Fourier amplitudes am because we are generally inter-ested in real-to-complex DFTs taking J real values Uj on a grid to J complex amplitude am = ar

m + iaim.

The property a∗m = a−m implies that the complex amplitudes are actually described by J real quantitiesand the highest spatial frequency is J/2.

We use a generic ordering (storage) for arm and ai

m and let a denote the real vector:

a = (ar0, a

r1, a

i1, a

r2, a

i2, . . . , a

rJ/2−1, a

iJ/2−1, a

rJ/2)

T

Necessarily ai0 = ai

J/2 = 0 and these are not included in the list. a is a real vector of length J .

In practice proper account must be taken of the ordering actually produced by the discrete Fourier transformfunction used. The amplitudes are generally not normalized and require an additional normalization (suchas division by J ).

Heat Equation

Consider timestepping the heat equation

∂u

∂t= Lu =

∂2u

∂x2

by the forward Euler method∂u

∂t'

Un+1 −Un

4t= LUn

then as usual this becomes:

Un+1 = (I + 4tL)Un = A+Un

L and A+ are not the tridiagonal matrices from the finite-difference approach.

The operators are applied in spectral space,

Un+1 = DFT−1 (I + 4tL) DFT Un

= DFT−1 A+ DFT Un

where they are diagonal

A+ ≡

1 −4tβ20

. . .1 −4tβ2

m

1 −4tβ2m

. . .1 −4tβ2

J/2

= diag{1 −4tβ2m}


30

We can write our timestepping scheme as:

Un DFT−→ an A+

−→ an+1 DFT−1

−→ Un+1

Backward Euler and Crank-Nicolson methods can be treated in exactly the same way. Note for example thatfor backward Euler:

Un+1 = DFT−1 (I−4tL)−1 DFT Un

= DFT−1 A−

−1 DFT Un

where

A− ≡

1 + 4tβ20

. . .1 + 4tβ2

m

1 + 4tβ2m

. . .1 + 4tβ2

J/2

= diag{1 + 4tβ2m}

Since A− is diagonal, finding its inverse is trivial: [A−

−1]jj = 1/A−jj .

Crank-Nicolson timestepping follows similarly.

Note that L, A+, etc. are diagonal only when the boundary conditions are periodic, permitting the use ofFourier expansions. For Dirichlet boundary conditions, for example, polynomial expansions are appropriate;the resulting matrices representing differential operators are banded or structured, but not diagonal.

Nonlinearity in the Pseudospectral Approach

Consider the PDE:∂u

∂t= Lu + N (u)

with L and N () linear and nonlinear operators, respectively. (Here we shall assume that N () does notdepend on ∂u

∂x , ∂2u∂x2 , etc. although this assumption in not essential.)

Discretize space and time in the usual way and timestep with Crank-Nicolson/Adams-Bashforth method:

Un+1 −Un

4t=

1

2

{

LUn+1 + LUn}

+1

2

{

3N(Un) −N(Un−1)}

where Nj(Un) is the numerical approximation to N (u) at grid point j.

Solving for Un+1 we obtain:

Un+1 = (I −4t

2L)−1

{

(I +4t

2L)Un +

4t

2

(

3N(Un) −N(Un−1))

}


31

We now wish to act with the linear operators in spectral space. Apply discrete Fourier Transform and itsinverse to the right-hand-side to obtain:

Un+1 = DFT−1 DFT (I−4t

2L)−1

{

(I +4t

2L)Un +

4t

2

(

3N(Un) −N(Un−1))

}

Un+1 = DFT−1 (I−4t

2L)−1

{

(I +4t

2L) DFT Un + DFT 4t

2

(

3N(Un) −N(Un−1))

}

where hats denote linear operators in spectral space.


32


Introduction to Finite Differences U - Isaac Newton … to Finite Differences Consider the heat...

Documents

Transcript of Introduction to Finite Differences U - Isaac Newton … to Finite Differences Consider the heat...