Notes Regarding the Karhunen–Loeve Expansion

1 Properties of the Karhunen–Loeve Expansion

As noted in Section 5.3 of [3], the Karhunen–Loeve expansion for a correlated second-order randomfield α(x, ω), with x ∈ D, mean α(x) and covariance function C(x, y), is

α(x, ω) = α(x) +∞∑n=1

√λnφn(x)Qn(ω). (1)

Here λn and φn are the eigenvalues and orthonormal eigenfunctions of C; that is, they solve theintegral equation ∫

DC(x, y)φn(y)dy = λnφn(x) (2)

for x ∈ D.To motivate (1) and (2), we start with Mercer’s Theorem, which states that because C(x, y) is

bounded, symmetric and positive definite, it has the spectral decomposition

C(x, y) =∞∑n=1

λnφn(x)φn(y)

where ∫DC(x, y)φn(y)dy = λnφn(x).

The eigenfunctions are orthogonal and form a complete set so∫Dφn(x)φm(x) = δmn. (3)

To determine statistical properties of the component of α(x, ω) quantified by the covariancefunction C(x, y), we consider

α(x, ω) = α(x) + β(x, ω)

where β(x, ω) has zero mean and covariance function C(x, y). We employ the expansion

β(x, ω) =∞∑n=1

Qn(ω)√λnφn(x)

so that

β(x, ω)β(y, ω) =

∞∑n=1

∞∑m=1

Qn(ω)Qm(ω)√λnλmφn(x)φm(y).

Since β(x, ω) has zero mean, it follows that

C(x, y) = E(β(x, ω)β(y, ω))

= 〈β(x, ω)β(y, ω)〉

=

∞∑n=1

∞∑m=1

〈Qn(ω)Qm(ω)〉√λnλmφn(x)φm(y)

1

where 〈X〉 denotes the expectation E(X). Since the eigenfunctions are orthonormal, it follows that

λkφk(x) =

∫DC(x, y)φk(y)dy

=

∞∑n=1

〈Qn(ω)Qk(ω)〉√λnλkφn(x).

We then multiply by φ`(x), integrate, and again invoke orthogonality to obtain

λk

∫Dφk(x)φ`(x)dx =

∞∑n=1

〈Qn(ω)Qk(ω)〉√λnλkδn`

⇒ λkδk` =√λkλ` 〈Qk(ω)Q`(ω)〉 .

Sincek = ` ⇒ 〈Qk(ω)Q`(ω)〉 = 1

k 6= ` ⇒ 〈Qk(ω)Q`(ω)〉 = 0,

it follows that〈Qk(ω)Q`(ω)〉 = δk`.

Hence the random variables satisfy the relations

E(Qn) = 0 , E(QnQm) = δmn. (4)

2 Choices for the Correlation Function C(x, y)

Two common choices for the correlation function are

C(x, y) = e−|x−y|/L , D = [−1, 1], (5)

where L is the correlation length, and

C(x, y) = min(x, y) , D = [0, 1], (6)

which arises when studying 1-D Weiner processes or Brownian motion. One reason these are com-monly considered is because both have analytic solutions.

Consider first (6). From the relation∫ 1

0min(x, y)φn(y)dy = λnφn(x),

we obtain the boundary condition φn(0) = 0. For fixed x, we note that∫ x

0yφn(y)dy + x

∫ 1

xφn(y)dy = λnφn(x)

which, when differentiated to obtain ∫ 1

xφn(y)dy = λnφ

′n(x),

2

yields the second boundary condition φ′n(1) = 0. We differentiate again to obtain the Sturm–Louivilleproblem

−φn(x) = λnφ′′n(x)

φn(0) = φ′n(1) = 0.

The solution

λn =1(

n+ 12

)2π2

, φn(x) =√

2 sin

[(n+

1

2

)πx

]=√

2 sin(x/√λn) (7)

yields the eigenvalues and eigenfunctions for (6).It is illustrated in [4] that the eigenvalues and eigenfunctions of (5) are

λn =

{2L

1+L2w2n

, if n is even,

2L1+L2v2n

, if n is odd,

and the eigenfunctions are

φn(x) =

sin(wnx)√1− sin(2wn)

2wn

, if n is even,

cos(vnx)√1+

sin(2vn)2vn

, if n is odd.

Here wn and vn are the solutions of the transcendental equations{Lw + tan(w) = 0 , for even n,

1− Lv tan(v) = 0 , for odd n.

You can get good initial guesses by plotting the functions and zooming to approximate the roots.You can obtain accurate values using the Matlab command fzero.m.

For both correlation functions, it is important to note the decay in λn, which allows one totruncate the expansion (1).

You can find a more mathematical discussion of the Karhunen–Loeve expansion in [1], which isposted in the class references.

3 Approximation of the Karhunen–Loeve Expansion

Due to the decay of the eigenvalues λN , we can employ approximate relations

α(x, ω) = α(x) +N∑n=1

√λnφn(x)Qn(ω). (8)

To approximate α(x) and φn(x), one can employ the expansions

α(x) ≈ αh(x) =J∑j=1

αjΦj(x)

φn(x) ≈ φhn(x) =

J∑j=1

φjnΦj(x)

(9)

3

where Φi(x) are, for example, finite element basis functions. The Karhunen–Loeve expansion is thenapproximated by

αN (x, ω) ≈ αNh (x, ω) =J∑j=1

αjΦj(x) +N∑n=1

√λhnQn(ω)

J∑j=1

φjnΦj(x). (10)

The coefficients {αj}Jj=1 are obtained from (8) by solving the linear system

M~α = f

where ~α = [α1, . . . , αJ ]T and

Mij =

∫D

Φi(x)Φj(x)dx , fi =

∫Dα(x)Φi(x)dx.

The approximation of the eigenvalue problem∫DC(x, y)φn(y)dy = λnφn(x)

yields ∫DC(x, y)

J∑j=1

φjnΦj(y)dy = λhn

J∑j=1

φjnΦj(x)

⇒J∑j=1

φjn

∫D

∫DC(x, y)Φj(y)Φi(x)dxdy = λhn

J∑j=1

φjn

∫D

Φj(x)Φi(x)dx.

This requires the solution of the generalized matrix eigenvalue problem

Kφh = λhnMφh

where

Kij =

∫D

∫DC(x, y)Φj(y)Φi(x)dxdy

and (φh)j = φhj .

4 Random Field for the Heat Equation

Consider the heat equation

∂T

∂t=

∂

∂x

(αN (x, ω)

∂T

∂x

)+ f(t, x) , −1 < x < 1, t > 0

T (t,−1, ω) = T`(ω) , T (t, 1, ω) = Tr(ω) t > 0

T (0, x, ω) = T0(ω) − 1 < x < 1.

(11)

To establish the well-posedness of solutions, one typically requires that 0 < αmin ≤ αN (x, ω) ≤ αmax.Unfortunately, this cannot be ensured for Gaussian random fields. The lower bound can be enforcedby employing the log-normal Karhunen–Loeve expansion

α(x, ω) = αmin + eα(x)+∑∞

n=1

√λnφn(x)Qn(ω).

However, this does not provide an upper bound. To address this, one can instead prove well-posednessin a stochastic sense as detailed in [2].

4

References

[1] A. Alexanderian, “A brief note on the Karhunen–Loeve expansion.”

[2] J. Charrier, “Strong and weak error estimates for elliptic partial differential equations with ran-dom coefficients,” SIAM Journal of Numerical Analysis, 50(1), pp. 216–246, 2012.

[3] R.C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, SIAM,Philadelphia, PA, 2014.

[4] H.L. Van Trees, Detection, Estimation and Modulation Theory, Part 1, John Wiley & and Sons,New York, 1968.

5