Post on 27-Aug-2018
Applications of Wavelets inNumerical Mathematics
Kees Verhoeven
1. Brief summary
2. Data compression
3. Denoising
4. Preconditioning
5. Adaptive grids
6. Integral equations
1
1. Brief Summary
• φ(t): scaling function.For φ the following 2-scale relation holds
φ(t) =∞∑
k=−∞
pkφ(2t− k), t ∈ IR.
• ψ(t): mother wavelet.For ψ the following 2-scale relation holds
ψ(t) =∞∑
k=−∞
qkφ(2t− k), t ∈ IR.
• The decomposition for φ reads
φ(2t−k) =∞∑
m=−∞h2m−kφ(t−m)+g2m−kψ(t−m), t ∈ IR.
2
2. Data Compression
We consider a function f
f : [0, 1]→ IR.
We want to approximate this function by a function v de-fined by
v =∑k
ckφk,
where φk|k = 1, . . . , N is a basis for the linear functionspace V .
The quality of the approximation can be expressed in termsof a norm
‖f − v‖.
An alternative is to expand f periodically. We thereforelook at the Fourier series of f
f(x) =∞∑
m=−∞cme
2πimx
and approximate this by
v(x) =M∑
m=−M
cme2πimx.
3
So
V =e2πmix |m = −M, . . . ,M
,
with dimension N = 2M + 1. The basis functions form anorthonormal system. Therefore
ck = (φk, f) =
∫ 1
0f(x)e−2πimxdx.
Error estimates
Given f , g : [0, 1]→ IR with the Fourier expansions
f =∞∑
m=−∞cme
2πimx, g =∞∑
m=−∞dme
2πimx.
Then∫ 1
0f(x)g(x)dx =
∞∑m=−∞
cmdm.
So ∫ 1
0|f(x)|2dx =
∞∑m=−∞
|cm|2.
The error then reads
ε2M := ‖f − v‖2 =∞∑
|m|>M
|cm|2.
In many cases properties of f lead to an error estimate ofthe type
εM ≤ CM−α, C, α > 0.
4
Example
Given
f(x) = x(x− 1
2)(x− 1), x ∈ [0, 1].
We can derive
cm =3
4iπ3m3 .
The error εM can therefore be estimated via
ε2M =9
8π6
∞∑m=M+1
m−6 ≤ 9
8π6
∫ ∞M
y−6dy =9
40π6M−5.
M L2-error√
940π6M−5
10 0.426 ·10−4 0.484 ·10−4
20 0.803 ·10−5 0.855 ·10−5
40 0.147 ·10−5 0.151 ·10−5
5
How about localized functions?
It seems sensible to approximate a localized function withbasis functions which also have compact support.
Stepfunctions:
φk =
h−12 , x ∈ [(k − 1)h, kh),
0, else.
Note that:
• h = N−1 and ‖φk‖ = 1
• more efficient for function evaluations
6
Comparison
We consider the following function.
The error of the approximation using the Fourier series atM = 64, is approximately 0.001. The approximation usingfirst order splines is 0.01 with N = 2M + 1 = 129.
We would like to use localized basis functions only wherethe function to be approximated behaves like a localizedfunction.
7
Therefore, we would like basis functions which all have thesame shape but different scales. Then, if we have
v =N∑k=1
ckφk
and |cj| ε, we also have∥∥∥∥∥f −∑k
ckφk
∥∥∥∥∥ ≤∥∥∥∥∥∥f −
∑k 6=j
ckφk
∥∥∥∥∥∥+ ‖cjφj‖.
This leads to data compression.
8
Example
We denote by VJ the space of piecewise constant basis func-tions on [0,1] with width h = 2−J and dimension N = 2J .
The space V0 has one basis function: the constant functionφ1 = 1.For V1 the usual basis functions are depicted here:
The coefficients ck behave like
ck = h−12
∫ a+h/2
a−h/2f(x)dx ≈ h
12f(a).
Can we chose a more appropriate basis?
9
First approach: construct basis for VJ by expanding thebasis of VJ−1.
Figure 1: An alternative basis for V1.
Figure 2: Together with the functions above an alternative basis for V2.
But: the new basis is no longer orthogonal.For the test function f(x) = x, the drop in the coefficientsseems to be like h3/2 (23/2 ≈ 3).
generation |ck|0 0.3541 0.1252 0.0443 0.016
10
A better alternative basis for VJ
Construct an orthogonal basis φi for VJ . This leads to
Figure 3: A better alternative basis for V1.
Figure 4: Together with the two functions above a better alternative basisfor V2.
Note that these are the Haar wavelets!
ψ(x) = ψ0,0(x) = φ2(x), ψ0,1(x) = φ3(x), ψ1,1(x) = φ4(x).
11
Now:
ck = h−12
(∫ aa−h/2 f(x)dx−
∫ a+h/2a f(x)dx
)= h−
12
(∫ aa−h/2(f(a) + (x− a)f ′(a) + . . .)dx
−∫ a+h/2a (f(a) + (x− a)f ′(a) + . . .)dx
)≈ 1
4f′(a)h3/2.
12
Comparison of this basis and homogeneous one:
Figure 5: Approximation of f (top left) with 32, 10, 9 basis functions,respectively.
13
Figure 6: Approximation of f with 22, 10 homogeneous basis functions,respectively.
If wavelets used as basis functions have several momentsequal to zero, then reduction will be better.
Because, if
v =∑l∈ZZ
cl,Iφl,I +J−1∑k=I
∑l∈ZZ
dl,kψl,k,
then
|dk,J | ≤ Chd+3/2,
where d is the number of moments equal to zero.
14
Example
M L2-error for spline d = 1 L2-error for spline d = 3
256 1.149 ·10−3 2.829 ·10−4
128 1.149 ·10−3 2.829 ·10−4
64 1.356 ·10−3 2.829 ·10−4
32 2.623 ·10−3 1.143 ·10−3
15
3. Denoising
Suppose we have a signal with some noise. We can make awavelet decomposition up to a certain depth.If the coefficients of the wavelets remain relatively large(say |dj, k| > ε), then we have some localized noise.So cancel these contributions by forcing dj,k = 0 at L levelsdeep.
Then, we can reconstruct the filtered signal.
16
The role of L can be seen as follows:
Figure 7: Filtering of f with L = 1, 3, 5 levels (ε = 0.1).
17
But: if the original signal is not periodic, we encounterproblems at the boundaries.
Figure 9: Filtering of non-periodic f with 3 levels (ε = 0.01).
19
4. Preconditioning
Consider
D(u) = f
on a domain Ω, with the differential operator D elliptic. IfD is linear, this would lead to a linear system
Dc = r,
with
Dj,k = (φj,Dφk), rj = (φj, f).
The matrix D is called the stiffness matrix. If we use aniterative method to solve this system, the speed of con-vergence strongly depends on the condition number of thematrix D, κ(D) = ‖D‖‖D−1‖, with
‖D‖ = sup‖c‖=1
cTDc, ‖D−1‖−1 = inf‖c‖=1
cTDc.
For symmetric matrices, we can express the condition num-ber in terms of the eigenvalues:
κ(D) =λmax
λmin.
20
Example
We consider
D(u) = −d2u
dx2 + u = f, x ∈ (0, 1),
with periodical boundary conditions.We assume that the numerical solution v can be writtenas a linear combination of certain scaling functions whichspan VJ .Galerkin’s method and integration by parts gives us
Dc = r,
with
Di,j =
(dφi,Jdx
,dφj,Jdx
)+ (φi,J , φj,J),
and, as before
rj = (φj, f).
For linear B-splines:(dφi,Jdx
,dφj,Jdx
)=
∫2J/2
d
dxφ(2Jx−i)2J/2 d
dxφ(2Jx−j)dx.
The derivatives are piecewise constant, and therefore wederive
Di,j =
2N 2 + 2
3 , i = j,
−N 2 + 16 , i = (j ± 1) mod N,
0 else.
This is a circulant matrix, that is Di,j = d(i− j).
21
We define the symbol of a circulant matrix as
D(z) =∑j
Di,jzi−j.
For D now follows
λmax = maxzN=1|D(z)|, λmin = min
zN=1|D(z)|.
For the differential equation we have
D(z) =∑j
((dφi,Jdx
,dφj,Jdx
)+ (φi,J , φj,J)
)zi−j.
We write
D(z) = D1(z) +D2(z),
with
D1(z) =∑j
(dφi,Jdx
,dφj,Jdx
)zi−j, D2(z) =
∑j
(φi,J , φj,J)zi−j.
The second term can be recognized as D2(z) = Rφ(z). Inthe same manner we can derive D1(z) = 22JRφ(z).Using this and the 2-scale relations, we can derive
D(z) = N 2(2− z − z−1) +1
6(4 + z + z−1).
Calculating the biggest and smallest eigenvalue, we see that
κ1 = 4N 2 +1
3.
If N →∞, κ1 →∞.
22
We now use wavelets
v =∑k,j
dk,jψk,j.
The stiffness matrix D then looks like
Dl,m,j,k =
(dψm,ldx
,dψk,jdx
)+ (ψm,l, ψk,j).
After some algebra using Riesz functions and 2-scale rela-tions, we can show that
κ2(D) ≤ C, for all J.
Comparison
2J κ1 κ2
16 1024.3 45.432 4096.3 49.764 16384.3 52.9128 65536.3 55.4256 262144.3 57.3
Again: this strongly depends on the periodicity of the bound-ary conditions.
23
5. Adaptive grids
We consider a hyperbolic PDE
∂u(x, t)
∂t= F(u,
∂u
∂x, . . .),
together with initial condition and periodic boundary con-ditions.
The approximation of the initial condition u(x, 0) is doneby
v(x, 0) =N−1∑i=0
ci,I(0)φi,I(x) +J−1∑j=I
∑i∈Ij
di,j(0)ψi,j(x).
Here N = 2J is the amount of intervals on the coarsestgrid I, the set Ij is a subset of all possible wavelets on thegrids j = I, . . . , J −1. These sets Ij are found by making awavelet decomposition and leave out all wavelet coefficientsbelow a certain threshold ε.
But: this would mean that we have to approximate thefunction on the finest grid first!
24
We ignore all contributions ’below’ a wavelet for which|dk,l| ≤ ε. (filled circles mean |dk,l| > ε, open circles mean:|dk,l| ≤ ε)
Adaptivity means that wavelets left out in previous timesteps can occur again.
25
Example: Burgers equation
Figure 10: Approximation on t = 0, 112 , 2
12 , 312 , 4
12 , 512 , for Burgers equation.
The number of basis functions with coefficient above threshold is 32, 56,122, 114, 114 and 114, respectively.
26
Example: wave equation
Figure 11: Approximation on t = 0, 0.3 and 0.5, respectively, for the waveequation. The number of basis functions with coefficient above threshold isapproximately 60.
27
6. Integral Equations
We consider
u(x) =
∫K(x; t)u(t)dt+ f(x).
We take
v(x) =∑j
cjφj(x).
Galerkin’s method would leave us with
Ac = r,
with
Aj,k = (φj, φk)−∫ ∫
φj(x)K(x; t)φk(t)dxdt, rj = (φj, f).
Often this A is well conditioned, but full.
Using wavelets reduces the number of nonzero elements.We represent
v(x) =∑l∈ZZ
cl,Iφl,I +J−1∑k=I
∑l∈ZZ
dl,kψl,k.
Look at the second term of A
Kl,m,j,k =
∫ ∫ψm,l(x)K(x; t)ψk,j(t)dxdt.
We make the following assumption on K(x; t)∣∣∣∣ ∂d∂xdK(x; t)
∣∣∣∣+
∣∣∣∣ ∂d∂tdK(x; t)
∣∣∣∣ ≤ Cd1
|x− t|d+1 ,
for a certain d > 0.
28
Making use of the Taylor series of K(x; t) and taking awavelet with d zero moments, we can derive
|Kl,m,j,k| ≤ C1
|x0 − t0|d+1 .
Using this we can bring down the number of nonzero ele-ments (or better: the number of elements with value abovea certain threshold ε).
Example
N ε = 10−6 ε = 10−9 ε = 10−12
24 74% 92% 92%48 19% 85% 96%96 5.1% 54% 78%192 1.1% 16% 50%384 0.34% 3.5% 25%
Table 1: The number of elements above threshold ε, with Daubechieswavelets with K = 2.
N ε = 10−6 ε = 10−9 ε = 10−12
24 66% 92% 92%48 12% 93% 96%96 3.1% 47% 90%192 0.85% 12% 56%384 0.32% 2.4% 21%
Table 2: The number of elements above threshold ε, with Daubechieswavelets with K = 5.
29