Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf ·...

37
Chapter 5 Brownian Motion and the Wiener Process In continuous time, stochastic systems described by recursions of the form x k+1 = f (x k ,u k ,w k ), x R n ,u R m ,w R p ,k Z + , x 0 w, w k ,k Z + , i.i.d. N (0, Σ), are replaced by systems described by ( ˆ Ito) stochastic differential equations of the form dx t = f (x t ,u t )dt + g(x t ,u t )dw t , t R + , where x 0 is a random initial condition, x 0 w and dw t denotes the random measure arising from the what may be viewed intuitively as the infinitesimal increments of a Wiener process. The increment dw t will be seen to be of order dt in the sense that E(w t+dt - w t )(w t+dt - w t ) T = Idt. The Physical Basis of Brownian Motion The motivation for Wiener’s construction of the Wiener process came from the work of Brown (1926) and Einstein and Shmoluchovski (1906) on what is called the physical Brownian Motion process (see Einstein [1956], Chandrasekhar (in Wax [1959]), and Shuss [1980], who we are partly following in the discussion below). Brown was the first to notice that small particles and grains maintained perpetual irregular motions when suspended in a fluid. This is due to molecular collisions: colloidal particles of radius 50μ (μ = 10 -4 cm) will undergo approximately 10 21 collisions per second with the molecules of an immersing fluid. (For comparison we note that in ideal gases at normal temperatures and pressures there are 6.02 × 10 23 × (22.4 × 10 3 ) -1 =2.7 × 10 19 molecules per cc. By lattice packing the average distance between molecules is 30 Angst. = 30 × 10 -4 μ = 30 × 10 -8 cm. and each molecule makes approximately 10 9 collisions per second.) The following is a sketch of the mathematical theory of physical Brownian motion which is based upon the references cited above, and whose modelling argument displays an interesting 1

Transcript of Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf ·...

Page 1: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Chapter 5

Brownian Motion and the Wiener Process

In continuous time, stochastic systems described by recursions of the form

xk+1 = f(xk, uk, wk), x ∈ Rn, u ∈ Rm, w ∈ Rp, k ∈ Z+,

x0

∐w,wk, k ∈ Z+, i.i.d. N(0,Σ), are replaced by systems described by (Ito) stochastic

differential equations of the form

dxt = f(xt, ut)dt+ g(xt, ut)dwt, t ∈ R+,

where x0 is a random initial condition, x0

∐w and dwt denotes the random measure arising

from the what may be viewed intuitively as the infinitesimal increments of a Wiener process.

The increment dwt will be seen to be of order√dt in the sense that

E(wt+dt − wt)(wt+dt − wt)T = Idt.

The Physical Basis of Brownian Motion

The motivation for Wiener’s construction of the Wiener process came from the work

of Brown (1926) and Einstein and Shmoluchovski (1906) on what is called the physical

Brownian Motion process (see Einstein [1956], Chandrasekhar (in Wax [1959]), and Shuss

[1980], who we are partly following in the discussion below). Brown was the first to notice

that small particles and grains maintained perpetual irregular motions when suspended in a

fluid. This is due to molecular collisions: colloidal particles of radius 50µ (µ = 10−4 cm) will

undergo approximately 1021 collisions per second with the molecules of an immersing fluid.

(For comparison we note that in ideal gases at normal temperatures and pressures there are

6.02 × 1023 × (22.4 × 103)−1 = 2.7 × 1019 molecules per cc. By lattice packing the average

distance between molecules is 30 Angst. = 30 × 10−4µ = 30 × 10−8 cm. and each molecule

makes approximately 109 collisions per second.)

The following is a sketch of the mathematical theory of physical Brownian motion which is

based upon the references cited above, and whose modelling argument displays an interesting

1

Page 2: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

three time scale (macro, mezzo and micro) structure. Since a particle is much more massive

than the molecules hitting it, we may consider the motion to be the result of 2 forces:

(i) Friction: By Stokes’ Law, the drag force per unit mass acting on a spherical particle of

radius a is

fd = −βv ∈ R3, where β =6πaη

m> 0,

where η is the coefficient of viscosity.

(ii) Collision forces: we assume there is a randomly fluctuating force f(t) ∈ R3, t ∈ R+, per

unit mass where:

(i) f(t) is statistically independent of v(t);

(ii) f(t) variations are much more frequent than those of v(t);

(iii) the average of f(t) is zero for all t.

Consequently Newton’s equations of motion for the particle are

dv(t)

dt= −βIv(t) + f(t), (1)

where this is a differential equation with random right hand side. Since such equations and

their solution have not yet been constructed we proceed formally. First consider

P (v(t) ∈ A|v(0) = v0) =

∫A

p(v, t|v0)dv,

where p(v, t, v0) → δ(v − v0) as t→ 0.

From statistical physics it is know that, as t→∞, p(v, t, v0) must approach the Maxwellian

density (parametrized by the temperature T ) of the surrounding medium (independently of

v0); hence

limt→∞

p(v, t, v0) =1

(2π)3/2(kTm

)3/2exp

−1

2[v1 v2 v3]

kTm

0 0

0 kTm

0

0 0 kTm

−1

v1

v2

v3

=1

(2π)3/2(kTm

)3/2exp

{−1

2

m

kT‖v‖2

}(2)

2

Page 3: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Since the formal solution to (??) is

v(t) = e−β(t−t0)v0 +

∫ t

t0

e−β(t−s)f(s)ds,

we obtain

v(t)−∫ t

t0

e−β(t−s)f(s)ds→ 0, as t0 → −∞

and in the limit

v(t) =

∫ t

−∞e−β(t−s)f(s)ds ≈

∫ t

0

e−β(t−s)f(s)ds, t� 0.

For t� 0 consider approximating Riemann sums for the integral. Then

∫ t

0

e−β(t−s)f(s)ds ≈ e−βt

b t∆tc∑

n=1

eβn( ∆t)f(n ∆t) ∆t

=N∑

n=1

eβ(n∆t−t) ∆ bn

where ∆ bn denotes the random acceleration of the particle over [n ∆t, (n + 1) ∆t], where

these events are assumed to be independent on distinct intervals.

Next assume ∆t is large with respect to the average period of fluctuation of f due to

successive molecular collisions. (For instance, if ∆t = 10−6 seconds, ∆ bn will be the sum

of 0(1021)× 10−6 = 0(1015) collisions.) Then we may write

∆ bn =√M

(1√M

M∑k=1

(snk)

),

where M � 1 and snk denotes the change in force (per unit mass) on the particle over the

associated sub-interval M−1 ∆ t due to molecular collisions.

From the theory of statistical thermodynamics, it is known that the distribution of these

changes in each dimension is such that {snk ; 1 ≤ k ≤ M} are independent for distinct k,

have zero mean and have variance E‖snk‖2 = 2qM−1 ∆t for some q > 0. The fact that the

this variance is linear in the time interval M−1 ∆t is the crucial step in the construction

of Brownian motion (and in the derivation of the properties of the Wiener process and

consequently the Ito calculus).

3

Page 4: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Consequently, from the Central Limit Theorem, we have

∆ bn ∼ N(0, diag3(2q ∆t)),

where diag3(α) denotes a 3× 3 diagonal matrix with diagonal entry α.

Finally we obtain

E‖v(t)‖2 =N∑

n=1

(2q( ∆t))e2β(n ∆t−t) (3)

→ 2q

∫ t

0

e2β(s−t)ds = 2q1

2β[1− e2β(−t)],

as N →∞. But from the Maxwellian density we have E‖v(t)‖2 → kTm

in each dimension as

t→∞, and hence q = βkTm

.

Position Distribution Calculation via State Space Solution in R6

Solving the six dimensional state space system

d

[x

v

]=

[0 I

0 − βI

][x

v

]dt+

[0

I

]f(t)dt, t ∈ R+, (4)

with initial condition[x0

v0

]for x gives

xt = [1 0]

{e[

0 I0 −βI ]t

[x0

v0

]+

∫ t

0

e[0 I0 −βI ](t−s)

[0

f(s)

]ds

}t ∈ R+

= x0 +(1− e−βt)

βv0 +

∫ t

0

(1− e−β(t−s))

βf(s)ds. (5)

since e[0 I0 −βt ] =

[I 1−e−βt

βI

0 e−βtI

].

Hence xt− (x0 + (1−e−βt)β

v0) ∈ R3 is a zero mean Gaussian random variable with variance

in each dimension given by

σ2 = (2q)

∫ t

0

(1− e−β(t−s))2

β2ds =

q

β3(2βt− 3 + 4e−βt − e−2βt) =

q

β3γ(t) =

βkT

β3mγ(t).

Hence Covxt = [diag( kTγmβ2 )] ∈ R3×3 and

p(x, t|x0, v0) =

{mβ2

2πkT [γ(t)]

}3/2

exp

−mβ2‖x− {x0 + (1−e−βt)v0

β}‖2

2kT [γ(t)]

. (6)

4

Page 5: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Take γ(t) ∆ 2βt for t� 0; then averaging over a zero mean symmetric distribution p(v0) for

v0 yields

p(x, t|x0) =

∫R3

p(x, t|x0, v0)p(v0)dv0 =1

(2π2Dt)3/2exp

{−1

2

‖x− x0‖2

2Dt

},

where

4Dt ∆ 2kTγ

mβ2=

4kT t

and so

D =kT

mβ=

kT

6πaη.

We conclude that

∂p

∂t= D ∆ p ≡ D

(∂2

∂x21

+∂2

∂x22

+∂2

∂x23

)p,

with p(x, t|x0) → δ(x− x0) as t→ 0.

Hence we may say that an idealization of physical Brownian Motion is a stochastic process

in R3 such that

P (x(t) ∈ A|x0) = (4πDt)−3/2

∫A

e−‖x−x0‖2/4Dtdx, A ∈ B(R3),

and such that

(i) the increments x(t + v)− x(t), x(t)− x(t− u) are independent for all t ≥ 0, v ≥ 0, u ≥

0, t− u ≥ 0, and, further, are functionally independent of t ≥ 0;

(ii) the paths {x(t); t ≥ 0} are continuous;

(iii) the joint distribution of x(t1), ·, ·, ·, x(tn) is Gaussian for all n and all t1 < · · · < tn.

(Notice that the x is a Markov process; but it has been constructed as an approximation to

a part of a state (x, v) of the Markov process in (??).) This physical analysis leads to the

following definition of a Wiener process as a standardized version of Brownian Motion.

5

Page 6: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

The Mathematical Theory of the Wiener Process

Rigorous mathematical analyses which establish the Wiener process nature of the limit-

ing behaviour of the particle models (as the number of particles and the collision rates go to

infinity) are to be found in [F. Spitzer, Journal of Math. and Mech. Vol 18, No. 19, 1969, pp.

973-989] and [R. Holley, Z. Wahrsheinlichkeitsth. verw. Geb,17, pp 181-219, 1971]; in this

context the classical text by Khinchin [Mathematical Principles of Statisticial Mechanics,

Dover, 1969] should also be cited.

Definition 5.1 A Wiener process w is an Rn valued stochastic process such that

Axiom W1. w0 = 0, w.p.1,

Axiom W2. p(wt|ws) = 1(2π)n/2

1(t−s)n/2 exp− 1

2(t−s)‖wt − ws‖2, 0 ≤ s < t,

Axiom W3. The process increments satisfy the independent increment property,∐{wpi

wqi}N

1 , for all N ∈ Z1 and all qN1 , p

N1 ∈ R, such that 0 ≤ q1 < p1 ≤ q2 < · · · ≤ qN < pN .

We observe that Axiom W2 is equivalent to

py(α|ws) =1

(2π)n/2

1

|t− s|n/2exp− 1

2(t− s)‖α‖2,

when α ∆ wt − ws; and that Axiom W3 may be written

p(wpN− wqN

, · · · , wp1 − wq1) =N∏

i=1

p(wpi− wqi

).

The transition distribution of a stochastic process x is defined as P (xt ∈ A|xs) for all A ∈

B(Rn) and all s < t, s, t ∈ R, and, by the definition of a density, we have the representation

P (xt ∈ A|xs) =

∫A

pt,s(x|xs)dx, s < t, A ∈ B(Rn),

when a transition density exists.

6

Page 7: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

For a Wiener process, Axiom W2 gives the representation

P (wt ∈ A|ws) =

∫A

1

(2π)n/2

1

|t− s|n/2exp−1

2

{‖w − ws‖2

t− s

}dwt, A ∈ B(Rn),

and, in case A =∏n

i=1[ai, bi],

P (wt ∈n∏

i=1

[ai, bi]|ws) =

∫[a1,b1]

· · ·∫

[an,bn]

1

(2π)n/2

1

|t− s|n/2exp−1

2

{‖w − ws‖2

t− s

}dw.

The finite dimensional distributions of a Wiener process are Gaussian of the form given in

(??) below. This is the case since, in general, assuming the indicated conditional densities

exist, we have that for any m ∈ Z+, any collection of sets Am1 in B(Rn), and any collection

of instants tm1 with 0 < ti − ti−1, i = 1, · · ·m,

P (wt1 ∈ A1, · · · , wtm ∈ Am)

=

∫Am

· · ·∫

A1

pm,m−1,··· ,1(wm, wm−1, · · · , w1)dw1 · · · dwm

=

∫A1

· · ·∫

Am

p(wm|wm−11 )p(wm−1|wm−2

1 ) · · · p(w1)dwm · · · dw1

=

∫A1

· · ·∫

Am−wm−1

p((wm − wm−1)|(wi − wi−1)m−11 ) · · · p(w1 − w0)dwm · · · dw1,

where for simplicity we have written wj for wtj , 1 ≤ k ≤ m, wm ∆ wm−wm−1, · · · , w1 ∆ w1−

w0, and where we have invoked Axiom W1 and the Change of Variables of Formula (hence-

forth denoted CVF). However, in the case under consideration, Axioms W2 and W3 together

imply that the conditional densities above exist and take the Gaussian form in the third ex-

pression below:

P (wt1 ∈ A1, · · · , wtm ∈ Am)

=

∫A1

· · ·∫

Am−wm−1

p(wm − wm−1) · · · p(w1 − w0)dwm · · · dw1 (by Axiom W3)

=

∫A1

· · ·∫

Am−wm−1

1

(2π)n/2

1

|tm − tm−1|n/2exp−1

2

1

(tm − tm−1)‖wm − wm−1‖2dwm

× 1

(2π)n/2

1

|tm − tm−2|n/2exp−1

2

1

(tm−1 − tm−2)‖wm−1 − wm−2‖2dwm−1

· · · × 1

(2π)n/2

1

tn/21

exp−1

2

1

t1‖w1‖2dw1. (by Axiom W2) (7)

7

Page 8: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

From the axioms W1 - W3, or from the expression above, it follows that the finite

dimensional densities of x are Gaussian and are given explicitly by:

p(wt1 , · · ·wtm) =1

(2π)mn2

1∏mi=1(ti − ti−1)n/2

×

exp−1

2[wT

tm − wTtm−1

, wTtm−1

− wTtm−2

, · · · , wTt1]

(tm − tm−1)I 0

(tm−1 − tm−2)I

0. . . t1I

−1

×

wtm − wtm−1

wtm−1 − wtm−2

wt1

.But when

Lm ∆

I −I . . . 0

0 I. . . 0

. . . I − I

0 0 I

(mn×mn)

and

Vm ∆

tm tm−1 · · · t1

tm−1 tm−1 · · · t1...

......

t1 t1 · · · t1

⊗ In, (mn×mn)

8

Page 9: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

we have the equality

[wTtm , · · · , w

Tt1][Vm]−1

wtm

...

wt1

= [wTtm , · · · , w

Tt1]LT

m[Lm Vm LTm]−1Lm

wtm

...

wt1

= [wTtm − wT

tm−1, wT

tm−1− wT

tm−2, · · · , wT

t1]

(tm − tm−1)I 0

(tm−1 − tm−2)I

0. . . t1I

−1

wtm − wtm−1

wtm−1 − wtm−2

wt1

.Hence

p(wt1 , · · · , wtm) =1

(2π)mn2

1

|Vm|1/2exp−1

2‖wtm , · · · , wt1‖2

V −1m. (8)

We observe that

Vm ≡ V tmt1

=

...

· · · [EwtiwTtj] · · ·

...

,since, taking ti < tj for definiteness,

EwtiwTtj

= Ewti [(wtj − wti) + wti ]T

= E(wti − w0)(wtj − wti)T + E(wti − w0)(wti − w0)

T

= E(wti − 0)E(wtj − wti) + E(wti − w0)(wti − w0)T (by Axiom W1)

= 0 + I(ti − 0) (by Axioms W2, W3)

= I min(ti, tj).

We note that the family of finite dimensional distributions constructed above necessarily

satisfies the Compatibility Condition and hence, if it is taken as the starting point, the

9

Page 10: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Daniell-Kolmogorov Theorem implies the existence of the Wiener process with the properties

stated in Definition 4.1.

Lemma 5.1

The following three sets of axioms for a Wiener process w are equivalent:

(i) W1. w0 = 0, w.p.1,

W2. p(wt|ws) = 1(2π)n/2

1|t−s|n/2 exp−1

2

{1

(t−s)‖wt − ws‖2

}, 0 < s < t,

W3. wt − ws

∐{wpi

− wqi}N

1 , for all N ∈ Z1 and all s, t, qN1 , p

N1 ∈ R, such that

0 ≤ q1 < p1 ≤ q2 < · · · ≤ qN < pN ≤ s < t.

(ii) B1. wt, t ≥ 0, is a Gaussian process,

B2. Ewt = 0, for all t ∈ R+,

B3. EwtwTs = min(t, s)I, t, x ∈ R+.

(iii) C1. w0 = 0, w.p.1,

C2. W2 holds,

C3. w is a Markov process.

Proof We show (i) ⇒ (ii) ⇒ (iii) ⇒ (i)

(1) Assume (W1) (W2) (W3) holds for a stochastic process w. Then it was proved earlier

that w is a Gaussian process, hence B1 holds. Since

Ewt = EE|w0(wt − 0) = E|w0(wt − w0) (by W1)

=

∫Rn

wtp(wt|0)dwt = 0, (by A2)

B(2) holds. And next,

EwtwTs = E((wt − ws) + ws)(ws)

T s < t

= E(wt − ws)E(ws − 0)T + E(ws − 0)(ws − 0)T

= min(t, s)I, (by W3, W2)

gives B3.

10

Page 11: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

(2) Assume B1 - B3 hold; then B3 gives Ew20 = 0 and hence w0 = 0 w.p.1, which is C1.

Next,

p(wt|ws) = p(wt, ws)(p(ws))−1

{with Ewt = Ews = 0 (B2)

and EwpwTq = min(p, q)I (B3)

=1

(2π)n/2

sn/2

|s(t− s)|n/2exp−1

2

1

s(t− s)

{sw2

t − 2swswt + tw2s − (t− s)w2

s

},

i.e. wt − ws ∼ N(0, (t− s)),

which is C2, as required.

Finally w is a Markov process since by B1 and B3, w is a Gaussian process with

orthogonal, and hence independent, increments. Hence

p(wtn|wtn−1

t0 ) = p(wtn − wtn−1|(wti − wti−1)n−11 )

But p(wtn − wtn−1) = p(wtn − wtn−1) = p(wtn|wtn−1) by C(2), and so C3 holds.

(2) C1-C3 ⇒ W1 - W2 is obvious. The equalities

p(wt − ws|{wpi− wqi

}N1 ) = p(wt|ws, {wpi

− wqi}N

1 ) (by CVF)

= p(wt|ws) (by C3)

= p(wt − ws), (by C2)

show that C2-C3 give W3.

The Wiener process has almost surely continuous sample paths (see Doob (1953, p 393))

and is almost surely of unbounded variation on any finite interval (see Doob (1953, p 395)).

Lemma 5.2

For the Weiner process w define

(i) ws1(t) ∆ wt+s − ws, t, s ∈ R+,

(ii) w2(t) ∆ tw(1t), t > 0, w2(0) ∆ 0 a.s.

Then ws1 is a Wiener process with respect to the standard Wiener measure conditioned on

the value of ws, and w2 is a Wiener process.

11

Page 12: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Proof.

(i) ws1(t) is a Wiener process , with respect to the probability distribution conditional on

ws since the axioms for a WP are satisfied as follows:

Axiom W1 is satisfied since ws1(0) = (wt+s − ws)|t=0 = 0. w.p.1. Axiom W12 holds

because, for p > q,

p(ws1(p)|ws

1(q)) ≡ p(wp+s − ws|wq+s − ws, ws)

= p((wp+s − wq+s) + (wq+s − ws)|wq+s, ws)

= p(wp+s − wq+s|wq+s, ws).1 (by CVF)

= p(wp+s − wq+s|wq+s − ws, ws − w0) (by Axiom W1

= p(wp+s − wq+s|wq+s − w0) (by Axiom W3)

=p(wp+s − wq+s, wq+s − w0)

p(wq+s − w0)

= p(wp+s|wq+s) (by definition cond. density and CVF)

=1

(2π)n/2

1

|p− q|n/2exp− 1

2(p− q)‖ws

1(p)− ws1(q)‖2 (by Axiom W2)

where we have used the abbreviated notation that p(x|y) denotes px|y(α|β)|x,y.

Finally Axiom W3 (for w) holds for w1 since

ws1(t2)− ws

1(t1) = wt2+s − ws − wt1+s + ws = wt2+s − wt1+s,

and this dif, by Axiom W3 (for w), is independent of wti+s − wtj+s = ws1(ti)− ws

1(tj),

whenever the intervals have disjoint interiors, as required.

(ii) Axiom W1 holds for the process w2 since (treating the scalar case for convenience and

without loss of generality)

limt→0

Et2w2(1

t) = lim

t→0t2

1

t= 0

implies that

limt→0

w2(t) = limt→0

tw(1

t) = 0 a.s.

12

Page 13: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Axiom W3 clearly holds (with t1 > t2 ≥ t3 > t4 ≥ · · · implying

1t1< 1

t2≤ 1

t3< 1

t4≤ · · · ).

Axiom W2 is the case since tw(1t) and sw(1

s) are jointly Gaussian random variables,

with mean

Etw(1

t) = 0 = Esw(

t

s),

and with covariance

E

tw(1t)

sw(1s)

(tw(1

t), sw(

1

s)

)=

t2

ttsmin(1

t, 1

s)

tsmin(1t, 1

s) s2

s

=

t s

s s

.Hence

p(tw(1

t)|sw(

1

s)) = p

(tw(1t), sw(1

s))

p(sw(1s))

=

1(2π)

1(2π)1/2

1(s1/2|t−s|1/2)

1(s1/2)

exp

−1

2(w2(t), w2(s))

t s

s s

−1(w2(t)

w2(s)

)+

1

2sw2

2(s)

=

1

(2π)1/2|t− s|1/2exp−1

2

(sw22(t)− 2sw2(t)w2(s) + sw2

2(s))

s(t− s)

=1

(2π)1/2

1

|t− s|1/2exp−1

2‖w2(t)− ws(s)‖2,

as required.

An application of the Law of the Iterated Logarithm (see e.g. Caines(1988)) shows

that the process w2 is continuous at 0 almost surely.

13

Page 14: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Self similarity of Wiener processes

Consider the magnified or miniaturized, i.e. scaled, version of the Wiener process given

by

wc(t) ∆ cw

(t

c2

), c > 0, t ∈ R+.

Then

(i) wc(t)|t=0 = cw

(t

c2

)= 0 w.p.1,

(ii) pwc(wc(t)|wc(s)) = pwc(c.w

(t

c2

)|c.w

( sc2

))

= pw(w

(t

c2

)|w( sc2

))| ∂(w)

∂(cw)|

=1

(2π)n/2

1

| Itc2− Is

c2|n/2

exp−1

2

{1

| tc2− s

c2|1

c2‖cw

(t

c2

)− cw

( sc2

)‖2

} ∣∣∣∣ ∂(w)

∂(cw)

∣∣∣∣=

1

(2π)n/2

1∣∣ Itc2− Is

c2

∣∣n/2|Ic|−1 exp−1

2

{1

|t− s|‖wc(t)− wc(s)‖2

},

= pw(wc(t)|wc(s)).

So the measure governing wc(t) given wc(s) is identical to that governing w(t) given

w(s).

(iii) w a Wiener process implies(w

(t

c2

)− w

( sc2

))∐{w(pi

c2

)− w

( qic2

)}N

i=1

when 0 ≤ qi

c2< pi

c2≤ · · · ≤ qN

c2< pN

c2≤ s < t, and so multiplying by c it is clear that

wt − ws

∐{wpi

− wqi}N

1 . Hence wc is a Brownian motion for any c > 0.

14

Page 15: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Stochastic Integration and Linear Stochastic Differential Equations

Stochastic integrals with respect to Wiener process increments are defined via the mean

square limits of approximating finite sums. Here we present an introduction to the basic

ideas.

By definition, for an Rn valued Wiener process,∫ t1

t0

dwt = wt1 − wt0 , t0 < t1, t0, t1 ∈ R+, (9)

which is the increment of the Wiener process on [t0, t1].

Next consider a sequence of (m× n) matrices {Ai; 0 ≤ i ≤ N}, instants {t0 < t1 < · · · <

tN+1}, t0 = T0, tN+1 = T1 and the associated matricial step functionAN(t) ∆∑N

i=0Ai(sti(t)−

sti+1(t)), t ∈ R+, where sτ (t) = {0 if t < τ, 1 if τ ≤ t; t ∈ R}. We give the definition of the

integral of a matricial step function against the increments of the standard vector Weiner

process w. It is seen to specialize to (??) when A is the constant identity function,

We set∫ tN+1

t0

AN(t)dwt =

∫ tN+1

t0

N∑i=0

Ai(sti(t)− sti+1(t))dwt

∆N∑

i=0

Ai

∫ ti+1

ti

dwt (where (sti(t)− sti+1(t) = 0, t /∈ [ti, ti+1])

=N∑

i=0

Ai(wti+1− wti)

We see that the definition automatically gives the linearity of integration with respect to the

integrand. Furthermore, since E(wti+1− wti)(wtj+1

− wtj)T = (ti+1 − ti)I, if i = j, and is 0

otherwise,

E(

∫ tN+1

t0

AN(t)dwt)(

∫ tN+1

t0

AN(t)dwt)T =

N∑i=0

(ti+1 − ti)AiATi (10)

Now let us assume that the sequence of deterministic matrix step funtions of time

{AN(·);N ∈ Z1} converges to a (measurable in t ∈ R) matrix function A(·) ∈ L2 on

[t0 = T0, t(N+1) = T1] in such a way that∫ T1

T0

(AN(t)− A(t))(AN(t)− A(t))Tdt→ 0,

15

Page 16: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

as N → ∞; in other words, let {AN(·);N ∈ Z1} converge in L2 to the L2 matrix function

A(·). But then the sequence {AN(·);N ∈ Z1} is a Cauchy sequence in L2 and hence (reader

check) {∫ T1

T0AN(t)dwt;N ∈ Z1} constitutes a Cauchy sequence in the Hilbert space Hw

spanned by the process w. So by the completeness of the Hilbert space Hw we conclude that∫ T1

T0AN(t)dwt converges in mean square to a matrix random variable. We denote the limit

by∫ T1

T0A(t)dwt, i.e.

E‖∫ T1

T0

AN(t)dwt −∫ T1

T0

A(t)dwt‖2 → 0,

as N →∞ which is readily verified to be unique for all sequences of functions {AN(·);N ∈

Z1} converging to A(·) in L2. It is consistent and convenient at this point to make the

definition

d(

∫ t

0

A(s)dws) ∆ A(t)dwt.

We note that any L2 matrix function of time on a finite interval is the L2 limit of a

sequence of L2 matrix step functions.

Lemma 5.3

Let the sequences of vector random variables {IN ;N ∈ Z+} and {JN ;N ∈ Z+} form a

Cauchy sequences in the Hilbert space Hw generated by the process w. Then their mean

sqaure limits I and J exist and satisfy

(i) limN→∞ EIN = EI <∞,

(ii) limN,M→∞ EINJTM = EIJT <∞.

Proof

(i) Since {IN ;N ∈ Z+} forms a Cauchy sequence in the necessarily complete Hilbert space

Hw, there exists a limiting vector random variable I such that

limN→∞(E‖IN − I‖2)1/2 = 0 and similarly for the sequence {JN ;N ∈ Z+}. Next we

observe that {EIN ;N ∈ Z+} forms a Cauchy sequence in Rn ; this is the case since

the inequalities

‖EIN − EIM‖ ≤ E‖IN − IM‖ ≤ (E‖IN − IM‖2)1/2,

16

Page 17: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

show the leftmost term goes to zero as N,M →∞ because {IN ;N ∈ Z+} is a Cauchy

sequence in Hw. Hence a unique (non-random) limit limN→∞ EIN of the sequence

{EIN ;N ∈ Z+} exists in Rn. Finally, the inequalities

‖EI − limN→∞EIN‖ ≤ ‖EI − EIM‖+ ‖EIM − limN→∞EIN‖

≤ (E‖I − IM‖2)1/2 + ‖EIM − limN →∞EIN‖,

for all M , show that EI = limN→∞EIN <∞..

(ii) For IN , IM andI as above, the inequality

‖EIJT − EIMJTN‖ = ‖E(I − IM)(J − JN)T + EIM(J − JN)T + EIN(J − JM)T‖

≤ (E‖I − IM‖2)1/2(E‖J − JN‖2)1/2 + (E‖IM‖2)1/2(E‖J − JN‖2)1/2

+ (E‖IN‖2)1/2(E‖J − JN‖2)1/2

shows that limN,M→∞ EINITM = EIIT <∞, as required.

Since E∫ T1

T0AN(t)dwr = 0 for all N , Lemma 4.3 gives

E

∫ T1

T0

A(t)dwt = E limN→∞

∫ T1

T0

AN(t)dwt = limN→∞

E

∫ T1

T0

AN(t)dwt = 0.

When the sequence of step functions {AN(·);N ∈ Z1} converges to A(·) ∈ L2, and

T1 ≤ T2 (for definiteness), that

E(

∫ T1

T0

A(t)dwt)(

∫ T2

T0

A(t)dwt) = E limN→∞

[(

∫ T2

T0

AN(t)dwt)(

∫ T1

T0

AN(t)dwt)T

]= lim

N→∞E

[∫ T1

T0

AN(t)dwt +

∫ T2

T1

AN(t)dwt)(

∫ T1

T0

AN(t)dwt)T

](by Lemma 4.3)

= limN→∞

∫ T1

T0

AN(t)ANT

(t)dt

=

∫ T1

T0

A(t)A(t)dt. (11)

17

Page 18: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

We have now constructed stochastic integrals of deterministic functions with respect to

the Wiener process and may consider linear SDEs of the form

dxt = Atxtdt+Btutdt+ Ctdwt, t ≥ t0, (12)

where xt0 is a random variable such that E‖xt0‖2 <∞, xt0

∐w and A,B,C and u are piece-

wise continuous functions on the finite interval [t0, t]. (That is to say, functions which are

continuous on the interiors of a finite set of intervals whose union is [t0, t] and which are

everywhere continuous from the right and everywhere have finite limits from the left). Then

(??) is interpreted to be equivalent to the stochastic integral equation

SIE xt = xt0 +

∫ t

t0

Asxsds+

∫ t

t0

Bsusds+

∫ t

t0

Csdws, t ≥ t0,

with the stated conditions on xt0 , w. A solution to the SDE is then defined to be any solution

to the SIE above which is w.p.1 continuous and satisfies the given hypotheses on xt0 . Then

it may be verified that

xt = Φ(t, t0)xt0 +

∫ t

t0

Φ(t, s)Bsusds+

∫ t

t0

Φ(t, s)Csdws, t ≥ t0, (13)

with the stated conditions on xt0 , w gives a well defined stochastic process x when

d

dtΦ(t, s) = A(t)Φ(t, s), Φ(s, s) = I,

for all t ≥ s, s, t ∈ R, defines the fundamental matrix Φ(·, ·) of A(·).

Furthermore, it may be verified that when Ext0 = 0, and Ext0xTt0

= Σ, (??) gives the

unique solution to (??) and the covariance function of {xt −∫ t

t0Φ(t, s)usds; t ≥ t0} is given

by

E(xt −∫ t

t0

Φ(t, s)usds)(xt+τ −∫ t+τ

t0

Φ(t+ τ, s)usds)T = E(Φ(t; t0)xt0)(Φ(t+ τ, t0)xt0)

T

+

∫ t

t0

Φ(t, s)CsCTs ΦT (t+ τ, s)ds,

τ > 0,

18

Page 19: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

with a corresponding expression in case τ ≤ 0. Evaluating the first covariance on the right

hand side of the equation above, and setting u ≡ 0 for simplicity, we obtain

ExtxTt+τ = Φ(t, t0)ΣΦT (t+ τ, t0) +

∫ t

t0

Φ(t, s)CsCTs ΦT (t+ τ, s)ds.

In general ( see e.g. [Ikeda-Watanabe, 1981]) it may be shown that under reasonable

smoothness and growth conditions an f(·, ·) and g(·, ·), a solution process exists for the

general non-linear SDE

dxt = f(xt, ut)dt+ g(xt, ut)dwt, t ≥ t0,

when xt0 is a random initial condition satisfying E‖xt0‖2 < ∞ and x0

∐w, and u is a

continuous function.

Continuous Time Parameter Linear Filtering: The Kalman-Bucy Filter

In analogy with the discrete time state estimation problem, the continuous time Kalman

- Bucy filtering problem is formulated as follows: the system equation is given by

dxt = Fxtdt+Q1/2dwt, t ≥ t0, xt ∈ Rn, wt ∈ Rw, (14)

and the observation equation is

dyt = Hxtdt+R1/2dvt, t ≥ t0, yt ∈ Rp, vt ∈ Rv. (15)

Here w and v are Wiener processes such that for all t, s ∈ R,

E(wt − w0)(vs − v0) = 0,

E(wt − w0)(ws − w0) = Iw min(t, s),

E(vt − v0)(vs − v0) = Iv min(t, s),

and such that

19

Page 20: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Ext0wTt = 0, Ext0v

Ts = 0 s ≥ t0, t ≥ t0. (16)

where xt0 has a joint Gaussian distribution with wt, vt for all t ≥ t0.

In this case the corresponding continuous time parameter equations generating xt = E|txt

have the well known Kalman-Bucy filter form

dxt = Fxtdt+ ∆ t[dyt −Hxtdt], t ≥ t0, (17)

with

∆ t = VtHTR−1, t ≥ t0,

where the associated Riccati equation generating the state estimation error covariance pro-

cess {Vt; t ≥ t0} is given by the deterministic equation

dVt

dt= FVt + VtF

T − VtHTR−1HVt +Q, t ≥ t0, (18)

Vt0 = E(xt0 − xt0)(xt0 − xt0)T .

For a clear derivation of these equations see e.g. [Davis]. In the continuous time case, as

in the discrete time parameter case, the filter equations for a system and observation model

with deterministic time varying matrices is obtained by merely subscripting the matrices in

(17), (18) by the time variable t.

20

Page 21: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Stationary Continuous Time Processes Generated by LSSS

We recall the following definition of a strictly stationary stochastic process.

Definition 5.2. A strictly stationary stochastic process is a stochastic process for which all

finite dimensional distributions are shift invariant with respect to the time parameter, i.e.

when T is an interval in R1 or Z, for all N ∈ Z1, all A1, · · · , AN ∈ B(Rn), all t1, · · · , tN ∈ T ,

and all τ ∈ R1 such that ti + τ ∈ T, 1 ≤ i ≤ N ,

P (xt1 ∈ A1, xt2 ∈ A2, · · · , xtN ∈ AN) = P (xt1+τ ∈ A1, xt2+τ ∈ A2, · · · , xtN+τ ∈ AN)

And we also recall that for any sequence of vector valued random variables is said to con-

verge in mean square, or in quadratic mean, to a random variable x (denoted m.s.limn→∞xn)

defined on the same underlying probability space if

limn→∞ E‖xn − x‖2 = 0.

For simplicity, we begin by considering the scalar process x generated by the SDE

dxt,−T = −αxt,−Tdt+ dwt, α > 0, x−T,−T ∇ xIC ∼ N(0,Σ), (19)

with xIC

∐w∞−T for each T ∈ R. In order to have a well defined sample path with respect

to T, T →∞, we take wt, t < 0, to be w−t for a Wiener process w on [0,∞), and for t ≥ 0,

take wt = vt, v a Wiener process independent of w. There is no loss of generality in taking

w0 = 0 since it is only the increments of w which enter (??). Its solution

xt,−T = e−α(t+T )xIC +

∫ t

−T

e−α(t−τ)dwτ

satisfies

xt,−T →∫ t

−∞e−α(t−τ)dwτ ∇ x∞t , t ∈ R.

in mean square as T →∞. In order to find the mean and covariance function of the limiting

stochastic process x∞t , we proceed as follows. Applying Lemma 5.3, and using the fact the

mean square (i.e. L2) limit of e−α(t+T )xIC is 0, we obtain

21

Page 22: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Ex∞t = Em.s.limT→∞

[e−α(t+T )xIC +

∫ t

−T

e−α(t−τ)dwτ

]= lim

T→∞E

[∫ t

−T

e−α(t−τ)dwτ

]= 0.

Applying Lemma 5.3 again yields

Ex∞t1 x∞t2

=

E

[m.s.limT→∞(e−α(t1+T )xIC +

∫ t1

−T

e−α(t1+τ)dwτ )

] [m.s.limT ′→∞(e−α(t2+T ′)xIC +

∫ t

−T ′2e−α(t2−τ)dwτ )

]

= limT,T ′→∞

E

∫ t1

−T

∫ t2

−T ′e−α(t1−τ)e−α(t2−ρ)dwτdwρ + 0,

since∫ t

−Tne−α(t−τ)dwτ and e−α(t+Tn)xIC are L2 Cauchy sequences (reader check).

Hence

Ex∞t1 x∞t2

= limT,T ′→∞

{ E∫ t1∧t2

−T

∫ t1∧t2

−T

e−α(t1−τ)−α(t2−ρ)dwτdwρ

+E

∫ t1∧t2

−T

∫ −T

−T ′e−α(t1−τ)−α(t1−ρ)dwτdwρ

+E

∫ t1∨t2

t1∧t2

∫ t1∧t2

−T ′e−α(t1−τ)−α(t2−ρ)dwτdwρ }

= limT→∞

∫ t1∧t2

−T

e−α(t1+t2)+2ατdτ + 0,

where the reader is recommended to check the decomposition of the double integral via the

appropriate diagram, and where we have taken −T > −T ′ for definiteness, the converse case

being similar. Hence

Ex∞t1 x∞t2

=

∫ t1∧t2

−∞e−α(t1+t2)+2ατdτ =

1

2αe−α(t1+t2)e2α(t1∧t2).

Let t1 < t2 (the case t2 < t1 is symmetric to this), then

Ex∞t1 x∞t2

=1

2αe−α(t2−t1) =

1

2αe−α((t2−τ)−(t1−τ)) = Ex∞t1−τx

∞t2−τ

22

Page 23: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

and hence Rt1,t2 is shift invariant. In particular, Ex2t = 1

2α.

Because L2 convergence implies convergence in probability, the FDDs of a process x∞

are the limits of the FDDs of any sequence of processes converging in mean square to x∞;

for instance, for any t1, t2 ∈ R, A1, A2 ∈ B(Rn), P (x∞t1 ∈ A1, x∞t2∈ A2) = limn→∞ P (xn

t1∈

A1, xnt2∈ A2) where xn → x∞ in m.s. as n→∞. Hence the process x∞ = {x∞t ; t ∈ R} has a

family of finite dimensional distributions which can (in principle) all be computed by taking

the limit in the calculation of all possible moments of the corresponding Gaussian process

{xt,−T ; t ≥ −T}. (Equivalently, one may consider the limit of the characteristic function

E{exp iω(M∑

j=1

αjxtj,−T)}

for all M ∈ Z1, and all (α1, · · · , αM) ∈ RM , ω ∈ R.)

But it may be verified that these limits equal the corresponding moments (or equivalently

characteristic function values) for a Gaussian process with mean 0 and covariance

Rt1,t2 = Rt2,t3 =1

2αe−α|t2−t1|, t1, t2 ∈ R.

Hence the limiting process x∞t given as the mean square limit of xt,−T , t ∈ R as T →∞, is

a Gaussian process which is strictly stationary.

23

Page 24: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Stationary Continuous Time LS3 Processes: The Lyapunov Equation

Continuous time results for (6.1), which are analogous to the discrete time results in the

previous subsection, have a somewhat more elegant appearance. Consider the continuous

time system with n dimensional state process and m dimensional Brownian motion input

given by

LS3 dxt = Axtdt+ Cdwt, t ∈ R+, (20)

with x0 ∼ N(0,Σ),Σ0 < ∞ and x0

∐w. Assume that A is asymptotically stable, i.e.

max1≤i≤n(Reλi(A)) < 0.

From (??) with u = 0 we obtain

Rt0t+τ,t = Φ(t+ τ, t0)Σ0Φ

T (t, t0) +

∫ t

t0

Φ(t+ τ, u)CCT ΦT (t, u)du.

Let Π(t, t0) ∆ Rt0t,t, then differentiation yields the (time varying) Lyapunov equation

d

dtΠ(t, t0) = AΠ(t, t0) + Π(t, t0)A

T + CCT , t ≥ t0,

Π(t0, t0) = Σ0. (21)

Next consider Π(t)−Π∞ ∆ Π(t, t0)−Π∞, where Π∞ is assumed to satisfy the (time invariant)

Lyapunov equation

Π∞AT + AΠ∞ + CCT = 0. (22)

Then

d

dt(Πt − Π∞) = ΠtA

T + AΠt + CCT − 0

= (Πt − Π∞)AT + A(Πt − Π∞),

with initial condition Πt0 − Π∞ = Σ− Π∞. Hence

Πt − Π∞ = eA(t−t0)(Σ− Π∞)eAT (t−t0), t ≥ t0,

and the asymptotic stability of A implies

Πt − Π∞ → 0

24

Page 25: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

as t → ∞. To show (??) has a solution, we observe that PT (X) ∆∫ T

0eAsCXCT eAT sds

converges to a finite limit for any matrix X, because A is asymptotically stable. But∫ T

0

d

ds

[eAsCCT eAT s

]ds = eATCCT eAT T − CCT ,

and also ∫ T

0

d

ds

[eAsCCT eAT s

]ds = A

∫ T

0

eAsCCT eAT sds+

∫ T

0

eAsCCT eAT sdsAT

∇ APT (I) + PT (I)AT .

Then since PT (I) →∫∞

0eAsCCT eAT sds and eAT

CCT eAT T → 0 as T →∞ we obtain

AP∞(I) + P∞(I)AT = −CCT , (23)

which shows (??) has a solution in the given integral form. Next, since the asymptotic

stability of A implies the solution to (??) is unique (reader check), we may identify P∞ and

P∞(I) to obtain

Πt → Π∞ = P∞(I) =

∫ ∞

0

eAsCCT eAT sds,

as t→∞.

Finally we see that the mean square limiting process x∞ satisfies

Ex∞t+τx∞T

t = Em.s.limT→∞(eAτxt,−T +

∫ t+τ

t

eA(t+τ−s)dws)m.s.limT→∞xTt,−T τ ≥ 0

= eAτ limT→∞

Ext,−TxTt,−T + 0

= eAτ [

∫ t

−∞eAsCCTAT sds] (by Lemma 5.3)

= eAτΠ∞,

and similarly

Ex∞t+τx∞T

t = Ex∞t x∞T

t−τ = Π∞e−AT τ , τ < 0.

This last property may also be seen to hold by taking the initial state of the linear system to

be distributed according to the invariant distribution N(O,Π∞). Then at any later instant

Ex∞t+τx∞T

t = eAτΠt = eAτΠ∞. These results are summarized in the theorem statement below.

25

Page 26: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Theorem 5.2

For the asymptotically stable continuous time LS3 system (??) the properties of a con-

tinuous time process x∞ corresponding to those of (i)-(iv) of Theorem 5.1 hold, with (??)

replaced by

0 = AΠ∞ + Π∞AT + CCT

and with the covariance functional relation replaced by

Ex∞t+τx∞T

t =

{eAτΠ∞, τ ≥ 0,

Π∞e−AT τ , τ < 0.

26

Page 27: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Second Order Processes

Just as for a discrete time process, a continuous time wide sense stationary Rn valued

process is defined as a second order process for which

Ext = µ, ∀t ∈ R1,

R(t, s) = E(xt − µ)(xs − µ) = E(xt−s − µ)(x0 − µ)

∇ R(t− s), ∀t, s ∈ R1.

Example 5.6

As in Theorem 5.2, the scalar LS3 process x generated by

dxt = αxtdt+ qdwt, α < 0, t ∈ R+,

with x0 ∼ N(0, π), x0

∐w and π satisfying 0 = απ+ πα+ q2, is a wss process on R+. From

the explicit calculation for the system (??), Extxs = R(t − s) = −q2

2αe−α|t−s|, and for any

distribution N(0, σ20) for the initial condition x0,

ExT+txT+s → R(t− s),

as T →∞.

Definition 5.6

An Rn valued stochastic process {xt; t ∈ R} is quadratic mean (q.m.) continuous if

E‖xt+h−xt‖2 → 0 as h→ 0 for all t ∈ R.

When it exists, the q.m. limit

d

dtxt ∆ q.m.limh→0

1

h(xt+h − xt) (24)

of a second order process x is called the quadratic mean (q.m.) derivative of x at t ∈ R. If

ddtxt exists for all t ∈ R then x is called a q.m. differentiable process.

27

Page 28: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

It may be verified that a zero mean second order process for which the covariance R(t, s)

is jointly continuous in (t, s) at (u, u) for any u ∈ R is q.m. continuous for all t ∈ R. Con-

versely, a second order process x which is q.m. continuous for all t, s ∈ R has a covariance

function which is continuous in (t, s).

Theorem 5.6

(a) A second order process x is q.m. differentiable if ∂2R(t,s)∂t∂s

exists and is continuous at

(t, t) for all t ∈ R.

(b) If a second order process x is q.m. differentiable, then the second partial differentials

taken in either order exist.

Proof

For notational convenience we shall only consider scalar processes in this proof.

(a) To begin, observe that the limit (??) exists if and only if the Cauchy condition

limh,h′→0

E(1

h(xt+h − xt)−

1

h′(xt+h′ − xt))

2 = 0, (25)

holds.

Now the existence and continuity at (t, t) of the mixed partial derivative ∂2R(t,s)∂t∂s

of R(t, s)

implies that its value is independent of the order of evaluation of the partial derivatives.

This implies that a unique limit

lim(h,h′)→(0,0)h′−1{h−1[R(t+ h′, t+ h)−R(t+ h′, t)]− h−1[R(t, t+ h)−R(t, t)]}

= lim(h,h′)→(0,0)(hh′)−1[(Ext+h′xt+h−Ext+h′xt)−(Extxt+h−Extxt)] = lim(h,h′)→0E ∆ (h, h′)

28

Page 29: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

exists for all sequences {(h, h′)} converging to (0, 0), where

∆ (h, h′) ∆ (hh′)−1(xt+h − xt)(xt+h′ − xt).

Hence

lim(h,h′)→(0,0)(E[ ∆ (h, h) + ∆ (h′, h′)− 2 ∆ (h, h′)])

= limh→0

E ∆ (h, h) + limh→0

E ∆ (h′, h′)− lim(h,h′)→(0,0)2E ∆ (h, h′) = 0.

However, the left-most expression above is equal to E( 1h(xt+h − xt)) − ( 1

h′(xt+h′ − xt))

2

(reader check), and hence the proven convergence to zero establishes that the differences

{h−1(xt+h − xt);h ∈ R} form an L2 Cauchy sequence as h → 0. This establishes the exis-

tence of the q.m. differential of the process x for t ∈ R+.

(b) If the process x is q.m. differentiable, the approximating differences form an L2

Cauchy sequence as h → 0. Hence the approximating differences for the second partial dif-

ferentials in either order exist, as required.

For processes which are q.m. differentiable we evidently have

E(d

dtx)2 =

∂2R(t, t)

∂s∂t|t=s ∀t ∈ R+.

Example 5.6

A Wiener process is q.m. continuous at any t ∈ R1+, and this fact is implied by the joint

continuity of the covariance function min(t, s)I; t, s ∈ R+. However, the second partial

derivatives do not exist for the covariance function of a Wiener process, and this implies

that it is not q.m. differentiable. This, of course, is evident by a direct calculation involving

approximating differences of the Wiener process.

29

Page 30: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Spectral Theory for WSS Continuous Time Stochastic Process

Let x be an Rn valued zero mean q.m. continuous wss stochastic processes with matrix

covariance function R(τ); τ ∈ R1. Assume∫∞−∞ ‖R(τ)‖dτ ≤ ∞.

Since x is q.m. continuous, R(.) is continuous and so R(.) ∈ L1∩C0, hence we may define

the spectral density matrix of x via

Φ(ω) ∆

∫ ∞

−∞e−2πiωtR(t)dt, ω ∈ R.

When iω replaces eiθ in the matrix function {Φ(ω);ω ∈ R} it can be seen that the three

conjugate transpose, conjugate and positivity properties of a spectral density matrix hold as

given in Definition 5.3 for (discrete time) spectral density matrices; namely Φ(.) is a positive

complex Hermitian matrix.

Since R(.) ∈ L1∩C0, it may be verified that Φ(.) ∈ L1∩C0, and so the inversion formula

Ext+τxt = R(τ) =

∫ ∞

−∞e2πiωtΦ(ω)dω, ω ∈ R,

holds. Next, in analogy with the discrete time case, we may consider the action of a (not

necessarily non-anticipative) linear system with L2 matrix impulse response {H(τ); τ ∈ R}

on a wss stochastic process satisfying the hypotheses of this section.

In the time domain we may define the output process y by

yt =

∫ ∞

−∞H(τ)x(t− τ)dτ, τ ∈ R.

Since H(.) ∈ L2 the transfer function

H(ω) ∆

∫ ∞

−∞e−2πiωtH(t)dt, ω ∈ R,

also lies in L2. Finally this gives the fundamental relation

30

Page 31: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Ry(τ) = Eyt+τyt

= E(

∫ ∞

−∞H(p)x(t+ τ − p)dp)(

∫ ∞

−∞H(q)x(t− q)dp)

=

∫ ∞

−∞e2πiωtH(ω)Φx(ω)H(ω)

Tdω, ω ∈ R.

The formula above immediately yields the continuous time version of the Wiener-Khinchin

formula for the class of processes under consideration; namely

Φy(ω) = H(ω)Φx(ω)H(ω)T, ω ∈ R.

Orthogonal Representations: The Karhunen-Loeve Expansion

Again in this section we assume all processes to have zero mean. Let x be a q.m.

continuous second order (not necessarily stationary) stochastic process taking values in the

Hilbert space H. For convenience we take x to be a scalar process. We seek a set of random

second order time independent basis functions for H, say {ψn(ω);n ∈ Z+}, such that a

representation of the form

xt(ω) =∞∑

n=0

αn,tψn(ω) t ∈ R+. (26)

holds for some set of time dependent coefficients {αn,t;n ∈ Z+} which are non-random.

Moreover, we seek an orthonormal (o.n.) complex valued family {Zn(ω);n ∈ Z+} such that

E|Zn(ω)|2 = 1,n ∈ Z+,

and

〈Zn, Zm〉 ∆ EZn(ω)Zm(ω) = 0 n 6= m, n,m ∈ Z+.

Consider the non-random set of time functions

σn(t) ∆ 〈xt, Zn〉 = Ext(ω)Zn(ω), n ∈ Z+

31

Page 32: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

where it is assumed that {Zn;n ∈ Z+} is an o.n. family.

Then

0 ≤ E|xt −∞∑

n=0

〈xt, Zn〉Zn|2

= E|xt|2 − 2∞∑

n=0

〈xt, Zn〉〈xt, Zn〉+∞∑

n=0

|〈xt, Zn〉|2

= E|xt|2 −∞∑

n=0

|〈xt, Zn〉|2.

Consider the Hilbert space H and assume that for all xt ∈ H

‖xt‖2 = 〈xt, xt〉 =∞∑

n=0

|〈xt, Zn〉|2;

then we say {Zn;n ∈ Z+} is a complete o.n. (c.o.n.) family for the space H.

In this case we necessarily have

xt(ω) = q.m.limN→∞

N∑n=0

σn(t)Zn(ω)

∇∞∑

n=0

σn(t)Zn(ω).

The functions {σn(t); t ∈ R} may be verified to be continuous because x is q.m. continuous.

(Reader check.)

Assume xt is not contained in any proper subspace of {Zn}∞n=0. Then the functions σn(·)

are linearly independent; otherwise

N∑n=0

anσn(t) = 0, for some {an}N0 ,

and hence

0 =N∑

n=0

an〈xt, Zn〉 = 〈xt,

N∑n=0

anZn〉;

which implies

xt ⊥ {N∑

n=0

anZn} while xt =∞∑

n=0

αn(t)Zn.

32

Page 33: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

But this contradicts the proper subspace condition.

Separable Covariance Function

Suppose xt =∑∞

n=0 σn(t)Zn, t ∈ R, where {Zn;n ∈ Z+} is a c.o.n. family. Then

R(t, s) = Extxs = E[∞∑

n=0

σn(t)Zn][∞∑

m=0

σm(s)Zm]

=∞∑

n=0

σn(t)σn(s), ∀t, s ∈ R. (27)

A covariance function of the form (??) is called separable.

Mercer’s Theorem [ ] states that continuous positive functions R(s, t); s, t ∈ R on L2×L2

possess complete orthonormal families of eigenfunctions with respect to which they have

expansions of the form (??) which converge uniformly over compact sets of the form {−M ≤

s, t ≤M ; s, t ∈ R}. Hence, in particular, continuous covariance functions are separable.

Clearly, if

R(s, t) =∞∑

n=−∞

σn(s)σn(t), s, t ∈ R, (28)

with∫∞−∞ σn(s)σm(s)ds = λnδnm, n,m ∈ Z, then∫ ∞

−∞R(s, t)σn(t)dt =

∫ ∞

−∞

(∞∑

k=0

σk(s)σk(t)

)σn(t)dt

=∞∑

k=0

σk(s)λnδk,n

= λnσn(s), ∀s ∈ R.

Theorem 5.7 Karhunen-Loeve

Let x be a q.m. continous second order process with covariance function R(t, s), t, s ∈ R.

(a) Let {ψn;n ∈ Z+} be the set of orthonormal eigenfunctions of R(., .) such that∫ ∞

−∞R(t, s)ψn(s)ds = λnψn(t), ∀t ∈ R,

33

Page 34: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

i.e. for which {λn;n ∈ Z+} is the set of eigenvalues. Further let

bn(ω) =√λn

−1∫ ∞

−∞x(ω, t)ψn(t)dt

for which, necessarily,

Ebnbm = δm,n, m, n ∈ Z+.

Then

x(ω, t) = q.m.limN→∞

N∑n=0

√λnψn(t)bn(ω) (29)

uniformly on compact intervals.

(b) Conversely, if x(ω, t) has an expansion of the form∫ ∞

−∞ψm(t)ψn(t)dt = δmn = Ebmbn m,n ∈ Z+,

then {ψn;n ∈ Z+} and the associated {λn;n ∈ Z+} are the eigenfunctions and eigen-

values respectively of R(·, ·).

Note that for real processes x, ψ and λ are real.

Proof

(a)

From the hypotheses of the theorem and the definition of the terms, we see that

{bn;n ∈ Z+} is a c.o.n. for the Hilbert space H spanned by x. Consequently, the

result follows directly from

E|xt −N∑

n=0

√λnψn(t)bn|2

=R(t, t)−N∑

n=0

λn|ψn(t)|2 → 0

as N →∞, where the latter convergence follows from Mercer’s Theorem.

34

Page 35: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

(b) If xt =∑∞

n=0

√λnψn(t)bn(ω), then

R(t, s) = Extxs =∞∑

n=0

λnψn(t)ψn(s).

Hence∫ ∞

−∞R(t, s)ψm(s)ds =

∫ ∞

−∞

∞∑n=0

λnψn(t)ψn(s)ψm(s)ds

= λmψm(t), ∀m ∈ Z+,∀t ∈ R.

Example 5.7 (Wong [1971, p.87])

Let R(t, s) = min(t, s), t, s ∈ R; in other words R(., .) is the covariance of the Wiener

process. Consider ∫ T

0

min(t, s)ψ(s)ds = λψ(t), 0 ≤ t ≤ T,

or, equivalently,∫ t

0

sψ(s)ds+ t

∫ T

t

ψ(s)ds = λψ(t), 0 ≤ s, t ≤ T.

This gives

tψ(t)− tψ(t) +

∫ T

t

ψ(s)ds = λd

dtψ(t),

and so

ψ(t) = −1

λψ(t), λ 6= 0,

with ψ(0) = 0, ddtψ(T ) = 0. This gives

ψ(t) = A sin1√λt, with cos

T√λ

= 0, implying√λ =

2T

(2n+ 1)π, n ∈ Z+,

where here, since sin(−x) = − sinx, we do not need to consider the negative integers n ∈ Z

in order to find additional eigenfunctions.

So, on Z+, the set of normalized eigenfunctions are

35

Page 36: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

ψn(t) =

(2

T

)1/2

sin

(2n+ 1

2

)(πt

T

), n ∈ Z+,

and hence the process x with the specified covariance R(t, s) satisfies

xt = q.m.limN→∞

N∑n=0

(2T

(2n+ 1)π

)(2

T

)1/2(sin

(2n+ 1)

T

(πt

2

))bn(ω),

where

bn(ω) =

(2T

(2n+ 1)π

)−1 ∫ T

0

(2

T

)1/2(sin

(2n+ 1

2

)(πt

T

))x(ω, t)dt

in q.m. Incidentally, as observed by Wong, Mercer’s Theorem in this case gives the expansion

min(t, s) =2

T

∞∑n=0

T 2

π2(n+ 12)2

(sin(n+

1

2)πt

T

)(sin(n+

1

2)πs

T

),

where the convergence above is uniform on [0, T ]2.

Weiner-Kolmogorov Filtering

36

Page 37: Chapter 5 Brownian Motion and the Wiener Processcim.mcgill.ca/~peterc/04lectures510/04five.pdf · 2004. 3. 23. · Chapter 5 Brownian Motion and the Wiener Process In continuous time,

Figure 1: Self-similarity of the Wiener process

37