Αλυσίδες Markov 1

8/20/2019 Αλυσίδες Markov 1

1/12

Markov

{X (t)} (Ω, F , P ). t

t, X (t) : Ω → R X (t)− 1(B ) ∈ F Borel B ∈ B (R )

t n t n = 0 , 1, . . . , {X n }n ≥ 0

t t ∈ R , t ≥ 0, {X (t)}t ≥ 0

{X n } {X (t)}

(Ω, F , P ).

Markov Markov

• {X n }n ≥ 0 {X (t)}t ≥ 0 S

X n = s j X (t) = s j s j ∈ S, X n X (t) s j

n t

• Markov

{X n }n ≥ 0 Markov

(i {X n }n ≥ 0 S.

(ii n = 0 , 1, . . . , si , s i 0 , s i 1 , . . . , s i n − 1 , s j ∈ S, Markov

P (X n +1 = s j | X n = s i , X n − 1 = s i n − 1 , . . . , X 1 = s i 1 , X 0 = s i 0 ) = P (X n +1 = s j | X n = s i ).

{X (t)}t ≥ 0 Markov

(i {X (t)}t ≥ 0 S.


2/12

(ii 0 ≤ t0 < t 1 < . . . < t n si , s i 0 , s i 1 , . . . , s i n − 1 , s j ∈ S, Markov

P (X t n = s j | X t n − 1 = s i , X t n − 2 = s i n − 1 , . . . , X t 0 = s i 0 ) = P (X t n = s j | X t n − 1 = s i ).

Markov Markov S = {1, 2, . . .}, S = {1, 2, . . . , |S |} , |S | < ∞ ,

S |S | = ∞ , S

Markov {X n }n ≥ 0 S = {1, 2, . . .} n ≥ 1 i, j ∈ S

P (X n +1 = j | X n = i) = P (X 1 = j | X 0 = i).

Markov

Markov {X n }n ≥ 0 S = {1, 2, . . .}. P

P ij = P (X n +1 = j | X n = i), i, j ∈ S n ≥ 0,

S |S | = k, P k × k, S

P

P

• P P ij ≥ 0 i, j ∈ S.

• P 1 j∈S P ij = 1 , i ∈ S. m, n ≥ 0 P (m, m + n)

P ij (m, m + n) = P (X m + n = j | X m = i), i, j ∈ S,

n n Markov

P (m, m +1) = P. P (m, m + n) m

Chapman-Kolmogorov m,n,r ≥ 0

P ij (m, m + n + r ) =l∈S

P i l (m, m + n) P lj (m + n, m + n + r ).


3/12

P (m, m + n + r ) = P (m, m + n) P (m + n, m + n + r )

P (m, m + n) = P n , n− P.

P (n) P (0, n ),

P ij (n) = P ij (0, n ), i, j ∈ S n ≥ 0,

Chapman-Kolmogorov

P (n) = P (0, n ) = P (m, m + n) = P n , n, m = 0 , 1, . . .

n = 0 , 1, . . . ,

µ(n )

= ( µ(n )i : i ∈ S )

X n µ(n )i = P (X n = i),

i ∈ S n = 0 , 1, . . . ),

µ(m + n ) = µ(m )P (n) µ(n ) = µ(0) P n .

Markov

{X n }n ≥ 0

(i

(ii P (X i ≤ x) =P (X j ≤ x) i, j = 0 , 1, . . . x ∈ R .

Y 0, Z . {X n }n ≥ 1,

{− 1, +1 }

n = 1 , 2, . . . ,

P (X n = +1) = p, P (X n = − 1) = q,

p, q ∈ (0, 1), p + q = 1 . {Y n }n ≥ 0,

Y n +1 = Y n + X n +1 , Y n = Y 0 +n

i=1

X i ,

Z p ∈ (0, 1). p = 12 ,


4/12

{Y n }n ≥ 0 Z p ∈ (0, 1) Markov S = Z

P ij = p, j = i + 1 ,q = 1 − p, j = i − 1,0,

i, j ∈ Z n = 0 , 1, . . . ,

P ij (n) = n

12 (n + j − 1)

p12 (n + j − 1) q

12 (n − j +1) , n + j − 1 ≥ 0 ,

0,

P ij (n) = P (X n = j | X 0 = i)

{Y n }n ≥ 0 Z p ∈ (0, 1) i ∈ Z .

(i limn →∞ P ii (n) = 0

(ii i,

P ii (m) = 1 − | p − q |, m ≥ 1,

P ii (n) = P (Y n = i | Y 0 = i) n ≥ 1.

p = q = 12 ,

Markov

Markov {X n }n ≥ 0 S = {1, 2, . . .} P.

i ∈ S recurrent

P ii (n) = P (X n = i | X 0 = i) = 1 , n ≥ 1.

i transient

{Y n }n ≥ 0 Z p ∈ (0, 1). i ∈ Z p = 12 ,

Markov


5/12

i, j ∈ S n ≥ 1, i n

j

f ij (n) = P (X n = j, X n − 1 = j, . . . , X 1 = j | X 0 = i),

i j

f ij =∞

n =1f ij (n).

i f ii = 1 f ii < 1.

n,

P ij (s) =∞

n =0sn P ij (n), F ij (s) =

∞

n =0sn f ij (n),

P ij (n) = P (X n = j | X 0 = i) P ij (0) = δ ij , Kronecker f ij (0) = 0 , i, j ∈ S. |s | < 1

f ij = F ij (1) .

i ∈ S

(i P ii (s) = 1 + F ii (s)P ii (s),

(ii P ij (s) = F ij (s)P jj (s), j = i.

i ∈ S

(i i ∞n =0 P ii (n) = ∞

(ii i ∞n =0 P ii (n) < ∞ .

i ∞n =0 P ij (n) < ∞ , j i limn →∞ P ij (n) = 0 , j

i ∈ S

(i i

P (X n = i, n | X 0 = i) = 1 ,

(ii j

P (X n = i, n | X 0 = i) = 0 .


6/12

i, j ∈ S, j, i,

T ij = min {n ≥ 1 : X n = j, X 0 = i},

T ij = ∞ , j.

τ ij = E [T ij ].

i = j, τ i = τ ii i.

i ∈ S

τ i = ∞n =1 nf ii (n), i ,

∞ , i .

τ i = F

ii (1) .

i ∈ S null re-current τ i = ∞ , non-null recurrent

τ i < ∞ .

Z

i ∈ S limn →∞ P ii (n) = 0 . limn →∞ P ji (n) = 0 , j ∈ S.

S Markov

A gcd A A

d( i) i ∈ S

d( i) = gcd {n ≥ 1 : P ii (n) > 0}.

i d(i) ≥ 2 d(i) = 1

i i

P ii (n) = 0 n d(i)

Z

2 p = 12 p = 12


7/12

i

Markov {X n }n ≥ 0 S = {1, 2, . . .} P.

i, j ∈ S i j n ≥ 1

P ij (n) = P (X n = j | X 0 = i) > 0.

i j “i → j ” j i “i ↔ j ” i j

“↔” S × S S × S S

i ↔ j

(i i j

(ii i j

(iii i j

(iv i j

(v i j d(i) = d( j )

(vi i j

C Markov C

• C P ij = 0 i ∈ C j /∈ C

• absorbing C

• irreducible i, j ∈ C i ↔ j i, j ∈ C

Markov irreducible i, j ∈ S i ↔ j

reducible


8/12

S Markov

S = T ∪N

j =1

C j ,

T, C 1, . . . , C N T C 1, . . . , C N

S = {1, 2, 3, 4} Markov

P =

14

14

14

14

0 0 1 00 0 0 11 0 0 0

.

1

1

f 11 (1) = f 11 (2) = f 11 (3) = f 11 (4) = 1 / 4, f 11 (n) = 0 , n ≥ 5, f 11 =

∞n =1 f 11 (n) = 1 1 τ 1 =

∞n =1 nf 11 (n) = 1 / 4 + 1 / 2 + 3 / 4 + 1 = 10 / 4 < ∞ 1

P 11 (1) > 0 1

S = {1, 2, 3, 4} Markov

P =

0 0 1212

0 1 0 00 0 0 11 0 0 0

.

(i) (ii)

(i) 2 P 22(1) > 0 {1, 3, 4}

1, 3 4 d(1) = d(3) = gcd {2, 3, . . .} = 1 d(4) = gcd {3, 5, . . .} = 1 1, 3 4

(ii)

f 11 (1) = 0 , f 11 (2) = 1 / 2, f 11 (3) = 1 / 2, f 11(n) = 0 , n ≥ 4,f 22(1) = 1 , f 22(n) = 0 , n ≥ 2,f 33(1) = 0 , f 33(2k) = 0 , k ≥ 1, f 33(2k + 1) = (1 / 2)k , k ≥ 1,f 44(1) = 0 , f 44(2) = 1 / 2, f 44(3) = 1 / 2, f 44(n) = 0 , n ≥ 4.


9/12

i τ i = ∞n =1 nf ii (n) τ 1 = 5 / 2, τ 2 = 1 , τ 3 =

5, τ 4 = 5 / 2

S = {1, 2, 3, 4, 5, 6} Markov

P =

12

12 0 0 0 0

14

34 0 0 0 0

14

14

14

14 0 0

14 0

14

14 0

14

0 0 0 0 1212

0 0 0 0 1212

.

(i) (ii)

(i) {1, 2} {5, 6}

3 4 {1, 2} {5, 6} 1 pii (1) > 0, i 3 4

1 2 5 6

(ii)

f 11 (n) = p11 = 12 , n = 1 , p12( p22)n − 2 p21 = 12 (

34 )

n − 2 14 , n ≥ 2.

1 τ 1 = ∞n =1 nf 11 (n) = 19 / 6

Markov {X n }n ≥ 0 S = {1, 2, . . .} P.

π = ( π j : j ∈ S ) Markov π

(i

π j ≥ 0,

j ∈ S

j∈S π j = 1

(ii πP = π,

i∈S

π i P ij = π j , j ∈ S.

π X n µ(0) = π µ(n ) = πP n = π

n > 0 π Markov

i, j ∈ S


10/12

(i limn →∞ P ij (n)

(ii i

limn →∞

P ij (n) = π j ,

π j Markov π j = 0 j ∈ S

Markov

π i π(i) > 0

Markov

π

π i = τ − 1i , i ∈ S,

τ i i

π = πP

Markov x x = xP

k ρi(k) i

k ρi (k) = E [N i | X 0 = k]

N i =∞

n =1I {X n = i}∩{T k ≥ n } ,

T k k N k = 1 ρk (k) = 1

ρi (k) =∞

n =1P (X n = i, T k ≥ n | X 0 = k).

k

T k = i∈S N i

τ k =i∈S

ρi (k).

k Markov ρ(k) = ( ρi (k) : i ∈ S ) ρ(k) = ρ(k)P ρi (k) < ∞ i ∈ S

k τ k < ∞ πi = ρi (k)/τ k


11/12

Markov x x = xP

i∈S xi < ∞ i∈S xi = ∞

s ∈ S

(i {y j : j = s} |y j | ≤ 1 j = s

yi = j : j = s

P ij y j , i = s,

(ii {y j : j = s} lim j →∞ y j = ∞ j = s

yi ≥ j : j = s

P ij y j , i = s.

S = {0, 1, 2, . . .}

P 0,0 = q, P i,i +1 = p, i ≥ 0, P i,i − 1 = q, i ≥ 1,

p, q ∈ (0, 1), p + q = 1 ρ = p/q

(i q < p s = 0 y j = 1 − ρ− j (i)

(ii π = πP π j =ρ j (1 − ρ) q > p

q > p

(iii q = p = 12 s = 0 y j = j j ≥ 1 (ii)

Markov S N P ij (N ) > 0 i, j ∈ S

limn →∞ P ij (n) = π j i, j ∈ S π j > 0 j ∈ S j∈S π j = 1

Markov

Markov


12/12

Markov πi

π i = τ − 1i , i ∈ S.

Markov

(i

(ii

(iii

Markov i, j ∈ S

limn →∞

P ij (n) = τ − 1 j

π π j = τ − 1 j j ∈ S

Markov π j = lim n →∞ P ij (n) = τ − 1 j i, j ∈ S

Markov

Αλυσίδες Markov 1

Documents

Transcript of Αλυσίδες Markov 1