8/20/2019 Αλυσίδες Markov 1
1/12
Markov
{X (t)} (Ω, F , P ). t
t, X (t) : Ω → R X (t)− 1(B ) ∈ F Borel B ∈ B (R )
t n t n = 0 , 1, . . . , {X n }n ≥ 0
t t ∈ R , t ≥ 0, {X (t)}t ≥ 0
{X n } {X (t)}
(Ω, F , P ).
Markov Markov
• {X n }n ≥ 0 {X (t)}t ≥ 0 S
X n = s j X (t) = s j s j ∈ S, X n X (t) s j
n t
• Markov
{X n }n ≥ 0 Markov
(i {X n }n ≥ 0 S.
(ii n = 0 , 1, . . . , si , s i 0 , s i 1 , . . . , s i n − 1 , s j ∈ S, Markov
P (X n +1 = s j | X n = s i , X n − 1 = s i n − 1 , . . . , X 1 = s i 1 , X 0 = s i 0 ) = P (X n +1 = s j | X n = s i ).
{X (t)}t ≥ 0 Markov
(i {X (t)}t ≥ 0 S.
8/20/2019 Αλυσίδες Markov 1
2/12
(ii 0 ≤ t0 < t 1 < . . . < t n si , s i 0 , s i 1 , . . . , s i n − 1 , s j ∈ S, Markov
P (X t n = s j | X t n − 1 = s i , X t n − 2 = s i n − 1 , . . . , X t 0 = s i 0 ) = P (X t n = s j | X t n − 1 = s i ).
Markov Markov S = {1, 2, . . .}, S = {1, 2, . . . , |S |} , |S | < ∞ ,
S |S | = ∞ , S
Markov {X n }n ≥ 0 S = {1, 2, . . .} n ≥ 1 i, j ∈ S
P (X n +1 = j | X n = i) = P (X 1 = j | X 0 = i).
Markov
Markov {X n }n ≥ 0 S = {1, 2, . . .}. P
P ij = P (X n +1 = j | X n = i), i, j ∈ S n ≥ 0,
S |S | = k, P k × k, S
P
P
• P P ij ≥ 0 i, j ∈ S.
• P 1 j∈S P ij = 1 , i ∈ S. m, n ≥ 0 P (m, m + n)
P ij (m, m + n) = P (X m + n = j | X m = i), i, j ∈ S,
n n Markov
P (m, m +1) = P. P (m, m + n) m
Chapman-Kolmogorov m,n,r ≥ 0
P ij (m, m + n + r ) =l∈S
P i l (m, m + n) P lj (m + n, m + n + r ).
8/20/2019 Αλυσίδες Markov 1
3/12
P (m, m + n + r ) = P (m, m + n) P (m + n, m + n + r )
P (m, m + n) = P n , n− P.
P (n) P (0, n ),
P ij (n) = P ij (0, n ), i, j ∈ S n ≥ 0,
Chapman-Kolmogorov
P (n) = P (0, n ) = P (m, m + n) = P n , n, m = 0 , 1, . . .
n = 0 , 1, . . . ,
µ(n )
= ( µ(n )i : i ∈ S )
X n µ(n )i = P (X n = i),
i ∈ S n = 0 , 1, . . . ),
µ(m + n ) = µ(m )P (n) µ(n ) = µ(0) P n .
Markov
{X n }n ≥ 0
(i
(ii P (X i ≤ x) =P (X j ≤ x) i, j = 0 , 1, . . . x ∈ R .
Y 0, Z . {X n }n ≥ 1,
{− 1, +1 }
n = 1 , 2, . . . ,
P (X n = +1) = p, P (X n = − 1) = q,
p, q ∈ (0, 1), p + q = 1 . {Y n }n ≥ 0,
Y n +1 = Y n + X n +1 , Y n = Y 0 +n
i=1
X i ,
Z p ∈ (0, 1). p = 12 ,
8/20/2019 Αλυσίδες Markov 1
4/12
{Y n }n ≥ 0 Z p ∈ (0, 1) Markov S = Z
P ij = p, j = i + 1 ,q = 1 − p, j = i − 1,0,
i, j ∈ Z n = 0 , 1, . . . ,
P ij (n) = n
12 (n + j − 1)
p12 (n + j − 1) q
12 (n − j +1) , n + j − 1 ≥ 0 ,
0,
P ij (n) = P (X n = j | X 0 = i)
{Y n }n ≥ 0 Z p ∈ (0, 1) i ∈ Z .
(i limn →∞ P ii (n) = 0
(ii i,
P ii (m) = 1 − | p − q |, m ≥ 1,
P ii (n) = P (Y n = i | Y 0 = i) n ≥ 1.
p = q = 12 ,
Markov
Markov {X n }n ≥ 0 S = {1, 2, . . .} P.
i ∈ S recurrent
P ii (n) = P (X n = i | X 0 = i) = 1 , n ≥ 1.
i transient
{Y n }n ≥ 0 Z p ∈ (0, 1). i ∈ Z p = 12 ,
Markov
8/20/2019 Αλυσίδες Markov 1
5/12
i, j ∈ S n ≥ 1, i n
j
f ij (n) = P (X n = j, X n − 1 = j, . . . , X 1 = j | X 0 = i),
i j
f ij =∞
n =1f ij (n).
i f ii = 1 f ii < 1.
n,
P ij (s) =∞
n =0sn P ij (n), F ij (s) =
∞
n =0sn f ij (n),
P ij (n) = P (X n = j | X 0 = i) P ij (0) = δ ij , Kronecker f ij (0) = 0 , i, j ∈ S. |s | < 1
f ij = F ij (1) .
i ∈ S
(i P ii (s) = 1 + F ii (s)P ii (s),
(ii P ij (s) = F ij (s)P jj (s), j = i.
i ∈ S
(i i ∞n =0 P ii (n) = ∞
(ii i ∞n =0 P ii (n) < ∞ .
i ∞n =0 P ij (n) < ∞ , j i limn →∞ P ij (n) = 0 , j
i ∈ S
(i i
P (X n = i, n | X 0 = i) = 1 ,
(ii j
P (X n = i, n | X 0 = i) = 0 .
8/20/2019 Αλυσίδες Markov 1
6/12
i, j ∈ S, j, i,
T ij = min {n ≥ 1 : X n = j, X 0 = i},
T ij = ∞ , j.
τ ij = E [T ij ].
i = j, τ i = τ ii i.
i ∈ S
τ i = ∞n =1 nf ii (n), i ,
∞ , i .
τ i = F
ii (1) .
i ∈ S null re-current τ i = ∞ , non-null recurrent
τ i < ∞ .
Z
i ∈ S limn →∞ P ii (n) = 0 . limn →∞ P ji (n) = 0 , j ∈ S.
S Markov
A gcd A A
d( i) i ∈ S
d( i) = gcd {n ≥ 1 : P ii (n) > 0}.
i d(i) ≥ 2 d(i) = 1
i i
P ii (n) = 0 n d(i)
Z
2 p = 12 p = 12
8/20/2019 Αλυσίδες Markov 1
7/12
i
Markov {X n }n ≥ 0 S = {1, 2, . . .} P.
i, j ∈ S i j n ≥ 1
P ij (n) = P (X n = j | X 0 = i) > 0.
i j “i → j ” j i “i ↔ j ” i j
“↔” S × S S × S S
i ↔ j
(i i j
(ii i j
(iii i j
(iv i j
(v i j d(i) = d( j )
(vi i j
C Markov C
• C P ij = 0 i ∈ C j /∈ C
• absorbing C
• irreducible i, j ∈ C i ↔ j i, j ∈ C
Markov irreducible i, j ∈ S i ↔ j
reducible
8/20/2019 Αλυσίδες Markov 1
8/12
S Markov
S = T ∪N
j =1
C j ,
T, C 1, . . . , C N T C 1, . . . , C N
S = {1, 2, 3, 4} Markov
P =
14
14
14
14
0 0 1 00 0 0 11 0 0 0
.
1
1
f 11 (1) = f 11 (2) = f 11 (3) = f 11 (4) = 1 / 4, f 11 (n) = 0 , n ≥ 5, f 11 =
∞n =1 f 11 (n) = 1 1 τ 1 =
∞n =1 nf 11 (n) = 1 / 4 + 1 / 2 + 3 / 4 + 1 = 10 / 4 < ∞ 1
P 11 (1) > 0 1
S = {1, 2, 3, 4} Markov
P =
0 0 1212
0 1 0 00 0 0 11 0 0 0
.
(i) (ii)
(i) 2 P 22(1) > 0 {1, 3, 4}
1, 3 4 d(1) = d(3) = gcd {2, 3, . . .} = 1 d(4) = gcd {3, 5, . . .} = 1 1, 3 4
(ii)
f 11 (1) = 0 , f 11 (2) = 1 / 2, f 11 (3) = 1 / 2, f 11(n) = 0 , n ≥ 4,f 22(1) = 1 , f 22(n) = 0 , n ≥ 2,f 33(1) = 0 , f 33(2k) = 0 , k ≥ 1, f 33(2k + 1) = (1 / 2)k , k ≥ 1,f 44(1) = 0 , f 44(2) = 1 / 2, f 44(3) = 1 / 2, f 44(n) = 0 , n ≥ 4.
8/20/2019 Αλυσίδες Markov 1
9/12
i τ i = ∞n =1 nf ii (n) τ 1 = 5 / 2, τ 2 = 1 , τ 3 =
5, τ 4 = 5 / 2
S = {1, 2, 3, 4, 5, 6} Markov
P =
12
12 0 0 0 0
14
34 0 0 0 0
14
14
14
14 0 0
14 0
14
14 0
14
0 0 0 0 1212
0 0 0 0 1212
.
(i) (ii)
(i) {1, 2} {5, 6}
3 4 {1, 2} {5, 6} 1 pii (1) > 0, i 3 4
1 2 5 6
(ii)
f 11 (n) = p11 = 12 , n = 1 , p12( p22)n − 2 p21 = 12 (
34 )
n − 2 14 , n ≥ 2.
1 τ 1 = ∞n =1 nf 11 (n) = 19 / 6
Markov {X n }n ≥ 0 S = {1, 2, . . .} P.
π = ( π j : j ∈ S ) Markov π
(i
π j ≥ 0,
j ∈ S
j∈S π j = 1
(ii πP = π,
i∈S
π i P ij = π j , j ∈ S.
π X n µ(0) = π µ(n ) = πP n = π
n > 0 π Markov
i, j ∈ S
8/20/2019 Αλυσίδες Markov 1
10/12
(i limn →∞ P ij (n)
(ii i
limn →∞
P ij (n) = π j ,
π j Markov π j = 0 j ∈ S
Markov
π i π(i) > 0
Markov
π
π i = τ − 1i , i ∈ S,
τ i i
π = πP
Markov x x = xP
k ρi(k) i
k ρi (k) = E [N i | X 0 = k]
N i =∞
n =1I {X n = i}∩{T k ≥ n } ,
T k k N k = 1 ρk (k) = 1
ρi (k) =∞
n =1P (X n = i, T k ≥ n | X 0 = k).
k
T k = i∈S N i
τ k =i∈S
ρi (k).
k Markov ρ(k) = ( ρi (k) : i ∈ S ) ρ(k) = ρ(k)P ρi (k) < ∞ i ∈ S
k τ k < ∞ πi = ρi (k)/τ k
8/20/2019 Αλυσίδες Markov 1
11/12
Markov x x = xP
i∈S xi < ∞ i∈S xi = ∞
s ∈ S
(i {y j : j = s} |y j | ≤ 1 j = s
yi = j : j = s
P ij y j , i = s,
(ii {y j : j = s} lim j →∞ y j = ∞ j = s
yi ≥ j : j = s
P ij y j , i = s.
S = {0, 1, 2, . . .}
P 0,0 = q, P i,i +1 = p, i ≥ 0, P i,i − 1 = q, i ≥ 1,
p, q ∈ (0, 1), p + q = 1 ρ = p/q
(i q < p s = 0 y j = 1 − ρ− j (i)
(ii π = πP π j =ρ j (1 − ρ) q > p
q > p
(iii q = p = 12 s = 0 y j = j j ≥ 1 (ii)
Markov S N P ij (N ) > 0 i, j ∈ S
limn →∞ P ij (n) = π j i, j ∈ S π j > 0 j ∈ S j∈S π j = 1
Markov
Markov
8/20/2019 Αλυσίδες Markov 1
12/12
Markov πi
π i = τ − 1i , i ∈ S.
Markov
(i
(ii
(iii
Markov i, j ∈ S
limn →∞
P ij (n) = τ − 1 j
π π j = τ − 1 j j ∈ S
Markov π j = lim n →∞ P ij (n) = τ − 1 j i, j ∈ S
Markov
Top Related