A First Course in Stochastic Processes Chapter Two: Markov Chains.
-
Upload
lester-weaver -
Category
Documents
-
view
232 -
download
0
Transcript of A First Course in Stochastic Processes Chapter Two: Markov Chains.
A First Course in Stochastic Processes
Chapter Two: Markov Chains
xNx
N
p
N
p
x
NpXxX
1|Pr 12
N
XNX 1
2 ,Bin~
X2=X1=1 X2=2
N
XNX 2
3 ,Bin~
X3=1 X4=3
X1
N
XN 1,Bin X2
N
XN 2,Bin X3
N
XN 3,Bin
X4
N
XN 4,Bin X5
N
XN 5,Bin etc
xNx
tt N
p
N
p
x
NpXxX
1|Pr 1
P =
jitt iXjX ,1 |Pr P
tX1tX 0 1 2 3 4 5 6 … 20
0 1 0 0 0 0 0 0 … 0
1 0.358 0.377 0.189 0.0596 0.0133 0.00224 0.000295 … 0.000
2 0.122 0.27 0.285 0.19 0.0898 0.0319 0.00887 … 0.000
3 0.0388 0.137 0.229 0.243 0.182 0.103 0.0454 … 0.000
4 0.0115 0.0576 0.137 0.205 0.218 0.175 0.109 … 0.000
5 0.00317 0.0211 0.0669 0.134 0.19 0.202 0.169 … 0.000
6 0.000798 0.00684 0.0278 0.0716 0.13 0.179 0.192 … 0.000
… … … … … … … … …
20 0 0 0 0 0 0 0 … 1
Example Two: Nucleotide evolution
A
G
C
T
Types of point mutation
A G
T C
Purine
Pyramidine Transitions
Transitions
Transversions
α
β β β β
α
A . α β β
α . β β
β β . α
β β α .
A G C T
T
C
G
Kimura’s 2 parameter model (K2P)
P =
jitt iXjX ,1 |Pr P
tX
1tX
GG TCAC A
G
C
T
A
G
C
T
CTATGA AG TTCGC
The Markov Property
GG TCAC A
G
C
T
A
G
C
T
CTATGA
The Markov Property
etc
XXXXX
XXXXX
XX
ttttt
ttttt
tt
,...,,,|Pr
,...,,,|Pr
|Pr
3211
3211
1
ATCAG
CTGAG
AG
tt
tttt
XxX
XXXXxX
|Pr
,...,,,|Pr
1
0211
Markov Chain properties
accessible aperiodic
communicate recurrent
irreducible transient
A . α β β
α . β β
β β . α
β β α .
A G C T
T
C
G
Accessible
P =
tX
1tX
0
0but 0 2,
1, ACAC PP
A . α β β
α . β β
β β . α
β β α .
A G C T
T
C
G
Accessible
P =
tX
1tX
0 0
0 0
A (and G) are no longer accessible from C (or T).
0, nACP
A . α β β
α . β β
β β . α
β β α .
A G C T
T
C
G
Accessible
P =
tX
1tX
0 0
0 0
But C (and T) are still accessible from A (or G).
0, nCAP
A . α β β
α . β β
β β . α
β β α .
A G C T
T
C
G
Communicate
P =
tX
1tX
Reciprocal accessibility
CA
A . α β β
α . β β
β β . α
β β α .
A G C T
T
C
G
Irreducible
P =
tX
1tX
All elements communicate
0, njiP jiji , allfor
Non-irreducible
A . α β β
α . β β
β β . α
β β α .
A G C T
T
C
GP =
tX
1tX
0 0
0 0
0 0
0 0 =P1
P2
0
0
. α
α .
P1 = A
G
A G
C
T
C T
P2 = . α
α .
• Reflexivity
• Symmetry
• Transitivity
Repercussions of communication
ii ,definitionBy
ijji then If
kikjji then and If
Periodicity
0 1 2 3 4 5 6
0 0 1 0 0 0 0 0
1 0 0 1 0 0 0 0
2 0 0 0 1 0 0 0
3 0 0 0 0 1 0 0
4 0 0 0 0 0 1 0
5 0 0 0 0 0 0 1
6 1 0 0 0 0 0 0
P =
tX 1tX
Periodicity
• The period d(i) of an element i is defined as the greatest common divisor of the numbers of the generations in which the element is visited.
• Most Markov Chains that we deal with do not exhibit periodicity.
• A Markov Chain is aperiodicif d(i) = 1 for all i.
Recurrence
1,
n
niiP
1,
n
niiP
recurrent11
,
n
niif
transient11
,
n
niif
More on Recurrence
•and i is recurrent then j is recurrent
• In a one-dimensional symmetric random walk the origin is recurrent
• In a two-dimensional symmetric random walk the origin is recurrent
• In a three-dimensional symmetric random walk the origin is transient
ji If
Markov Chain properties
accessible aperiodic
communicate recurrent
irreducible transient
Markov Chains
Examples
X1=1
X1 X2 X3
X4 X5
N
XN 1,Bin
N
XN 2,Bin
N
XN 3,Bin
N
XN 4,Bin
N
XN 5,Bin etc
pN ,Bin pN ,Bin pN ,Bin
pN ,Bin pN ,Bin
P =
0 1 2 3 4 5 6 … 20
0 p 0 p 1 p 2 p 3 p 4 p 5 p 6 … p 20
1 p 0 p 1 p 2 p 3 p 4 p 5 p 6 … p 20
2 p 0 p 1 p 2 p 3 p 4 p 5 p 6 … p 20
3 p 0 p 1 p 2 p 3 p 4 p 5 p 6 … p 20
4 p 0 p 1 p 2 p 3 p 4 p 5 p 6 … p 20
5 p 0 p 1 p 2 p 3 p 4 p 5 p 6 … p 20
6 p 0 p 1 p 2 p 3 p 4 p 5 p 6 … p 20
… … … … … … … … … …
20 p 0 p 1 p 2 p 3 p 4 p 5 p 6 … p 20
jittt jXiXjX ,11 Pr|Pr P
Diffusion across a permeable membrane (1D random walk)
otherwise0
1 if2
1 if2
,ij
a
ia
ija
ia
jiP
aX t ,...,2,1,0
Brownian motion (2D random walk)
otherwise0
1 if2
1
1 if2
1
,,ij
ij
jiji YX
max
max
,...,2,1,0
,...,2,1,0
YY
XX
t
t
jijiji ,,, YXP
Wright-Fisher allele frequency model
X1=1
Haldane (1927) branching process model of fixation probability
,...2,1,0tX
tX 2 3 4 4 4 4 2
Haldane (1927) branching process model of fixation probability
mXT
m
XXXX
YXYX
x
xX
sGsG
XXXT
sGsGsG
sGsGsG
sssxXsG
)()(
...
)()()(
)()()(
...)2Pr()1Pr()0Pr()Pr()(
21
0
2
21
Haldane (1927) branching process model of fixation probability
xtt
x
tttt
tttt
XXX
sXxX
sXXsXX
sXXXX
sGsG t
tt
)|Pr(
...)|3Pr()|2Pr(
)|1Pr()|0Pr(
)()(
10
31
21
11
1|1
Pi,j = coefficient of sj in the above generating function
Haldane (1927) branching process model of fixation probability
Probability of fixation = 2s
Markov Chain properties
accessible aperiodic
communicate recurrent
irreducible transient