Advanced Statistical Computing Week 5: EM Algorithmavdvaart/ASC/EM.pdf · 2012. 10. 1. · Week 5:...

of 23/23
Advanced Statistical Computing Week 5: EM Algorithm Aad van der Vaart Fall 2012
  • date post

    20-Jan-2021
  • Category

    Documents

  • view

    1
  • download

    0

Embed Size (px)

Transcript of Advanced Statistical Computing Week 5: EM Algorithmavdvaart/ASC/EM.pdf · 2012. 10. 1. · Week 5:...

  • Advanced Statistical ComputingWeek 5: EM Algorithm

    Aad van der Vaart

    Fall 2012

  • Contents

    2

    EM Algorithm

    Mixtures

    Hidden Markov models

  • EM Algorithm

  • EM-algorithm

    4

    SETTING:Observation X, likelihood θ 7→ pθ(X), hard to maximize and find MLE θ̂).

    X can be viewed as 1st coordinate of (X,Y ) with density (x, y) 7→ p̄θ(x, y):

    pθ(x) =

    p̄θ(x, y) dµ(y).

    EM-ALGORITHM: GIVEN θ̃0 REPEAT• E-step: compute θ 7→ Eθ̃i

    (

    log p̄θ(X,Y )|X)

    .• M-step: θ̃i+1 =: point of maximum of this function.

    θ̃0, θ̃1, . . . often tends to MLE, but may not converge, converge slowly,or converge to local maximum.

    [ Y may be missing data, of augmented data, invented for convenience.]

  • EM-Algorithm — increases target

    5

    LEMMA θ̃0, θ̃1, . . . generated by EM-algorithm satisfies pθ̃0(X) ≤ pθ̃1(X) ≤ · · · .

    PROOFp̄θ(x, y) = pθ(y|x)pθ(x).

    Eθ̃i(

    log p̄θ(X,Y )|X)

    = Eθ̃i(

    log pθ(Y |X)|X)

    + log pθ(X).

    Because θ̃i+1 maximizes left side over θ, it suffices to show

    Eθ̃i(

    log pθ̃i+1(Y |X)|X)

    ≤ Eθ̃i(

    log pθ̃i(Y |X)|X)

    .

    Or −K(p, q): = Ep log(q/p)(Y ) ≤ 0 for p = pθ̃i , q = pθ̃i+1 , conditioned on X.

    Now Kullback-Leibler divergence K(p; q) is nonnegative for any p, q.

    This does not prove that θ̃i converges to the MLE!

  • EM-Algorithm — linear convergence

    6

    The speed of the EM-algorithm is linear, with slow convergence if the augmentedmodel is statistically much more informative than the data model.

  • Mixtures

  • Mixtures

    8

    SETTINGObservations random sample X1, . . . , Xn from density

    pθ(x) =k∑

    j=1

    pjf(x; ηj), θ = (p1, . . . , pk, η1, . . . , ηk).

    AUGMENTED DATA

    P(Yi = j) = pj , Xi|Yi = j ∼ f(·; ηj), i = 1, . . . , n.

    Full likelihood

    pθ(X1, . . . , Xn, Y1, . . . , Yn) =n∏

    i=1

    k∏

    j=1

    (

    pjf(Xi; ηj))1{Yi=j} .

  • Mixtures — E-step, M-step

    9

    E-step: given (p̃, ẽta):

    Ep̃,η̃

    (

    logn∏

    i=1

    k∏

    j=1

    (

    pjf(Xi, ηj))1{Yi=j} |X1, . . . , Xn

    )

    =n∑

    i=1

    k∑

    j=1

    log(

    pjf(Xi, ηj))

    α̃i,j ,α̃i,j : = Pp̃,η̃

    (

    Yi = j|Xi)

    =p̃jf(Xi, η̃j)

    c p̃cf(Xi, η̃c)

    =[

    k∑

    j=1

    log pj

    (

    n∑

    i=1

    α̃i,j

    )]

    +k∑

    j=1

    [

    n∑

    i=1

    log f(Xi; ηj)α̃i,j

    ]

    .

    M-step: for j = 1, . . . , k:

    pnewj =1

    n

    n∑

    i=1

    α̃i,j , ηnewj = argmax

    η

    n∑

    i=1

    log f(Xi; η)α̃i,j .

    [ If the f(·; η) have a common parameter, then the computation of the ηj does not separate as theydo here.]

  • Mixtures — Example

    10

    EXAMPLEIf f(·; η) ∼ N(η, 1), then

    n∑

    i=1

    log f(Xi, η)α̃i,j = −1

    2

    ∑ni=1α̃i,j(Xi − η)

    2 + Const.

    ηnewj =

    ∑n

    i=1α̃ijXi∑n

    i=1α̃i,j.

    EXAMPLEIf f(·; η) ∼ Γ(r, η), then

    n∑

    i=1

    log f(Xi, η)α̃i,j =n∑

    i=1

    (r log η − ηXi)α̃i,j + Const.

    ηnewj =r∑n

    i=1α̃i,j∑n

    i=1α̃i,jXi.

  • R

    11

    0 5 10 150.

    000.

    100.

    200.

    30

    > n=100> shape=c(2,2,2); eta=c(1,6,.2); prob=c(1/4,1/8,5/8)> component=sample(c(1,2,3),n,replace=TRUE,prob=prob)> x=rgamma(n,shape=shape[component],rate=eta[component])

  • R — EM, known shape

    12

    > k=3; a=matrix(0,n,k); p=c(1/3,1/3,1/3); eta=c(1,2,3); change=1> while (change>0.0001){+ for (j in 1:k) a[,j]=p[j]*dgamma(x,2,eta[j])+ a=diag(1/apply(a,1,sum))%*%a+ etanew=2*apply(a,2,sum)/matrix(x,1,n)%*%a+ pnew=apply(a,2,mean)+ change=sum(abs(etanew-eta)+abs(pnew-p))+ print(rbind(pnew,etanew))+ eta=etanew; p=pnew}[ --- output deleted ---- ]

    [,1] [,2] [,3]pnew 0.6259239 0.3161804 0.05789564

    0.2157931 1.7430514 7.57683781

    0 5 10 15

    0.00

    0.10

    0.20

    0.30

  • R — packages

    13

    > library(mixtools)> mod=gammamixEM(x,k=3)number of iterations= 323> summary(mod)Error in summary.mixEM(mod) : Unknown mixEM object of type gammamixEM> mod[[2]]; mod[[3]][1] 0.37441469 0.57523322 0.05035209

    comp.1 comp.2 comp.3alpha 1.6203475 2.092346 20.9880430beta 0.6184701 4.126267 0.7926715

    0 5 10 15

    0.00

    0.10

    0.20

    0.30

    [ Besides package mixtools, there is also flexmix, and ... (?)]

  • Mixtures — warnings

    14

    Not all mixtures are identifiable from the data: multiple parameter vectors maygive the same mixture.

    Maximum likelihood may work only if the parameter set is restricted. (Notableexample: location scale mixtures, if the scale parameter approaches zero, thelikelihood may tend to infinity.)

    EM tends to be slow for large data sets, and might get stuck in local maxima (?)

  • Hidden Markov models

  • Hidden Markov model

    16

    Y1 Y2 Y3

    . . .

    Yn−1 Yn

    X1 X2 X3

    . . .

    Xn−1 Xn

    Markov chain of hidden states Y1, Y2, . . . ,; only outputs X1, X2, . . . observed.

    Xi given Yi conditionally independent of all other variables.

    EXAMPLES• speech recognition: states abstract, outputs Fourier coding of sounds.• genomics: states are introns/exons, outputs nucleotides• genomics: states are # chromosomal duplicates, outputs noisy

    measurements• genetics: states inheritance vectors, output measured markers.• cell biology: states of ion channels, outputs current or no current• economics: state of economy, output # firms in default.

  • Hidden Markov model

    17

    Y1 Y2 Y3

    . . .

    Yn−1 Yn

    X1 X2 X3

    . . .

    Xn−1 Xn

    Markov chain of hidden states Y1, Y2, . . . ,; only outputs X1, X2, . . . observed.

    Xi given Yi conditionally independent of all other variables.

    Parameters• density π of Y1• transition density p(yi| yi−1) of the Markov chain.• output density q(xi| yi).

    Full likelihood

    π(y1)p(y2| y1)× · · · × p(yn| yn−1) q(x1| y1)× · · · × q(xn| yn).

  • HMM — E and M-step

    18

    E-step:

    Eπ̃,p̃,q̃

    (

    log π(Y1)

    n∏

    i=2

    p(Yi|Yi−1)

    n∏

    i=1

    q(Xi|Yi)|X1, . . . , Xn

    )

    = Eπ̃,p̃,q̃

    (

    log π(Y1)|X1, . . . , Xn

    )

    +n∑

    i=2

    Eπ̃,p̃,q̃

    (

    log p(Yi|Yi−1)|X1, . . . , Xn

    )

    +

    n∑

    i=1

    Eπ̃,p̃,q̃

    (

    log q(Xi|Yi)|X1, . . . , Xn

    )

    .

    M-step:• depends on the specification of models for π, p, q.• if state space is finite p is typically left free.• only current estimate of law of (Yi−1, Yi) given X1, . . . , Xn needed, which

    are computed using the forward and backward algorithm.

  • Baum-Welch

    19

    The EM-algorithm for the HMM with finite state space, and completelyunspecified distributions π, p, q, is called Baum-Welch algorithm.

    If π and p are left free:πnew = p

    Y1|X1,...,Xnπ̃,p̃,q̃ (y).

    pnew(v|u) =

    ∑ni=2 p

    Yi−1,Yi|X1,...,Xnπ̃,p̃,q̃ (u, v)

    ∑ni=2 p

    Yi−1|X1,...,Xnπ̃,p̃,q̃ (u)

    .

    If q is also left free (possible for finite output space, but not often the case):

    qnew(x| y) =

    i:Xi=xpYi|X1,...,Xi−1,Xi=x,Xi+1,...,Xnπ̃,p̃,q̃ (y)

    x∈X

    i:Xi=xpYi|X1,...,Xi−1,Xi=x,Xi+1,...,Xnπ̃,p̃,q̃ (y)

    .

    [ To compute these expressions need density of (Yi−1, Yi) given X1, . . . , Xn. This is computedusing the forward and backward algorithm.]

  • Viterbi

    20

    Y1 Y2 Y3

    . . .

    Yn−1 Yn

    X1 X2 X3

    . . .

    Xn−1 Xn

    The Viterbi algorithm computes the most likely state path given the outcomes:

    argmaxy1,...,yn

    P(Y1 = y1, . . . , Yn = yn|X1, . . . , Xn).

  • R

    21

    0 20 40 60 80 1000

    12

    34

    5

    > library(HiddenMarkov)> Pi=matrix(c(0.7,0.3,0.2,0.8),2,2,byrow=TRUE); delta=c(0.3,0.7)> n=100; pn=list(size=rep(5,n)); pm=list(prob=c(0.3,0.8))> myhmm=dthmm(NULL,Pi=Pi,delta=delta,distn="binom",pn=pn,pm=pm)> x=simulate(myhmm,n)>> plot(1:n,x$x,type="s",xlab="",ylab="")> lines(1:n,x$y-1,col=2,type="s")

    [ Markov chain with two states, transition matrix Π, initial distribution δ. Outputs are from thebinomial(5, p)- distribution, with θ = 0.3 from state 1 and θ = 0.8 from state 2. Red: states, Black:outputs.]

  • R

    22

    0 20 40 60 80 1000

    12

    34

    5

    > mod=BaumWelch(x); mod$Pi; mod$pm[---- output deleted ---]

    [,1] [,2][1,] 0.6287149 0.3712851[2,] 0.2637289 0.7362711$prob[1] 0.3173456 0.8313127

    [ Markov chain with two states, transition matrix Π =0.7 0.30.2 0.8

    , initial distribution δ = (0.3, 0.7).

    Outputs are from the binomial(5, p)- distribution, with θ = 0.3 from state 1 and θ = 0.8 from state 2.]

  • R

    23

    0 20 40 60 80 1000

    12

    34

    5

    > Viterbi(x)[1] 2 2 2 2 2 2 2 1 1 2 2 2 2 2 1 1 1 1 1 2 2 2 2 2 2 2 1 1 1[36] 2 2 2 2 2 2 2 2 2 1 1 1 2 1 1 2 2 2 2 2 2 2 1 1 1 1 2 2 1[71] 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 1 1

    > lines(1:n,Viterbi(x)-1,col=3,lw=2)

    [ Red: true states, Black: outputs; Green: reconstructed states.]

    EM AlgorithmEM-algorithmEM-Algorithm — increases targetEM-Algorithm — linear convergence

    MixturesMixturesMixtures — E-step, M-stepMixtures — ExampleRR — EM, known shapeR — packagesMixtures — warnings

    Hidden Markov modelsHidden Markov modelHidden Markov modelHMM — E and M-stepBaum-WelchViterbiRRR