Advanced Statistical ComputingWeek 5: EM Algorithm

Aad van der Vaart

Fall 2012

Contents

2

EM Algorithm

Mixtures

Hidden Markov models

EM Algorithm

EM-algorithm

4

SETTING:Observation X, likelihood θ 7→ pθ(X), hard to maximize and find MLE θ).

X can be viewed as 1st coordinate of (X,Y ) with density (x, y) 7→ pθ(x, y):

pθ(x) =

∫

pθ(x, y) dµ(y).

EM-ALGORITHM: GIVEN θ0 REPEAT• E-step: compute θ 7→ Eθi

(

log pθ(X,Y )|X)

.• M-step: θi+1 =: point of maximum of this function.

θ0, θ1, . . . often tends to MLE, but may not converge, converge slowly,or converge to local maximum.

[ Y may be missing data, of augmented data, invented for convenience.]

EM-Algorithm — increases target

5

LEMMA θ0, θ1, . . . generated by EM-algorithm satisfies pθ0(X) ≤ pθ1(X) ≤ · · · .

PROOFpθ(x, y) = pθ(y|x)pθ(x).

Eθi

(

log pθ(X,Y )|X)

= Eθi

(

log pθ(Y |X)|X)

+ log pθ(X).

Because θi+1 maximizes left side over θ, it suffices to show

Eθi

(

log pθi+1(Y |X)|X

)

≤ Eθi

(

log pθi(Y |X)|X)

.

Or −K(p, q): = Ep log(q/p)(Y ) ≤ 0 for p = pθi , q = pθi+1, conditioned on X.

Now Kullback-Leibler divergence K(p; q) is nonnegative for any p, q.

This does not prove that θi converges to the MLE!

EM-Algorithm — linear convergence

6

The speed of the EM-algorithm is linear, with slow convergence if the augmentedmodel is statistically much more informative than the data model.

Mixtures

Mixtures

8

SETTINGObservations random sample X1, . . . , Xn from density

pθ(x) =k∑

j=1

pjf(x; ηj), θ = (p1, . . . , pk, η1, . . . , ηk).

AUGMENTED DATA

P(Yi = j) = pj , Xi|Yi = j ∼ f(·; ηj), i = 1, . . . , n.

Full likelihood

pθ(X1, . . . , Xn, Y1, . . . , Yn) =n∏

i=1

k∏

j=1

(

pjf(Xi; ηj))1{Yi=j} .

Mixtures — E-step, M-step

9

E-step: given (p, eta):

Ep,η

(

logn∏

i=1

k∏

j=1

(

pjf(Xi, ηj))1{Yi=j} |X1, . . . , Xn

)

=n∑

i=1

k∑

j=1

log(

pjf(Xi, ηj))

αi,j ,αi,j : = Pp,η

(

Yi = j|Xi

)

=pjf(Xi, ηj)

∑

c pcf(Xi, ηc)

=[

k∑

j=1

log pj

(

n∑

i=1

αi,j

)]

+k∑

j=1

[

n∑

i=1

log f(Xi; ηj)αi,j

]

.

M-step: for j = 1, . . . , k:

pnewj =1

n

n∑

i=1

αi,j , ηnewj = argmaxη

n∑

i=1

log f(Xi; η)αi,j .

[ If the f(·; η) have a common parameter, then the computation of the ηj does not separate as theydo here.]

Mixtures — Example

10

EXAMPLEIf f(·; η) ∼ N(η, 1), then

n∑

i=1

log f(Xi, η)αi,j = − 1

2

∑ni=1

αi,j(Xi − η)2 + Const.

ηnewj =

∑n

i=1αijXi

∑ni=1

αi,j

.

EXAMPLEIf f(·; η) ∼ Γ(r, η), then

n∑

i=1

log f(Xi, η)αi,j =n∑

i=1

(r log η − ηXi)αi,j + Const.

ηnewj =r∑n

i=1αi,j

∑ni=1

αi,jXi

.

R

11

0 5 10 150.

000.

100.

200.

30

> n=100> shape=c(2,2,2); eta=c(1,6,.2); prob=c(1/4,1/8,5/8)> component=sample(c(1,2,3),n,replace=TRUE,prob=prob)> x=rgamma(n,shape=shape[component],rate=eta[component])

R — EM, known shape

12

> k=3; a=matrix(0,n,k); p=c(1/3,1/3,1/3); eta=c(1,2,3); change=1> while (change>0.0001){+ for (j in 1:k) a[,j]=p[j]*dgamma(x,2,eta[j])+ a=diag(1/apply(a,1,sum))%*%a+ etanew=2*apply(a,2,sum)/matrix(x,1,n)%*%a+ pnew=apply(a,2,mean)+ change=sum(abs(etanew-eta)+abs(pnew-p))+ print(rbind(pnew,etanew))+ eta=etanew; p=pnew}[ --- output deleted ---- ]

[,1] [,2] [,3]pnew 0.6259239 0.3161804 0.05789564

0.2157931 1.7430514 7.57683781

0 5 10 15

0.00

0.10

0.20

0.30

R — packages

13

> library(mixtools)> mod=gammamixEM(x,k=3)number of iterations= 323> summary(mod)Error in summary.mixEM(mod) : Unknown mixEM object of type gammamixEM> mod[[2]]; mod[[3]][1] 0.37441469 0.57523322 0.05035209

comp.1 comp.2 comp.3alpha 1.6203475 2.092346 20.9880430beta 0.6184701 4.126267 0.7926715

0 5 10 15

0.00

0.10

0.20

0.30

[ Besides package mixtools, there is also flexmix, and ... (?)]

Mixtures — warnings

14

Not all mixtures are identifiable from the data: multiple parameter vectors maygive the same mixture.

Maximum likelihood may work only if the parameter set is restricted. (Notableexample: location scale mixtures, if the scale parameter approaches zero, thelikelihood may tend to infinity.)

EM tends to be slow for large data sets, and might get stuck in local maxima (?)

Hidden Markov models

Hidden Markov model

16

Y1 Y2 Y3

. . .

Yn−1 Yn

X1 X2 X3

. . .

Xn−1 Xn

Markov chain of hidden states Y1, Y2, . . . ,; only outputs X1, X2, . . . observed.

Xi given Yi conditionally independent of all other variables.

EXAMPLES• speech recognition: states abstract, outputs Fourier coding of sounds.• genomics: states are introns/exons, outputs nucleotides• genomics: states are # chromosomal duplicates, outputs noisy

measurements• genetics: states inheritance vectors, output measured markers.• cell biology: states of ion channels, outputs current or no current• economics: state of economy, output # firms in default.

Hidden Markov model

17

Y1 Y2 Y3

. . .

Yn−1 Yn

X1 X2 X3

. . .

Xn−1 Xn

Markov chain of hidden states Y1, Y2, . . . ,; only outputs X1, X2, . . . observed.

Xi given Yi conditionally independent of all other variables.

Parameters• density π of Y1

• transition density p(yi| yi−1) of the Markov chain.• output density q(xi| yi).

Full likelihood

π(y1)p(y2| y1)× · · · × p(yn| yn−1) q(x1| y1)× · · · × q(xn| yn).

HMM — E and M-step

18

E-step:

Eπ,p,q

(

log π(Y1)

n∏

i=2

p(Yi|Yi−1)

n∏

i=1

q(Xi|Yi)|X1, . . . , Xn

)

= Eπ,p,q

(

log π(Y1)|X1, . . . , Xn

)

+n∑

i=2

Eπ,p,q

(

log p(Yi|Yi−1)|X1, . . . , Xn

)

+

n∑

i=1

Eπ,p,q

(

log q(Xi|Yi)|X1, . . . , Xn

)

.

M-step:• depends on the specification of models for π, p, q.• if state space is finite p is typically left free.• only current estimate of law of (Yi−1, Yi) given X1, . . . , Xn needed, which

are computed using the forward and backward algorithm.

Baum-Welch

19

The EM-algorithm for the HMM with finite state space, and completelyunspecified distributions π, p, q, is called Baum-Welch algorithm.

If π and p are left free:πnew = p

Y1|X1,...,Xn

π,p,q (y).

pnew(v|u) =

∑ni=2

pYi−1,Yi|X1,...,Xn

π,p,q (u, v)∑n

i=2pYi−1|X1,...,Xn

π,p,q (u).

If q is also left free (possible for finite output space, but not often the case):

qnew(x| y) =

∑

i:Xi=x pYi|X1,...,Xi−1,Xi=x,Xi+1,...,Xn

π,p,q (y)∑

x∈X

∑

i:Xi=x pYi|X1,...,Xi−1,Xi=x,Xi+1,...,Xn

π,p,q (y).

[ To compute these expressions need density of (Yi−1, Yi) given X1, . . . , Xn. This is computedusing the forward and backward algorithm.]

Viterbi

20

Y1 Y2 Y3

. . .

Yn−1 Yn

X1 X2 X3

. . .

Xn−1 Xn

The Viterbi algorithm computes the most likely state path given the outcomes:

argmaxy1,...,yn

P(Y1 = y1, . . . , Yn = yn|X1, . . . , Xn).

R

21

0 20 40 60 80 1000

12

34

5

> library(HiddenMarkov)> Pi=matrix(c(0.7,0.3,0.2,0.8),2,2,byrow=TRUE); delta=c(0.3,0.7)> n=100; pn=list(size=rep(5,n)); pm=list(prob=c(0.3,0.8))> myhmm=dthmm(NULL,Pi=Pi,delta=delta,distn="binom",pn=pn,pm=pm)> x=simulate(myhmm,n)>> plot(1:n,x$x,type="s",xlab="",ylab="")> lines(1:n,x$y-1,col=2,type="s")

[ Markov chain with two states, transition matrix Π, initial distribution δ. Outputs are from thebinomial(5, p)- distribution, with θ = 0.3 from state 1 and θ = 0.8 from state 2. Red: states, Black:outputs.]

R

22

0 20 40 60 80 1000

12

34

5

> mod=BaumWelch(x); mod$Pi; mod$pm[---- output deleted ---]

[,1] [,2][1,] 0.6287149 0.3712851[2,] 0.2637289 0.7362711$prob[1] 0.3173456 0.8313127

[ Markov chain with two states, transition matrix Π =0.7 0.30.2 0.8

, initial distribution δ = (0.3, 0.7).

Outputs are from the binomial(5, p)- distribution, with θ = 0.3 from state 1 and θ = 0.8 from state 2.]

R

23

0 20 40 60 80 1000

12

34

5

> Viterbi(x)[1] 2 2 2 2 2 2 2 1 1 2 2 2 2 2 1 1 1 1 1 2 2 2 2 2 2 2 1 1 1[36] 2 2 2 2 2 2 2 2 2 1 1 1 2 1 1 2 2 2 2 2 2 2 1 1 1 1 2 2 1[71] 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 1 1

> lines(1:n,Viterbi(x)-1,col=3,lw=2)

[ Red: true states, Black: outputs; Green: reconstructed states.]