A Note on ZINB-VAE
2
A Note on ZINB-VAE[1] Tomonari MASADA @ Nagasaki University September 8, 2017 p(x n,g ,h n,g ,y n,g ,w n,g , z n ; W h , W w ) = p(x n,g |h n,g ,y n,g )p(h n,g |z n ; W h )p(y n,g |w n,g )p(w n,g |z n ; W w )p(z n ) = δ 0 (x n,g ) hn,g δ yn,g (x n,g ) 1-hn,g × f h (z n ) hn,g (1 - f h (z n )) 1-hn,g × w yn,g n,g e -wn,g y n,g ! × f w,2 (z n ) fw,1(zn) Γ(f w,1 (z n )) w fw,1(zn)-1 n,g e -fw,2(zn)wn,g × K Y k=1 1 √ 2π exp z 2 n,k 2 (1) p(x n,g ,y n,g ,w n,g , z n ; W h , W w ) = p(x n,g ,h n,g =0,y n,g ,w n,g , z n ; W h , W w )+ p(x n,g ,h n,g =1,y n,g ,w n,g , z n ; W h , W w ) = {δ yn,g (x n,g )(1 - f h (z n )) + δ 0 (x n,g )f h (z n )}× w yn,g n,g e -wn,g y n,g ! × f w,2 (z n ) fw,1(zn) Γ(f w,1 (z n )) w fw,1(zn)-1 n,g e -fw,2(zn)wn,g × K Y k=1 1 √ 2π exp - z 2 n,k 2 (2) p(x n,g ,w n,g , z n ; W h , W w )= p(x n,g |w n,g , z n ; W h )p(w n,g |z n ; W w )p(z n ) = Z p(x n,g ,y n,g ,w n,g , z n ; W h , W w )dy n,g = n (1 - f h (z n )) × w xn,g n,g e -wn,g x n,g ! + δ 0 (x n,g )f h (z n ) o × f w,2 (z n ) fw,1(zn) Γ(f w,1 (z n )) w fw,1(zn)-1 n,g e -fw,2(zn)wn,g × K Y k=1 1 √ 2π exp - z 2 n,k 2 = n (1 - f h (z n )) × w xn,g n,g e -wn,g x n,g ! + δ 0 (x n,g )f h (z n ) o × f w,2 (z n ) fw,1(zn) Γ(f w,1 (z n )) w fw,1(zn)-1 n,g e -fw,2(zn)wn,g × K Y k=1 1 √ 2π exp - z 2 n,k 2 (3) p(x n,g |z n ; W h , W w )= Z p(x n,g |w n,g , z n ; W h )p(w n,g |z n ; W w )dw n,g = Z n (1 - f h (z n )) × w xn,g n,g e -wn,g x n,g ! + δ 0 (x n,g )f h (z n ) o f w,2 (z n ) fw,1(zn) Γ(f w,1 (z n )) w fw,1(zn)-1 n,g e -fw,2(zn)wn,g dw n,g = (1 - f h (z n )) 1 x n,g ! f w,2 (z n ) fw,1(zn) Γ(f w,1 (z n )) Z w {xn,g+fw,1(zn)}-1 n,g e -{1+fw,2(zn)}wn,g dw n,g + δ 0 (x n,g )f h (z n ) = (1 - f h (z n )) 1 x n,g ! f w,2 (z n ) fw,1(zn) Γ(f w,1 (z n )) Γ(x n,g + f w,1 (z n )) {1+ f w,2 (z n )} xn,g+fw,1(zn) + δ 0 (x n,g )f h (z n ) (4) 1
-
Upload
tomonari-masada -
Category
Engineering
-
view
76 -
download
2
Transcript of A Note on ZINB-VAE
A Note on ZINB-VAE[1]
Tomonari MASADA @ Nagasaki University
September 8, 2017
p(xn,g, hn,g, yn,g, wn,g, zn;W h,Ww)
= p(xn,g|hn,g, yn,g)p(hn,g|zn;W h)p(yn,g|wn,g)p(wn,g|zn;Ww)p(zn)
= δ0(xn,g)hn,gδyn,g (xn,g)1−hn,g × fh(zn)hn,g (1− fh(zn))1−hn,g × wyn,gn,g e−wn,g
yn,g!
× fw,2(zn)fw,1(zn)
Γ(fw,1(zn))wfw,1(zn)−1
n,g e−fw,2(zn)wn,g ×K∏
k=1
1√2π
expz2n,k
2(1)
p(xn,g, yn,g, wn,g, zn;W h,Ww)
= p(xn,g, hn,g = 0, yn,g, wn,g, zn;W h,Ww) + p(xn,g, hn,g = 1, yn,g, wn,g, zn;W h,Ww)
= {δyn,g (xn,g)(1− fh(zn)) + δ0(xn,g)fh(zn)} × wyn,gn,g e−wn,g
yn,g!
× fw,2(zn)fw,1(zn)
Γ(fw,1(zn))wfw,1(zn)−1
n,g e−fw,2(zn)wn,g ×K∏
k=1
1√2π
exp
(−z2n,k
2
)(2)
p(xn,g, wn,g, zn;W h,Ww) = p(xn,g|wn,g, zn;W h)p(wn,g|zn;Ww)p(zn)
=
∫p(xn,g, yn,g, wn,g, zn;W h,Ww)dyn,g
={
(1− fh(zn))× wxn,gn,g e−wn,g
xn,g!+ δ0(xn,g)fh(zn)
}× fw,2(zn)
fw,1(zn)
Γ(fw,1(zn))wfw,1(zn)−1
n,g e−fw,2(zn)wn,g ×K∏
k=1
1√2π
exp
(−z2n,k
2
)={
(1− fh(zn))× wxn,gn,g e−wn,g
xn,g!+ δ0(xn,g)fh(zn)
}× fw,2(zn)
fw,1(zn)
Γ(fw,1(zn))wfw,1(zn)−1
n,g e−fw,2(zn)wn,g ×K∏
k=1
1√2π
exp
(−z2n,k
2
)(3)
p(xn,g|zn;W h,Ww) =
∫p(xn,g|wn,g, zn;W h)p(wn,g|zn;Ww)dwn,g
=
∫ {(1− fh(zn))× w
xn,gn,g e−wn,g
xn,g!+ δ0(xn,g)fh(zn)
}fw,2(zn)fw,1(zn)
Γ(fw,1(zn))wfw,1(zn)−1
n,g e−fw,2(zn)wn,gdwn,g
= (1− fh(zn))1
xn,g!
fw,2(zn)fw,1(zn)
Γ(fw,1(zn))
∫w{xn,g+fw,1(zn)}−1
n,g e−{1+fw,2(zn)}wn,gdwn,g
+ δ0(xn,g)fh(zn)
= (1− fh(zn))1
xn,g!
fw,2(zn)fw,1(zn)
Γ(fw,1(zn))
Γ(xn,g + fw,1(zn))
{1 + fw,2(zn)}xn,g+fw,1(zn)+ δ0(xn,g)fh(zn) (4)
1
The ELBO is obtained as follows.
log p(xn,g;W h,Ww) = log
∫p(xn,g|zn;W h,Ww)p(zn)dzn
≥∫q(zn|xn) log
p(xn,g|zn;W h,Ww)p(zn)
q(zn|xn)dzn (5)
We can perform a Monte Carlo approximation as below.∫q(zn|xn) log
p(xn,g|zn;W h,Ww)p(zn)
q(zn|xn)dzn
≈ 1
S
S∑s=1
log p(xn,g|z(s)n ;W h,Ww) +
∫q(zn|xn) log p(zn)dzn −
∫q(zn|xn) log q(zn|xn)dzn (6)
where z(s)n ≡ ε(s) � σn(xn) + µn(xn), and ε(s) ∼ N(0, I).
References
[1] R. Lopez, J. Regier, M. Jordan, and N. Yosef. A deep generative model for gene expression profilesfrom single-cell RNA sequencing. ArXiv e-prints, September 2017.
2
