STK4021 - Applied Bayesian Analysis and Numerical Methods€¦ · STK9021 - Applied Bayesian...
Transcript of STK4021 - Applied Bayesian Analysis and Numerical Methods€¦ · STK9021 - Applied Bayesian...
-
STK9021 - Applied Bayesian Analysis and Numerical MethodsCHAPTER 3: EXERCISE 1 AND 2
MOHAMMED AHMED KEDIR
-
3.10 Exercises
-
Solution: (1-A)
y
θ
1..J Multinomial distribution parameters
1,....... |Jy y Data which follows a multinomial distributions
Prior distributions on multinomial distribution (Dirichlet)
• Multinomial distribution: is a convenient way to represent discrete
variable that can take on of possible mutually exclusive states. e.g. 1
of K scheme T(0,0,0,1,0,0)y
1
1
| ( | ) jJ
j
j
Dir
We are interested in marginal posterior distribution for 1
1 2
1
| jJ
y
j
j
p
y θ
β
-
Cont’
Posterior distribution: | ( | ) ( )p p pθ y y θ θ1
1 1
j j
J Jy
j j
j j
1
1
j j
Jy
j
j
Which is Dirichlet 𝐷𝑖𝑟(𝜽|𝑦𝑗 + 𝛽𝑗
The joint probability density can be written as
1 2, 1 1 2 2 1 3 3 1 2 1 1 1 2 2( , ) | | , | , , ,m m m mf f f f f
e.g. For three variables, 1 2 3, ,
31 211 1
1 2 1 2 1 2 1 2, ,1 1p
-
Cont’: Marginal distributions of
Marginal distribution for the earlier three variables
31 211 1
1 2 1 2 1 2 1 2, ,1 1p
1
3 31 2 1 2
1 1 2 2
11 11 1 1 1
1 1 2 1 2 2 1 2 1 2 2
0
,
1 1
p p d
p d d
Let 2 11 u
321 2
3 32 21 2 1 2
2 3 31 2
111 1
1 1 1 1
0
1 11 11 1 1 1
1 1 1 1 1 1 1
0 0
11 11 1
1 1
0
1 1 2 3
1 1 1
1 1 1 1 1 1
1 1
,
u u du
u u du u u du
u u du
p Beta
32
111
2 3
0
1 ,u u du Beta
But
The marginal distribution of single is Beta distribution 𝜃j
𝜃j
-
Cont’ Similarly it can be shown that the marginal distribution of a subvector of θ is Dirichlet.
The marginal posterior distribution for three variables.
Let find the posterior marginal distribution of 𝜃others ,
The posterior marginal distribution will be Dirichlet.
Hence,
But we are interested in
𝜃1 , 𝜃2 , 1 21others
1 1 2 211 1
1 2 1 2 1 2, | 1others othersyy yp y
1
1
j j
Jy
j
j
11 2
1 2
, |p y
By change of variables: The transformation is one-to-one because we can solve
for 𝜃1 , 𝜃2 ,in terms of by 𝛼, 𝜏 𝜃1 = 𝛼𝜏
𝜃2 = 𝛼 − 1 𝜏
-
Cont.’
11 2
1 2
, |p y
1
2 1
1
2
1 2
1
, |, |
det(J)
p yp y
2 21 1
1 1 2 2 1 2 1 2
1 11
1 1 11
1 1 2 2 1 2 1 2
1 1 1
1 1
, | | , | ,
others others
others others
y yy
y y yy y
others othersp y Beta y y Beta y y y
Hence, 1 1 2 2| | ,p y Beta y y
2 1
2 21 2 2 11 2 1 2 2 2
1 2 1 21 2
, 1 1 1
,1 1
The Jacobean
1 1 2 2 1 2 1 2
1 1 2 2 1 2 1 2
| | , | ,
| | , | ,
others others
others others
p y Beta y y Beta y y y d
p y Beta y y Beta y y y d
-
(1-B)
Data which follows a binomial distributions
Binomial distribution parameters
Beta Prior
1 2, |y y
1 2| ,
From earlier class on conjugacy we know that the posterior distribution will be Beta distribution
2 2 2 21 1 1 11 11 1
1 2 1 1 2 2| , 1 1 1 ( | , )y yy y
p y y Beta y y
Which is A Beta distribution similar to the result 1-A
-
Exercise 2:
-
Soln:ABC news survey of registered voters before and after debate
1
1
| ( | ) ( )
j
j
Jy
j
j
Jy
j
j
p p p
θ y y θ θ
Which is a multinomial distribution
1 1
1 1
j j
J Jy y
j j
j j
Which is a Dirichlet distribution with 𝐷𝑖𝑟(𝜽|𝑦𝑗 + 1
Since they are independent:
Pre-debate:
Post-debate:
1 2 3, , | 295,308,39pre pre prep y Direchlet
1 2 3, , | 289,333,20post post postp y Direchlet
-
Cont’ For j=1,2, let be the proportion of voters who prefer Bush out of Total of who preferred Bush and Dukakis
j
People who prefer bush Pre debate
112
1 2 2
post debate preference Bush,
post debate preference Dukakis
postpost
post post post
People who prefer bush Post debate
111
1 2 2
pre debate preference Bush,
pre debate preference Dukakis
prepre
pre pre pre
But we are interested in 2 1z
From the earlier exercise we know that 1 1 2 2| | ,j jp y Beta y y
1 1
2 2
| | 295,308
| | 289,333
pre
post
p y Beta
p y Beta
1
2
1 2
2 1 2
, |(z ( ), w )
det(J) z ww
p yp
Analytic solution:
1
2
1294 307 3321 2 288
0
, |(z) 1 1
det(J) z ww
p yp dw z w z w w w dw
-
Numerical Solution: with R
Step 1: Simulate N number of samples from
Step2 : Simulate N number of samples from
Step 3: Obtain the posterior difference by
Step 4: Find the mean of the difference greater than zero
1 | 295,308prep y Beta
2 | 289,333postp y Beta
2 1 |p y 1 2| |post prep y p y
alpha1
-
Result
[1] "posterior probability that there was a shift toward Bush is 0.1982000000"