STK4021 - Applied Bayesian Analysis and Numerical Methods€¦ · STK9021 - Applied Bayesian...

13
STK9021 - Applied Bayesian Analysis and Numerical Methods CHAPTER 3: EXERCISE 1 AND 2 MOHAMMED AHMED KEDIR

Transcript of STK4021 - Applied Bayesian Analysis and Numerical Methods€¦ · STK9021 - Applied Bayesian...

  • STK9021 - Applied Bayesian Analysis and Numerical MethodsCHAPTER 3: EXERCISE 1 AND 2

    MOHAMMED AHMED KEDIR

  • 3.10 Exercises

  • Solution: (1-A)

    y

    θ

    1..J Multinomial distribution parameters

    1,....... |Jy y Data which follows a multinomial distributions

    Prior distributions on multinomial distribution (Dirichlet)

    • Multinomial distribution: is a convenient way to represent discrete

    variable that can take on of possible mutually exclusive states. e.g. 1

    of K scheme T(0,0,0,1,0,0)y

    1

    1

    | ( | ) jJ

    j

    j

    Dir

    We are interested in marginal posterior distribution for 1

    1 2

    1

    | jJ

    y

    j

    j

    p

    y θ

    β

  • Cont’

    Posterior distribution: | ( | ) ( )p p pθ y y θ θ1

    1 1

    j j

    J Jy

    j j

    j j

    1

    1

    j j

    Jy

    j

    j

    Which is Dirichlet 𝐷𝑖𝑟(𝜽|𝑦𝑗 + 𝛽𝑗

    The joint probability density can be written as

    1 2, 1 1 2 2 1 3 3 1 2 1 1 1 2 2( , ) | | , | , , ,m m m mf f f f f

    e.g. For three variables, 1 2 3, ,

    31 211 1

    1 2 1 2 1 2 1 2, ,1 1p

  • Cont’: Marginal distributions of

    Marginal distribution for the earlier three variables

    31 211 1

    1 2 1 2 1 2 1 2, ,1 1p

    1

    3 31 2 1 2

    1 1 2 2

    11 11 1 1 1

    1 1 2 1 2 2 1 2 1 2 2

    0

    ,

    1 1

    p p d

    p d d

    Let 2 11 u

    321 2

    3 32 21 2 1 2

    2 3 31 2

    111 1

    1 1 1 1

    0

    1 11 11 1 1 1

    1 1 1 1 1 1 1

    0 0

    11 11 1

    1 1

    0

    1 1 2 3

    1 1 1

    1 1 1 1 1 1

    1 1

    ,

    u u du

    u u du u u du

    u u du

    p Beta

    32

    111

    2 3

    0

    1 ,u u du Beta

    But

    The marginal distribution of single is Beta distribution 𝜃j

    𝜃j

  • Cont’ Similarly it can be shown that the marginal distribution of a subvector of θ is Dirichlet.

    The marginal posterior distribution for three variables.

    Let find the posterior marginal distribution of 𝜃others ,

    The posterior marginal distribution will be Dirichlet.

    Hence,

    But we are interested in

    𝜃1 , 𝜃2 , 1 21others

    1 1 2 211 1

    1 2 1 2 1 2, | 1others othersyy yp y

    1

    1

    j j

    Jy

    j

    j

    11 2

    1 2

    , |p y

    By change of variables: The transformation is one-to-one because we can solve

    for 𝜃1 , 𝜃2 ,in terms of by 𝛼, 𝜏 𝜃1 = 𝛼𝜏

    𝜃2 = 𝛼 − 1 𝜏

  • Cont.’

    11 2

    1 2

    , |p y

    1

    2 1

    1

    2

    1 2

    1

    , |, |

    det(J)

    p yp y

    2 21 1

    1 1 2 2 1 2 1 2

    1 11

    1 1 11

    1 1 2 2 1 2 1 2

    1 1 1

    1 1

    , | | , | ,

    others others

    others others

    y yy

    y y yy y

    others othersp y Beta y y Beta y y y

    Hence, 1 1 2 2| | ,p y Beta y y

    2 1

    2 21 2 2 11 2 1 2 2 2

    1 2 1 21 2

    , 1 1 1

    ,1 1

    The Jacobean

    1 1 2 2 1 2 1 2

    1 1 2 2 1 2 1 2

    | | , | ,

    | | , | ,

    others others

    others others

    p y Beta y y Beta y y y d

    p y Beta y y Beta y y y d

  • (1-B)

    Data which follows a binomial distributions

    Binomial distribution parameters

    Beta Prior

    1 2, |y y

    1 2| ,

    From earlier class on conjugacy we know that the posterior distribution will be Beta distribution

    2 2 2 21 1 1 11 11 1

    1 2 1 1 2 2| , 1 1 1 ( | , )y yy y

    p y y Beta y y

    Which is A Beta distribution similar to the result 1-A

  • Exercise 2:

  • Soln:ABC news survey of registered voters before and after debate

    1

    1

    | ( | ) ( )

    j

    j

    Jy

    j

    j

    Jy

    j

    j

    p p p

    θ y y θ θ

    Which is a multinomial distribution

    1 1

    1 1

    j j

    J Jy y

    j j

    j j

    Which is a Dirichlet distribution with 𝐷𝑖𝑟(𝜽|𝑦𝑗 + 1

    Since they are independent:

    Pre-debate:

    Post-debate:

    1 2 3, , | 295,308,39pre pre prep y Direchlet

    1 2 3, , | 289,333,20post post postp y Direchlet

  • Cont’ For j=1,2, let be the proportion of voters who prefer Bush out of Total of who preferred Bush and Dukakis

    j

    People who prefer bush Pre debate

    112

    1 2 2

    post debate preference Bush,

    post debate preference Dukakis

    postpost

    post post post

    People who prefer bush Post debate

    111

    1 2 2

    pre debate preference Bush,

    pre debate preference Dukakis

    prepre

    pre pre pre

    But we are interested in 2 1z

    From the earlier exercise we know that 1 1 2 2| | ,j jp y Beta y y

    1 1

    2 2

    | | 295,308

    | | 289,333

    pre

    post

    p y Beta

    p y Beta

    1

    2

    1 2

    2 1 2

    , |(z ( ), w )

    det(J) z ww

    p yp

    Analytic solution:

    1

    2

    1294 307 3321 2 288

    0

    , |(z) 1 1

    det(J) z ww

    p yp dw z w z w w w dw

  • Numerical Solution: with R

    Step 1: Simulate N number of samples from

    Step2 : Simulate N number of samples from

    Step 3: Obtain the posterior difference by

    Step 4: Find the mean of the difference greater than zero

    1 | 295,308prep y Beta

    2 | 289,333postp y Beta

    2 1 |p y 1 2| |post prep y p y

    alpha1

  • Result

    [1] "posterior probability that there was a shift toward Bush is 0.1982000000"