STK4021 - Applied Bayesian Analysis and Numerical Methods STK9021 - Applied Bayesian Analysis and...

download STK4021 - Applied Bayesian Analysis and Numerical Methods STK9021 - Applied Bayesian Analysis and Numerical

of 13

  • date post

    19-Oct-2020
  • Category

    Documents

  • view

    0
  • download

    0

Embed Size (px)

Transcript of STK4021 - Applied Bayesian Analysis and Numerical Methods STK9021 - Applied Bayesian Analysis and...

  • STK9021 - Applied Bayesian Analysis and Numerical Methods CHAPTER 3: EXERCISE 1 AND 2

    MOHAMMED AHMED KEDIR

  • 3.10 Exercises

  • Solution: (1-A)

    y

    θ

    1..J Multinomial distribution parameters

    1,....... |Jy y  Data which follows a multinomial distributions

    Prior distributions on multinomial distribution (Dirichlet)

    • Multinomial distribution: is a convenient way to represent discrete

    variable that can take on of possible mutually exclusive states. e.g. 1

    of K scheme T(0,0,0,1,0,0)y 

    1

    1

    | ( | ) j J

    j

    j

    Dir 

         

    

    We are interested in marginal posterior distribution for 1

    1 2

     

      

      1

    | j J

    y

    j

    j

    p  

    y θ

    β

  • Cont’

    Posterior distribution:  | ( | ) ( )p p pθ y y θ θ 1

    1 1

    j j

    J J y

    j j

    j j

      

     

     

    1

    1

    j j

    J y

    j

    j

     

     

     Which is Dirichlet 𝐷𝑖𝑟(𝜽|𝑦𝑗 + 𝛽𝑗

    The joint probability density can be written as

           1 2, 1 1 2 2 1 3 3 1 2 1 1 1 2 2( , ) | | , | , , ,m m m mf f f f f              

    e.g. For three variables,  1 2 3, ,  

        31 2 11 1

    1 2 1 2 1 2 1 2, ,1 1p           

        

  • Cont’: Marginal distributions of

    Marginal distribution for the earlier three variables

        31 2 11 1

    1 2 1 2 1 2 1 2, ,1 1p           

        

       

          1

    3 31 2 1 2

    1 1 2 2

    1 1 11 1 1 1

    1 1 2 1 2 2 1 2 1 2 2

    0

    ,

    1 1

    p p d

    p d d

         

       

              

    

    

         

    

         

     

    Let  2 11 u       

               

       

       

    321 2

    3 32 21 2 1 2

    2 3 31 2

    1 11 1

    1 1 1 1

    0

    1 1 1 11 1 1 1

    1 1 1 1 1 1 1

    0 0

    1 1 11 1

    1 1

    0

    1 1 2 3

    1 1 1

    1 1 1 1 1 1

    1 1

    ,

    u u du

    u u du u u du

    u u du

    p Beta

     

         

       

       

          

     

       

     

        

       

        

            

      

     

     

        32

    1 11

    2 3

    0

    1 ,u u du Beta    

     But

    The marginal distribution of single is Beta distribution 𝜃j

    𝜃j

  • Cont’ Similarly it can be shown that the marginal distribution of a subvector of θ is Dirichlet.

    The marginal posterior distribution for three variables.

    Let find the posterior marginal distribution of 𝜃others ,

    The posterior marginal distribution will be Dirichlet.

    Hence,

    But we are interested in

    𝜃1 , 𝜃2 , 1 21others    

       1 1 2 2 11 1

    1 2 1 2 1 2, | 1 others othersyy yp y

               

      

    1

    1

    j j

    J y

    j

    j

     

     

    

    1 1 2

    1 2

    , |p y 

         

         

      By change of variables: The transformation is one-to-one because we can solve

    for 𝜃1 , 𝜃2 ,in terms of by 𝛼, 𝜏 𝜃1 = 𝛼𝜏

    𝜃2 = 𝛼 − 1 𝜏

  • Cont.’

    1 1 2

    1 2

    , |p y 

         

         

        1

    2 1

     

      

     

       

      1

    2

    1 2

    1

    , | , |

    det(J)

    p y p y

     

      

       

     

           

         

         

    2 21 1

    1 1 2 2 1 2 1 2

    1 11

    1 1 11

    1 1 2 2 1 2 1 2

    1 1 1

    1 1

    , | | , | ,

    others others

    others others

    y yy

    y y yy y

    others othersp y Beta y y Beta y y y

     

       

          

       

            

        

            

        

      

          

    Hence,    1 1 2 2| | ,p y Beta y y     

     

         

     

      2 1

    2 21 2 2 1 1 2 1 2 2 2

    1 2 1 2 1 2

    , 1 1 1

    , 1 1

             

              

     

         

             

     

    The Jacobean

         

         

    1 1 2 2 1 2 1 2

    1 1 2 2 1 2 1 2

    | | , | ,

    | | , | ,

    others others

    others others

    p y Beta y y Beta y y y d

    p y Beta y y Beta y y y d

            

            

          

          

  • (1-B)

    Data which follows a binomial distributions

    Binomial distribution parameters

    Beta Prior

    1 2, |y y 

    1 2| ,  

    From earlier class on conjugacy we know that the posterior distribution will be Beta distribution

           2 2 2 21 1 1 1 1 11 1

    1 2 1 1 2 2| , 1 1 1 ( | , ) y yy y

    p y y Beta y y            

                

    Which is A Beta distribution similar to the result 1-A

  • Exercise 2:

  • Soln: ABC news survey of registered voters before and after debate

     

    1

    1

    | ( | ) ( )

    j

    j

    J y

    j

    j

    J y

    j

    j

    p p p

    θ y y θ θ

    Which is a multinomial distribution

    1 1

    1 1

    j j

    J J y y

    j j

    j j

       

     

       Which is a Dirichlet distribution with 𝐷𝑖𝑟(𝜽|𝑦𝑗 + 1

    Since they are independent:

    Pre-debate:

    Post-debate:

       1 2 3, , | 295,308,39pre pre prep y Direchlet   

       1 2 3, , | 289,333,20post post postp y Direchlet   

  • Cont’ For j=1,2, let be the proportion of voters who prefer Bush out of Total of who preferred Bush and Dukakis

    j

    People who prefer bush Pre debate

    11 2

    1 2 2

    post debate preference Bush ,

    post debate preference Dukakis

    postpost

    post post post

     

      

     

      People who prefer bush Post debate

    11 1

    1 2 2

    pre debate preference Bush ,

    pre debate preference Dukakis

    prepre

    pre pre pre

     

      

     

     

    But we are interested in 2 1z   

    From the earlier exercise we know that    1 1 2 2| | ,j jp y Beta y y         

        1 1

    2 2

    | | 295,308

    | | 289,333

    pre

    post

    p y Beta

    p y Beta

     

     

     

    1

    2

    1 2

    2 1 2

    , | (z ( ), w )

    det(J) z w w

    p y p

        

     

       Analytic solution:

           

    1

    2

    1 294 307 3321 2 288

    0

    , | (z) 1 1

    det(J) z w w

    p y p dw z w z w w w dw

     

     