of 13

• date post

19-Oct-2020
• Category

## Documents

• view

0

0

Embed Size (px)

### Transcript of STK4021 - Applied Bayesian Analysis and Numerical Methods STK9021 - Applied Bayesian Analysis and...

• STK9021 - Applied Bayesian Analysis and Numerical Methods CHAPTER 3: EXERCISE 1 AND 2

MOHAMMED AHMED KEDIR

• 3.10 Exercises

• Solution: (1-A)

y

θ

1..J Multinomial distribution parameters

1,....... |Jy y  Data which follows a multinomial distributions

Prior distributions on multinomial distribution (Dirichlet)

• Multinomial distribution: is a convenient way to represent discrete

variable that can take on of possible mutually exclusive states. e.g. 1

of K scheme T(0,0,0,1,0,0)y 

1

1

| ( | ) j J

j

j

Dir 

     



We are interested in marginal posterior distribution for 1

1 2

 

  

  1

| j J

y

j

j

p  

y θ

β

• Cont’

Posterior distribution:  | ( | ) ( )p p pθ y y θ θ 1

1 1

j j

J J y

j j

j j

  

 

 

1

1

j j

J y

j

j

 

 

 Which is Dirichlet 𝐷𝑖𝑟(𝜽|𝑦𝑗 + 𝛽𝑗

The joint probability density can be written as

       1 2, 1 1 2 2 1 3 3 1 2 1 1 1 2 2( , ) | | , | , , ,m m m mf f f f f              

e.g. For three variables,  1 2 3, ,  

    31 2 11 1

1 2 1 2 1 2 1 2, ,1 1p           

    

• Cont’: Marginal distributions of

Marginal distribution for the earlier three variables

    31 2 11 1

1 2 1 2 1 2 1 2, ,1 1p           

    

   

      1

3 31 2 1 2

1 1 2 2

1 1 11 1 1 1

1 1 2 1 2 2 1 2 1 2 2

0

,

1 1

p p d

p d d

     

   

          





     



     

 

Let  2 11 u       

           

   

   

321 2

3 32 21 2 1 2

2 3 31 2

1 11 1

1 1 1 1

0

1 1 1 11 1 1 1

1 1 1 1 1 1 1

0 0

1 1 11 1

1 1

0

1 1 2 3

1 1 1

1 1 1 1 1 1

1 1

,

u u du

u u du u u du

u u du

p Beta

 

     

   

   

      

 

   

 

    

   

    

        

  

 

 

    32

1 11

2 3

0

1 ,u u du Beta    

 But

The marginal distribution of single is Beta distribution 𝜃j

𝜃j

• Cont’ Similarly it can be shown that the marginal distribution of a subvector of θ is Dirichlet.

The marginal posterior distribution for three variables.

Let find the posterior marginal distribution of 𝜃others ,

The posterior marginal distribution will be Dirichlet.

Hence,

But we are interested in

𝜃1 , 𝜃2 , 1 21others    

   1 1 2 2 11 1

1 2 1 2 1 2, | 1 others othersyy yp y

           

  

1

1

j j

J y

j

j

 

 



1 1 2

1 2

, |p y 

     

     

  By change of variables: The transformation is one-to-one because we can solve

for 𝜃1 , 𝜃2 ,in terms of by 𝛼, 𝜏 𝜃1 = 𝛼𝜏

𝜃2 = 𝛼 − 1 𝜏

• Cont.’

1 1 2

1 2

, |p y 

     

     

    1

2 1

 

  

 

   

  1

2

1 2

1

, | , |

det(J)

p y p y

 

  

   

 

       

     

     

2 21 1

1 1 2 2 1 2 1 2

1 11

1 1 11

1 1 2 2 1 2 1 2

1 1 1

1 1

, | | , | ,

others others

others others

y yy

y y yy y

others othersp y Beta y y Beta y y y

 

   

      

   

        

    

        

    

  

      

Hence,    1 1 2 2| | ,p y Beta y y     

 

     

 

  2 1

2 21 2 2 1 1 2 1 2 2 2

1 2 1 2 1 2

, 1 1 1

, 1 1

         

          

 

     

         

 

The Jacobean

     

     

1 1 2 2 1 2 1 2

1 1 2 2 1 2 1 2

| | , | ,

| | , | ,

others others

others others

p y Beta y y Beta y y y d

p y Beta y y Beta y y y d

        

        

      

      

• (1-B)

Data which follows a binomial distributions

Binomial distribution parameters

Beta Prior

1 2, |y y 

1 2| ,  

From earlier class on conjugacy we know that the posterior distribution will be Beta distribution

       2 2 2 21 1 1 1 1 11 1

1 2 1 1 2 2| , 1 1 1 ( | , ) y yy y

p y y Beta y y            

            

Which is A Beta distribution similar to the result 1-A

• Exercise 2:

• Soln: ABC news survey of registered voters before and after debate

 

1

1

| ( | ) ( )

j

j

J y

j

j

J y

j

j

p p p

θ y y θ θ

Which is a multinomial distribution

1 1

1 1

j j

J J y y

j j

j j

   

 

   Which is a Dirichlet distribution with 𝐷𝑖𝑟(𝜽|𝑦𝑗 + 1

Since they are independent:

Pre-debate:

Post-debate:

   1 2 3, , | 295,308,39pre pre prep y Direchlet   

   1 2 3, , | 289,333,20post post postp y Direchlet   

• Cont’ For j=1,2, let be the proportion of voters who prefer Bush out of Total of who preferred Bush and Dukakis

j

People who prefer bush Pre debate

11 2

1 2 2

post debate preference Bush ,

post debate preference Dukakis

postpost

post post post

 

  

 

  People who prefer bush Post debate

11 1

1 2 2

pre debate preference Bush ,

pre debate preference Dukakis

prepre

pre pre pre

 

  

 

 

But we are interested in 2 1z   

From the earlier exercise we know that    1 1 2 2| | ,j jp y Beta y y         

    1 1

2 2

| | 295,308

| | 289,333

pre

post

p y Beta

p y Beta

 

 

 

1

2

1 2

2 1 2

, | (z ( ), w )

det(J) z w w

p y p

    

 

   Analytic solution:

       

1

2

1 294 307 3321 2 288

0

, | (z) 1 1

det(J) z w w

p y p dw z w z w w w dw

 

 