Quant Toolbox > 23. Bayesian statistics > Analytical resultsConjugate distribution

Conjugate distribution

A conjugate distribution with respect to a Bayesian model (24.3) is aparametric distribution fη with a set of hyperparameters η such that

fpri(θ) = fηpri(θ) ⇒ fpos(θ) = fηpos

(θ) (24.16)

where fpri is the prior distribution (24.4) and fpos is the posteriordistribution (24.6).

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Feb-21-2017 - Last update

Quant Toolbox > 23. Bayesian statistics > Analytical resultsConjugate distribution

Conjugate distribution

Consider the exponential family

X ∼ Exp(θ, φ(·), h(·)) ⇔ f(x|θ) = h(x)eθ′φ(x)−ψ(θ) (23.16)

where θ ≡ (θ1, . . . , θl̄)′ are the natural parameters; h(x) > 0 is the base

measure; φ(x) ≡ (φ1(x), . . . , φl̄(x))′ are the features; and ψ(θ) is the

log-partition.

The conjugate distribution for the natural parameters is

Θ|η, ν ∼ Conj (ψ (·) ,η, ν) ⇔ fψ(θ|η, ν) = g(η, ν)eν(θ′η−ψ(θ)) (23.17)

where g(η, ν) normalizes the pdf to integrate to one

g(η, ν) =1∫

Rl̄ eν(θ′η−ψ(θ))dθ(23.18)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Feb-21-2017 - Last update

Quant Toolbox > 23. Bayesian statistics > Analytical resultsConjugate distribution

Properties of the conjugate distribution

• The conjugate (23.17) in general does not belong to the exponentialfamily.

• The hyperparameter η ∈ Rl̄ satisfies

η = ∇θψ(Mod{Θ}) (23.19)

• The parameter η is the most likely value of the expected featuresimplied by the conjugate

η ≈ E{φ(X)} (23.20)

Example: normal distribution with unit variance

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Feb-21-2017 - Last update

Quant Toolbox > 23. Bayesian statistics > Analytical resultsConjugate distribution

Properties of the conjugate distribution

• If the model (24.3) is exponential (24.17) and the prior (24.4) isconjugate (24.18), then the posterior (24.16) is also conjugate

X ∼ Exp(θ, φ(·), h(·))Θ|ηpri , νpri ∼ Conj (ψ (·) ,ηpri , νpri)

}⇒ Θ|ηpos , νpos ∼ Conj (ψ (·) ,ηpos , νpos) (24.25)

with new parameters

ηpos ≡νpri

νpri + 1ηpri +

1

νpri + 1φ(x), νpos ≡ νpri + 1 (24.26)

• The predictive distribution reads

fpred(x) = h(x)g(ηpos , νpos)

g( 11+νpos

φ(x) +νpos

1+νposηpos , 1 + νpos)

(24.27)

Example: normal distribution with unit variance

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Feb-21-2017 - Last update