Quant Toolbox > 22. Multivariate distributions > Exponential family distributions

Exponential family distributions

A random vector X ≡ (X1, . . . , Xn̄)′ ∈ Rn̄ has an exponential familydistribution

X ∼ Exp(θ, φ(·), h(·)) (22.108)

if its pdf can be written in canonical form

fθ(x) = h(x)eθ′φ(x)−ψ(θ) (22.109)

where• θ ≡ (θ1, . . . , θl̄)

′ ∈ Rl̄ are natural or canonical parameters• φ(x) ≡ (φ1(x), . . . , φl̄(x))′ are sufficient statistics, or features, orHamiltonians

• h(x) > 0 is a function Rn̄ → R known as base or auxiliary measure• ψ(·) is the log-partition function

ψ(θ) ≡ ln(

∫Rn̄

h(x)eθ′φ(x)dx) (22.110)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Mar-13-2017 - Last update

Quant Toolbox > 22. Multivariate distributions > Exponential family distributions

Exponential family distributionsFor an exponential distribution it is possible to maximize the likelihood(23.5) numerically toward the unique global maximum.

The log-likelihood

ln fθ(x) = θ′φ(x)− ψ(θ) + lnh(x) (22.111)

is concave in the parameters θ. Indeed• the expectation of the features is

η ≡ Eθ{φ(X)} = ∇θψ(θ) (22.112)

• the covariance of the features is

Cvθ{φ(X)} = ∇2θ,θψ(θ) = Cvθ{∇θ ln fθ(X)} (22.113)

Then ∇2θ,θψ(θ) � 0, i.e. ψ(θ) is convex and ln fθ(x) is concave.

Furthermore, ∇θψ(θ) is a one-to-one function, thus η can be used asan equivalent parameterization of the exponential family.

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Mar-13-2017 - Last update

Quant Toolbox > 22. Multivariate distributions > Exponential family distributionsNormal distribution

Normal distribution

From the canonical form

fθ(x) = h(x)eθ′φ(x)−ψ(θ) (22.109)

we obtain the multivariate normal distribution fNµ,σ2 (22.115) by setting

• natural parameters

θ ≡(

θµvec(θσ)

)≡(

(σ2)−1µ− 1

2vec((σ2)−1)

)(22.119)

• features

φ(x) ≡(φµ(x)φσ(x)

)≡(

xvec(xx′)

)(22.120)

• base measure h(x) ≡ (2π)−n̄/2

• log-partition function

ψ(θ) = − 14θ′µ(θσ)−1θµ − 1

2ln det(−2θσ) (22.121)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Mar-13-2017 - Last update

Quant Toolbox > 22. Multivariate distributions > Exponential family distributionsScenario-probability distribution

Scenario-probability distributionConsider a scenario-probability distribution

X ∼ {x(j), p(j)}j̄j=0 (22.125)

Then we can write

fθ(x) = h(x)eθ′φ(x)−ψ(θ), θ ∈ Rj̄ (22.126)

by setting [E.22.77]• natural parameters

θj ≡ lnp(j)

p(0), j = 1, . . . , j̄ (22.127)

• featuresφj(x) ≡ 1x=x(j) , j = 1, . . . , j̄ (22.129)

• base measureh(x) ≡

∑j̄j=0δ

(x(j))(x) (22.128)

• log-partition function

ψ(θ) = ln(1 +∑j̄j=1e

θj ) (22.130)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Mar-13-2017 - Last update