Appendix - Distributions

DISTRIBUTIONS 512

18.2 Continuous univariate distributions

Table 18.1 Beta density: Beta(α, β)

Model Examples

p(θ) =1

B(α,β) θα−1(1− θ)β−1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

θ

Density

α = 3, β = 3

α = 4, β = 1

α = 1, β = 1

α = 0.7, β = 0.7

α = 0.2, β = 6

with B(α, β) = Γ(α)Γ(β)Γ(α+β)

Condition: α > 0, β > 0Range: [0,1]

Parameters:

α, β: shape

Moments Program commands

mean: α(α+β) R: dbeta(theta,alpha,beta)

mode: α−1(α+β−2) WB/JAGS: theta ~ dbeta(alpha,beta)

variance: αβ(α+β)2(α+β+1) SAS: theta ~ beta(alpha,beta)

DISTRIBUTIONS 513

Table 18.2 Cauchy distribution: Cauchy(µ, σ)

Model Examples

p(θ) =1π

(σ

σ2+(θ−µ)2

)

0 2 4 6 8 10

0.0

0.1

0.2

0.3

0.4

0.5

θ

Density

µ = 5, σ = 1

µ = 4, σ = 2

µ = 7, σ = 1

N(µ = 5, σ = 1)

Condition: σ > 0Range: (−∞,∞)

Parameters:

µ: location, σ: scale


mean: - R: dcauchy(theta,mu,sigma)

mode: µ WB/JAGS: -

variance: - SAS: theta ~ cauchy(mu,sigma)

Note:Cauchy distribution is a special case of location-scale t-distribution:Cauchy(µ, σ) = t(1, µ, σ).

DISTRIBUTIONS 514

Table 18.3 Chi-squared density: χ2(ν)

Model Examples

p(θ) =1

Γ(ν/2)2ν/2 θ(ν/2)−1e−θ/2

0 2 4 6 80.0

0.2

0.4

0.6

0.8

1.0

θ

Density

ν = 1

ν = 5ν = 0.01 ν = 10

Condition: ν > 0Range: ν = 2 : [0,∞)

otherwise : (0,∞)

Parameters:

ν: degrees of freedom


mean: nu R: dchisq(theta,nu)

mode: ν − 2 (ν ≥ 2), otherwise − WB/JAGS: theta ~ dchisqr(nu)

variance: 2ν SAS: theta ~ chisq(nu)

Note:Chi-squared is a special case of a gamma distribution:χ2(ν) = Gamma(α = ν/2, β = 1/2) (rate).

JAGS offers a non-central χ2-distribution:‘theta ∼ dnchisqr(nu,delta)’, δ > 0 non-centrality parameter.

JAGS offers an F-distribution (ratio of 2 independent χ2s):‘theta ∼ df(nu1, nu2)’, with nu1, nu2 = dfs of numerator and denominator, resp.

DISTRIBUTIONS 515

Table 18.4 Exponential density: Exp(λ)

Model Examples

rate: p(θ) = λe−λθ

0 2 4 6 8

0.0

0.5

1.0

1.5

2.0

2.5

3.0

θD

ensity

λ = 1

λ = 2

λ = 4

λ = 0.1

rate = λ

Condition: λ > 0Range: [0,∞)

Parameters:

λ: rate


rate: λmean: 1

λ R: dexp(theta,lambda)

mode: 0 WB/JAGS: theta ~ dexp(lambda)

variance: 1λ2 SAS: theta ~ expon(iscale=lambda)

(scale) theta ~ expon(scale=ilambda)

Note:Exponential is special case of gamma distribution:Exp(λ)= Gamma(α = 1, λ).

DISTRIBUTIONS 516

Table 18.5 Gamma density: Gamma(α, β)

Model Examples

rate:

p(θ) = βα

Γ(α) θ(α−1) e−βθ

0 2 4 6 80.0

0.5

1.0

1.5

2.0

2.5

3.0

θ

Density

α = 4, β = 1

α = 0.1, β = 0.1

α = 20, β = 20

α = 1, β = 1

1/scale = β

Condition: α > 0, β > 0Range: α = 1 : (0,∞)

otherwise : [0,∞)

Parameters:

α: shape, β: rate


rate: βmean: α

β R: dgamma(theta,alpha,rate=beta)

(scale) dgamma(theta,alpha,scale=ibeta)

mode: α−1β (α ≥ 1) WB/JAGS: theta ~ dgamma(alpha,beta)

variance: αβ2 SAS: theta ~ gamma(alpha,iscale=beta)

(scale) theta ~ gamma(alpha,scale=ibeta)

Note:WB and JAGS offer a generalized gamma distribution GenGamma:

θ ∼ GenGamma(α, β∗, λ) ⇔ θ1/λ ∼ Gamma(α, β), with β∗ = β1/λ.

WB/JAGS command: ‘theta ∼ dgen.gamma(alpha,beta,lambda)’.

DISTRIBUTIONS 517

Table 18.6 Inverse chi-squared density: Inv− χ2(ν)

Model Examples

p(θ) =1

Γ(ν/2)2ν/2 θ−(ν/2+1)e−1/(2θ)

0 5 10 150

12

34

θ

Density

ν = 1

ν = 3

ν = 5

Condition: ν > 0Range: (0,∞)

Parameters:

ν: degrees of freedom


mean: 1ν−2 (ν > 2) R: dchisq(1/theta,nu)/theta^2

mode: 1ν+2 WB/JAGS: theta <- 1/itheta;

itheta ~ dchisqr(nu)

variance: 2(ν−2)2(ν−4) (ν > 4) SAS: theta ~ ichisq(nu)

Note:Inverse χ2 is a special case of the inverse gamma-distribution: (rate).Inv− χ2(ν) = IG(α = ν/2, β = 1/2) (rate).

Inverse χ2 is a special case of the scaled inverse χ2-distribution with ν s2 = 1.

DISTRIBUTIONS 518

Table 18.7 Inverse gamma density: IG(α, β)

Model Examples

rate:

p(θ) = 1βαΓ(α) θ

−(α+1) e−β/θ

0 2 4 6 80

12

34

θ

Density

α = 1, β = 1

α = 4, β = 1

α = 20, β = 20

Gamma(α = 4, β = 1)

1/scale = β

Condition: α > 0, β > 0Range: (0,∞)

Parameters:

α: shape, β: rate


rate: β

mean: β(α−1) R: dgamma(1/theta,alpha,rate=beta)/theta^2

(scale) dgamma(1/theta,alpha,scale=beta)/theta^2

mode: β(α+1) WB/JAGS: theta <- 1/itheta;

itheta ~ dgamma(alpha,beta)

variance: β2

(α−1)2(α−2) SAS: theta ~ igamma(alpha,iscale=beta)

(scale) theta ~ igamma(alpha,scale=ibeta)

Note:θ ∼ IG(α, β) ⇔ 1/θ ∼ Gamma(α, β).

DISTRIBUTIONS 519

Table 18.8 Laplace density: Laplace(µ, σ)

Model Examples

scale:

p(θ) = 12σ e−(θ−µ)/σ

0 2 4 6 8 100.0

0.1

0.2

0.3

0.4

0.5

0.6

θ

Density

µ = 5, σ = 1

µ = 4, σ = 2

µ = 7, σ = 1


Parameters:



scale: σmean: µ R: dlaplace(theta,mu,sigma)

mode: µ WB/JAGS: -

(rate) theta ~ ddexp(isigma)

variance: 2σ2 SAS: theta ~ laplace(mu,scale=sigma)

(rate) theta ~ laplace(mu,iscale=isigma)

Note:Laplace distribution is also called double exponential distribution.

R function dlaplace is available from R package ‘VGAM’.

DISTRIBUTIONS 520

Table 18.9 Logistic distribution: Logistic(µ, σ)

Model Examples

p(θ) =

exp(− θ−µ

σ

) [σ exp

(− θ−µ

σ

)]2

0 2 4 6 8 10

0.0

0.1

0.2

0.3

0.4

0.5

θ

Density

µ = 5, σ = 1

µ = 4, σ = 2

µ = 7, σ = 1

N(µ = 5, σ = 1)


Parameters:



mean: µ R: dlogis(theta,mu,sigma)

mode: µ WB/JAGS: theta ~ dlogis(mu,isigma) (rate)

variance: π2σ2

3 SAS: theta ~ logistic(mu,sigma)

DISTRIBUTIONS 521

Table 18.10 Lognormal distribution: LN(µ, σ2)

Model Examples

p(θ) =1

θσ√2π

exp(− (log(θ)−µ)2

2σ2

)

0 2 4 6 8 10

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

θ

Density

µ = 0, σ = 1

µ = 2, σ = 1

µ = 0, σ = 2

µ = 4, σ = 2

Condition: σ > 0Range: (0,∞)

Parameters:



mean: exp(µ+ σ2) R: dlnorm(theta,mu,sigma)

mode: exp(µ− σ2) WB/JAGS: theta ~ dlnorm(mu,isigma2)

variance:

exp(2(µ+ σ2))− exp(2µ+ σ2) SAS: theta ~ lognormal(mu,sd=sigma)

theta ~ lognormal(mu,var=sigma2)

theta ~ lognormal(mu,prec=isigma2)

DISTRIBUTIONS 522

Table 18.11 Normal distribution: N(µ, σ2)

Model Examples

p(θ) =1

σ√2π

exp(− (θ−µ)2

2σ2

)

0 2 4 6 8 10

0.0

0.1

0.2

0.3

0.4

0.5

θ

Density

µ = 5, σ = 1

µ = 4, σ = 2

µ = 7, σ = 1


Parameters:



mean: µ R: dnorm(theta,mu,sigma)

mode: µ WB/JAGS: theta ~ dnorm(mu,isigma2)

variance: σ2 SAS: theta ~ normal(mu,sd=sigma)

theta ~ normal(mu,var=sigma2)

theta ~ normal(mu,prec=isigma2)

DISTRIBUTIONS 523

Table 18.12 Location-scale Student’s t-distribution: t(ν, µ, σ)

Model Examples

p(θ) =

Γ( ν+12 )

Γ( ν2 )σ

√νπ

(1 + (θ−µ)2

νσ2

)− ν+12

0 2 4 6 8 100.0

0.1

0.2

0.3

0.4

0.5

θ

Density

ν = 2, µ = 5, σ = 1ν = 20, µ = 5, σ = 1

ν = 10, µ = 4, σ = 2

N(µ = 5, σ = 1)

Condition: σ > 0, ν > 0Range: (−∞,∞)

Parameters:

µ: location, σ: scaleν: degrees of freedom


mean: µ (if ν > 1) R: dt(nu,(theta-mu)/sigma)/sigma

mode: µ WB/JAGS: theta ~ dt(mu,isigma2,nu)

variance: νν−2σ

2 (if ν > 2) SAS: theta ~ t(mu,sd=sigma,nu)

theta ~ t(mu,var=sigma2,nu)

theta ~ t(mu,prec=isigma2,nu)

DISTRIBUTIONS 524

Table 18.13 Pareto distribution: Pareto(α, β)

Model Examples

p(θ) = αβ

(βθ

)α+1

1 2 3 4 5 6 7 80

12

34

θ

Density

α = 1, β = 1

α = 4, β = 1

α = 4, β = 4

Condition: α > 0, β > 0Range: (β,∞)

Parameters:

α: shape, β: location


mean: αβα−1 (if α > 1) R: dpareto(theta,beta,alpha)

mode: β WB/JAGS: theta ~ dpareto(alpha,beta)

variance: αβ2

(α−1)2(α−2) (if α > 2) SAS: theta ~ pareto(alpha,beta)

Note:R function dpareto is available from R package ‘VGAM’.

DISTRIBUTIONS 525

Table 18.14 Scaled inverse chi-squared density: Inv− χ2(ν, s2)

Model Examples

p(θ) =(ν/2)ν/2

Γ(ν/2) sνθ−(ν/2+1)e−νs

2/(2θ)

0 5 10 15

0.0

0.2

0.4

0.6

0.8

θ

Density

ν = 3, s2

= 1

ν = 5, s2

= 1

ν = 3, s2

= 5

Condition: ν > 0, s > 0Range: (0,∞)

Parameters:

ν: degrees of freedom, s2: scale


mean: νν−2s

2 (ν > 2) R: dchisq(nu*s^2/theta,nu)nu*

s^2/theta^2

mode: νν+2s

2 WB/JAGS: theta <- nu*s^2/itheta;

itheta ~ dchisqr(nu)

variance: 2ν2

(ν−2)2(ν−4)s4 (ν > 4) SAS: theta ~ sichisq(nu,s)

Note:Scaled inverse chi-squared is a special case of the inverse gamma distribution:Inv− χ2(ν, s2) = IG(α = ν/2, β = ν s2/2) (rate).

DISTRIBUTIONS 526

Table 18.15 Weibull distribution: Weibull(α, β)

Model Examples

p(θ) =

αβ

(θβ

)(α−1)

exp (−(θ/β)α)

0 2 4 6 8 10

0.0

0.2

0.4

0.6

0.8

1.0

θ

Density

α = 1, β = 1

α = 2, β = 1

α = 2, β = 2

α = 2, β = 4

Condition: α > 0, β > 0Range: α = 1 : [0,∞)

otherwise : (0,∞)

Parameters:

α: shape, β: scale


mean: βΓ(1 + 1/α) R: dweibull(theta,alpha,beta)

mode: β(1− 1/α)1/α (if α > 1) WB/JAGS: theta ~ dweib(alpha,ibeta)

variance:

β2[Γ(1 + 2/α)− Γ2(1 + 2/α)

]SAS: theta ~ weibull(0,alpha,beta)

Note:SAS: more general Weibull distribution with additional µ > 0 = lower limit of range:‘weibull(mu,alpha,beta)’, with θ/β in Weibull distribution replaced by (θ − µ)/β.

DISTRIBUTIONS 527

Table 18.16 Uniform distribution: U(α, β)

Model Examples

p(θ) = 1β−α

0 1 2 3 4 5

0.0

0.2

0.4

0.6

0.8

1.0

1.2

θDensity

α = 1, β = 2

α = 2, β = 5

Condition: β > αRange: [α, β]

Parameters:

α: lower limit, β: upper limit


mean: α+β2 R: dunif(theta,alpha,beta)

mode: - WB/JAGS: theta ~ dunif(alpha,beta)

variance:(β−α)2

12 SAS: theta ~ uniform(alpha,beta)

Note:Uniform is a special case of the beta distribution: U(0,1) = Beta(1,1).

DISTRIBUTIONS 528

18.3 Discrete univariate distributions

Table 18.17 Binomial distribution: Bin(n, π)

Model Examples

p(θ) =(nθ

)πθ(1− π)n−θ

0 2 4 6 8 10

0.0

0.1

0.2

0.3

0.4

θ

Distribution n = 10, π = 0.5

n = 10, π = 0.7

n = 10, π = 0.1

Conditions:

n = 0, 1, 2, . . .0 ≤ π ≤ 1

Range: θ ∈ {0, 1, . . . , n}Parameters:

n: sample sizeπ: probability of success


mean: nπ R: dbinom(theta,n,pi)

mode: ⌊(n+ 1)π⌋ WB/JAGS: theta ~ dbin(pi,n)

variance: nπ(1− π) SAS: theta ~ binomial(n,pi)

Note:⌊(n+ 1)π⌋ = greatest integer in value.

Special case: Bernoulli distribution = Bern(π) = Bin(1,π).Commands Bernoulli dist: R: dbern(pi), WB: dbern(pi), SAS: binary(pi).

DISTRIBUTIONS 529

Table 18.18 Categorical distribution: Cat(π)

Model Examples

p(θ) = πθ

0 1 2 3 4 5

0.0

0.2

0.4

0.6

0.8

1.0

θD

istr

ibution

π= (0.1,0.4,0.2,0.3)

Conditions:

πθ > 0,∑πθ = 1


πθ: class probabilities


mean: − R: dmultinom(theta,size=1,pi)

mode: − WB/JAGS: theta ~ dcat(pi)

variance: − SAS: theta ~ multinom(pi)

Note:Categorical is a special case of the multinomial distribution with n = 1.

JAGS only requires that πθ is positive, they must not add up to 1.

DISTRIBUTIONS 530

Table 18.19 Negative binomial distribution: NB(n, π)

Model Examples

p(θ) =(θ+n−1

θ

)πn(1− π)θ

0 5 10 15 20

0.00

0.05

0.10

0.15

0.20

0.25

0.30

θDistribution

n = 5, π = 0.5

n = 5, π = 0.7

n = 5, π = 0.2

Conditions:

n = 0, 1, 2, . . .0 ≤ π ≤ 1


n: number of successesπ: probability of success


mean: round(n(1−π)

π

)R: dnegbin(theta,n,pi)

mode: round(

(n−1)(1−π)π

)WB/JAGS: theta ~ dnegbin(pi,n)

variance:n(1−π)π2 SAS: theta ~ negbin(n,pi)

Note:Special case: Geometric distribution: geom(p)=NB(1,π).

We have seen alternative parametrizations of the negative binomial distribution in the book:Expression (3.15): π = β/(1 + β) and n = α a real value.Expression (6.19): π = 1/(1 + κλ) and n = 1/κ a real value.

DISTRIBUTIONS 531

Table 18.20 Poisson distribution: Poisson(λ)

Model Examples

p(θ) = λθ

θ! exp(−λ)

0 5 10 15 20

0.00

0.05

0.10

0.15

0.20

0.25

0.30

θDistribution

n = 5, π = 0.5

n = 5, π = 0.7

n = 5, π = 0.2

Condition: λ ≥ 0Range: θ ∈ {0, 1, . . . , n}Parameters:

λ: average number of counts


mean: λ R: dpois(theta,lambda)

mode: round(λ) WB/JAGS: theta ~ dpois(lambda)

variance: λ SAS: theta ~ poisson(lambda)

DISTRIBUTIONS 532

18.4 Multivariate distributions

Table 18.21 Dirichlet distribution: Dirichlet(α)

Model Program commands

p(θ) =Γ(

∑Jj=1 αj)∏J

j=1 Γ(αj)

∏Jj=1 θ

αj−1j

R: ddirichlet(vtheta,valpha)

Condition: αj > 0 (j = 1, . . . , J) WB/JAGS: vtheta[] ~ ddirich(valpha[])

Range: θj > 0,∑Jj=1 θj = 1 SAS: vtheta ~ dirich(valpha)

Parameters:

αj : probabilities

Moments

mean: αj/∑Jj=1 αj mode: (αj − 1)/

∑Jj=1 αj

variances:αj(

∑m αm−αj)

(∑

m αm)2(∑

m αm−αj)covariances: − αjαk

(∑

m αm)2(∑

m αm+1)

Table 18.22 Inverse Wishart distribution: IW(R, k)


p(Σ) = R: diwish(Sigma, k, Rinv)

cdet(R)k/2 det(Σ)−(k+p+1)/2 exp[−1

2 tr (Σ−1R)

](Rinv=R−1 in MCMCpack)

with

c−1 = 2kp/2πp(p−1)/4∏pj=1 Γ

(k+1−j

2

)Condition: R pos definite, k > 0 WB/JAGS: -

Range: Σ symmetric SAS: Sigma ~ iwishart(k,R)

Parameters:

k: degrees of freedom & R: inverse of cov matrix

Moments

mean: R/(k − p− 1) (if k > p+ 1) mode: R/(k + p+ 1)

DISTRIBUTIONS 533

Table 18.23 Multinomial distribution: Mult(k,π)


p(θ) = n!θ1!θ2!...θk!

∏kj=1 π

θjj , R: dmultinom(theta,size=n,prob=vpi)

Condition: WB/JAGS: vtheta[] ~ dmulti(pi[],n)∑kj=1 πj = 1

Range: θj ∈ {0, . . . , n} with∑kj=1 θj = n SAS: vtheta ~ multinom(vpi)

Parameters:

πj : probabilities

Moments

mean: n · πvariances: nπj(1− πj) covariances: −nπjπk

Table 18.24 Multivariate Normal distribution: Np(µ,Σ)


p(θ) = R: mvrnorm(vtheta,vmu,S) (MASS)1

(2π)p/2 det(Σ)1/2exp

[−1

2 (θ − µ)TΣ−1(θ − µ)]

Condition: WB/JAGS: vtheta[] ~ dmnorm(vmu[],S[,])

Σ positive definiteRange: −∞ < θj <∞ SAS: vtheta ~ mvn(vmu,S)

Parameters:

µ: mean vector & Σ: p× p covariance matrix

Moments

mean: µ mode: µ

variances: Σjj covariances: Σjk

DISTRIBUTIONS 534

Table 18.25 Multivariate Student’s t-distribution: Tν(µ,Σ)


p(θ) = R: -

cdet(Σ)−1/2[1 + 1

ν (θ − µ)TΣ−1(θ − µ)]−(ν+p)/2

with c = Γ[(ν+p)/2]Γ(ν/2)(kπ)p/2

Condition: WB/JAGS: vtheta[] ~ dmt(vmu[],S[,],nu)

Σ positive definite, ν > 0

Range: −∞ < θj <∞ SAS: -

Parameters:

µ: mean vectorΣ: p× p covariance matrixν: degrees of freedom

Moments

mean: µ (if ν > 1) mode: µ

variances: νν−2Σjj (if ν > 2) covariances: ν

ν−2Σjk (if ν > 2)

DISTRIBUTIONS 535

Table 18.26 Wishart distribution: Wishart(R, k)


p(Σ) = R: dwish(Sigma, k, Rinv)

cdet(R)−k/2 det(Σ)(k−p−1)/2 exp[− 1

2 tr (R−1Σ)

](Rinv=R−1 in MCMCpack)

with

c−1 = 2kp/2πp(p−1)/4∏pj=1 Γ

(k+1−j

2

)Condition: R pos definite, k > 0 WB/JAGS: Sigma[,] ~ dwish(R[,],k)

Range: Σ symmetric SAS: -

Parameters:

k: degrees of freedom & R: covariance matrix

Moments

mean: kR mode: (k − p− 1)R (if k > p+ 1)

variances: covariances:

var(Σij) = k(r2ij + riirjj) cov(Σij ,Σkl) = k(rikrjl + rilrjk)

Note:WinBUGS uses an alternative expression of the Wishart distribution: In the aboveexpression R is replaced by R−1 and hence represents a covariance matrix in WinBUGS.

Appendix - Distributions

Documents

Transcript of Appendix - Distributions