www.elte.hu

Bayesian Models for AstronomyADA8 Summer School

Rafael S. de Souza

rafael@caesar.elte.hu

May 24, 2016

Chapter 1

Gaussian Models

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Fitting a linear model in RGaussian

Linear ModelYi = β0 + β1xi + εi εi ∼ Norm(0, σ2)

R script

N = 100sd=0.25x1<-runif(nobs)mu <- 1 + 3*x1y<-rnorm(N,mu,sd)# Fit modellm(y ~ x1)

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 3/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Normal Linear ModelsGaussian

Linear ModelYi ∼ Normal(µi , σ

2) Stochastic partµi = β0 + β1x1+ · · ·+ βkxk Deterministic part

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 4/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Normal Linear ModelsGaussian

Linear ModelYi ∼ Normal(µi , σ

2)µi = β0 + β1x1 + β2x2

1

R script

x1<-runif(nobs)mu<-1+5*x1 -0.75*x1^2y<-rnorm(N,mu ,0.5)# Fit modellm(y~x1+I(x^2))

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 5/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Normal Linear ModelsGaussian

Linear ModelYi ∼ Normal(µi , σ

2)µi = β0 + β1x1 + β2x2

R script

mu<-2+3*x1 -1.5*x2y<-rnorm(N,mu,sd)# Fit modellm(y~x1+x2)

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 6/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Normal Linear ModelsGaussian

Linear ModelYi ∼ Normal(µi , σ

2)µi = β0 + β1x1 + β2x2+β3x2

1 + β4x22

R script

mu<-2+3*x1 -4.5*x1^2+ 1.5*x2+1.5*x2^2y<-rnorm(N,mu,sd)# Fit modellm(y~x1+x2+I(x1^2)+I(x2^2))

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 7/34

The Bayesian way

p(θ|y) = p(θ)p(y |θ)p(y)

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

JAGS: Just Another Gibbs Samplerhttp://mcmc-jags.sourceforge.net

A program for analysis of Bayesian hierarchical modelsusing Markov Chain Monte Carlo (MCMC) simulationwritten with three aims in mind:

To have a cross-platform engine for the BUGS (Bayesianinference Using Gibbs Sampling) languageTo be extensible, allowing users to write their ownfunctions, distributions and samplers.To be a platform for experimentation with ideas inBayesian modelling

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 9/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

JAGS: Just Another Gibbs Sampler

An intuitive program for analysis of Bayesian hierarchicalmodels using MCMC.

Normal Model JAGS syntax

Likelihood

Yi ∼ N (µi , σ2) Y [i] ∼ dnorm(mu[i],pow(sigma,−2)

µi = β1 + β2 × Xi mu[i] < −inprod(beta[],X [i , ])

Priorsβi ∼ N (0,104) beta[i] ∼ dnorm(0,0.001)σ ∼ U(0,100) sigma ∼ dunif (0,100)

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 10/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Synthetic Normal model with JAGS

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 11/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Gaussian ModelsErrors-in-measurements

Normal ModelLikelihood

Yobs;i ∼ N (Ytrue;i, ε2Y )

Xobs;i ∼ N (Xtrue;i, ε2X )

Ytrue;i ∼ N (µi , σ2)

µi = β1 + β2 × Xtrue;i

PriorsXtrue;i ∼ N (0,102)β1 ∼ N (0,104)β2 ∼ N (0,104)σ ∼ U(0,100)

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 12/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Astronomical application of Normal modelHubble residuals-data from Wolf, R. et al. arXiv:1602.02674

LINMIXJAGS

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 13/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Astronomical examples of non-Gaussian data

Gaussian models are powerful, but fall short in manysimple applications.

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 14/34

Beyond Gaussian

Generalized Linear Models

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Beyond Normal Linear RegressionThe tip of the iceberg

Non-exhaustive list of Regression Models:Linear/Gaussian models: x ∈ R

Generalized linear modelsGamma: x ∈ R>0Log-normal: x ∈ R>0Poisson: x ∈ Z≥0Negative-binomial: x ∈ Z≥0Beta: {0 < x < 1}Beta-binomial: {0 < x < 1}Bernoulli: x = 0 or x = 1

...

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 16/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Generalized Linear ModelsBeyond Gaussian

Generalized linear model is a natural extension of theGaussian linear regression that accounts for a moregeneral family of statistical distributions.

Linear ModelYi ∼ Normal(µi , σ

2)µi = β0 + β1x1 + · · ·+ βkxk

Generalized Linear ModelsY ∼ f (µi ,a(φ)V (µ))g(µ) = ηη = β0 + β1x1 + · · ·+ βkxk

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 17/34

Chapter II

Bernoulli Models

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Bernoulli Model

Describes a random variable whichtakes the value 1 with successprobability of p and 0 with failureprobability of 1− p.

Logistic Model

Yi ∼ Bernoulli(p) ≡ py (1− p)1−y

log p1−p = η

η = β0 + β1x1 + · · ·+ βkxk

Figure: Coin toss

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 19/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Bernoulli data

Figure: 3D visualization of a Bernoulli distributed data. Redpoints are the actual data and grey curve the fit.Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 20/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Normal vs Bernoulli model

Figure: Linear regression predicts fractional (or probability)values outside the [0,1] range.

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 21/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Bernoulli modelBivariate

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 22/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Bernoulli model in JAGS

Bernoulli ModelYi ∼ Bern(pi)log( pi

1−pi) = ηi

ηi = β0 + β1Xi

JAGSY [i] ∼ Bern(p[i])logit(p[i]) = η[i]η[i] = β[1] + β[2] ∗ X [i]

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 23/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Astronomical application of Bernoulli modelStar formation activity in early Universe, data from Biffi and Maio,arXiv:1309.2283

Bernoulli ModelSFR ∼ Bern(pi)log( pi

1−pi) = ηi

ηi = β0 + β1 log xmol+β2(log xmol)

2 +β3(log xmol)

3

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 24/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Astronomical application of Bernoulli modelStar formation activity in early Universe, data from Biffi and Maio,arXiv:1309.2283, see also de Souza et al. arxiv.org/abs/1409.7696

Bernoulli ModelSFR ∼ Bern(pi)log( pi

1−pi) = ηi

ηi = β0 + β1 log xmol+β2(log xmol)

2 +β3(log xmol)

3

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 25/34

Chapter III

Count Models

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Poisson ModelsNumber of events

Figure: 3D visualization of a Poisson distributed data.

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 27/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Poisson ModelsNumber of events

The Poisson distribution model the number of times anevent occurs in an interval of time or space.

Linear ModelsY ∼ Normal(µ, σ2)µ = β0 + β1x1 + · · ·+ βkxk

Poisson ModelY ∼ Poisson(µ) ≡ µye−µ

y!log(µ) = ηη = β0 + β1x1 + · · ·+ βkxk

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 28/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Poisson modelMean equals variance

Although traditional, it is rarely useful in real situations.

Figure: Poisson distributed data

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 29/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Negative binomial modelVariance exceeds the sample mean

The NB distribution can be seen as a Poisson distribution,where the mean is itself a random variable, distributed asa gamma distribution.

Negative Binomialdistribution

Γ(y+θ)Γ(θ)Γ(y+1)

(θ

µ+θ

)θ (1− θ

µ+θ

)y

Mean = µ Var = µ+ µ2

θ

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 30/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Negative Binomial modelSynthetic data

Accounts for Poisson over-dispersion

Figure: Negative binomial distributed data

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 31/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Astronomical application of COUNT modelsGlobular Cluster Population, see R. S. de Souza, et al. MNRAS, 453, 2,1928, 2015

Figure: Globular Cluster population as a function of the galaxybrightness

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 32/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

Final Remarks

Generalized linear models are a cornerstone of modernstatistics, but nearly Terra incognita in astronomicalinvestigations.Recent examples of GLM applications in astronomy arephoto-z (Gamma-distributed; Elliott et al., 2015), GCspopulation (NB-distributed; de Souza at al. 2015), AGNactivity (Bernoulli-distributed; Souza et al. 2016).

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 33/34

GaussianModels

GLMs

Bernoulli Models

COUNT Models

Final Remarks

References

References

R. S. de Souza, M. L. Dantas, et al. arXiv:1603.06256R. S. de Souza, E. Cameron, et al. A&C, 12, 21, 2015J. Elliott, R. S. de Souza, et al. A&C, 10, 6, 2015R. S. de Souza, J. M. Hilbe, et al. MNRAS, 453, 2,1928, 2015J. Hilbe, R. S. de Souza and E. E. O. Ishida, BayesianModels for Astrophysical data, Cambridge UniversityPress, in prepJ. Hilbe, Practical Guide to Logistic Regression,Chapman and Hall/CRC, 2015S. Andreon & B. Weaver, Bayesian Methods for thePhysical Sciences, Springer, 2015

Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 34/34