Bayesian Models for Astronomy

download Bayesian Models for Astronomy

of 34

  • date post

    11-Apr-2017
  • Category

    Science

  • view

    259
  • download

    2

Embed Size (px)

Transcript of Bayesian Models for Astronomy

  • www.elte.hu

    Bayesian Models for AstronomyADA8 Summer School

    Rafael S. de Souza

    rafael@caesar.elte.hu

    May 24, 2016

    rafael@caesar.elte.hu

  • Chapter 1

    Gaussian Models

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Fitting a linear model in RGaussian

    Linear ModelYi = 0 + 1xi + i i Norm(0, 2)

    R script

    N = 100sd=0.25x1

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Normal Linear ModelsGaussian

    Linear ModelYi Normal(i , 2) Stochastic parti = 0 + 1x1+ + kxk Deterministic part

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 4/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Normal Linear ModelsGaussian

    Linear ModelYi Normal(i , 2)i = 0 + 1x1 + 2x21

    R script

    x1

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Normal Linear ModelsGaussian

    Linear ModelYi Normal(i , 2)i = 0 + 1x1 + 2x2

    R script

    mu

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Normal Linear ModelsGaussian

    Linear ModelYi Normal(i , 2)i = 0 + 1x1 + 2x2+3x21 + 4x

    22

    R script

    mu

  • The Bayesian way

    p(|y) = p()p(y |)p(y)

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    JAGS: Just Another Gibbs Samplerhttp://mcmc-jags.sourceforge.net

    A program for analysis of Bayesian hierarchical modelsusing Markov Chain Monte Carlo (MCMC) simulationwritten with three aims in mind:

    To have a cross-platform engine for the BUGS (Bayesianinference Using Gibbs Sampling) languageTo be extensible, allowing users to write their ownfunctions, distributions and samplers.To be a platform for experimentation with ideas inBayesian modelling

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 9/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    JAGS: Just Another Gibbs Sampler

    An intuitive program for analysis of Bayesian hierarchicalmodels using MCMC.

    Normal Model JAGS syntax

    Likelihood

    Yi N (i , 2) Y [i] dnorm(mu[i],pow(sigma,2)i = 1 + 2 Xi mu[i] < inprod(beta[],X [i , ])

    Priorsi N (0,104) beta[i] dnorm(0,0.001) U(0,100) sigma dunif (0,100)

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 10/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Synthetic Normal model with JAGS

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 11/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Gaussian ModelsErrors-in-measurements

    Normal ModelLikelihood

    Yobs;i N (Ytrue;i, 2Y )Xobs;i N (Xtrue;i, 2X )Ytrue;i N (i , 2)i = 1 + 2 Xtrue;iPriorsXtrue;i N (0,102)1 N (0,104)2 N (0,104) U(0,100)

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 12/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Astronomical application of Normal modelHubble residuals-data from Wolf, R. et al. arXiv:1602.02674

    LINMIXJAGS

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 13/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Astronomical examples of non-Gaussian data

    Gaussian models are powerful, but fall short in manysimple applications.

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 14/34

  • Beyond Gaussian

    Generalized Linear Models

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Beyond Normal Linear RegressionThe tip of the iceberg

    Non-exhaustive list of Regression Models:Linear/Gaussian models: x R

    Generalized linear modelsGamma: x R>0Log-normal: x R>0Poisson: x Z0Negative-binomial: x Z0Beta: {0 < x < 1}Beta-binomial: {0 < x < 1}Bernoulli: x = 0 or x = 1

    ...

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 16/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Generalized Linear ModelsBeyond Gaussian

    Generalized linear model is a natural extension of theGaussian linear regression that accounts for a moregeneral family of statistical distributions.

    Linear ModelYi Normal(i , 2)i = 0 + 1x1 + + kxk

    Generalized Linear ModelsY f (i ,a()V ())g() = = 0 + 1x1 + + kxk

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 17/34

  • Chapter II

    Bernoulli Models

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Bernoulli Model

    Describes a random variable whichtakes the value 1 with successprobability of p and 0 with failureprobability of 1 p.

    Logistic Model

    Yi Bernoulli(p) py (1 p)1ylog p1p = = 0 + 1x1 + + kxk

    Figure: Coin toss

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 19/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Bernoulli data

    Figure: 3D visualization of a Bernoulli distributed data. Redpoints are the actual data and grey curve the fit.Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 20/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Normal vs Bernoulli model

    Figure: Linear regression predicts fractional (or probability)values outside the [0,1] range.

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 21/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Bernoulli modelBivariate

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 22/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Bernoulli model in JAGS

    Bernoulli ModelYi Bern(pi)log( pi1pi ) = ii = 0 + 1Xi

    JAGSY [i] Bern(p[i])logit(p[i]) = [i][i] = [1] + [2] X [i]

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 23/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Astronomical application of Bernoulli modelStar formation activity in early Universe, data from Biffi and Maio,arXiv:1309.2283

    Bernoulli ModelSFR Bern(pi)log( pi1pi ) = ii = 0 + 1 log xmol+2(log xmol)2 +3(log xmol)3

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 24/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Astronomical application of Bernoulli modelStar formation activity in early Universe, data from Biffi and Maio,arXiv:1309.2283, see also de Souza et al. arxiv.org/abs/1409.7696

    Bernoulli ModelSFR Bern(pi)log( pi1pi ) = ii = 0 + 1 log xmol+2(log xmol)2 +3(log xmol)3

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 25/34

    arxiv.org/abs/1409.7696

  • Chapter III

    Count Models

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Poisson ModelsNumber of events

    Figure: 3D visualization of a Poisson distributed data.

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 27/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Poisson ModelsNumber of events

    The Poisson distribution model the number of times anevent occurs in an interval of time or space.

    Linear ModelsY Normal(, 2) = 0 + 1x1 + + kxk

    Poisson ModelY Poisson()

    yey!

    log() = = 0 + 1x1 + + kxk

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 28/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Poisson modelMean equals variance

    Although traditional, it is rarely useful in real situations.

    Figure: Poisson distributed data

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 29/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Negative binomial modelVariance exceeds the sample mean

    The NB distribution can be seen as a Poisson distribution,where the mean is itself a random variable, distributed asa gamma distribution.

    Negative Binomialdistribution

    (y+)()(y+1)

    (

    +

    ) (1 +

    )yMean = Var = +

    2

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 30/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Negative Binomial modelSynthetic data

    Accounts for Poisson over-dispersion

    Figure: Negative binomial distributed data

    Rafael S. de Souza Bayesian Models for Astronomy May 24, 2016 31/34

  • GaussianModels

    GLMs

    Bernoulli Models

    COUNT Models

    Final Remarks

    References

    Astronomical application of COUNT modelsGl