Inverse Problems: From Regularization to Bayesian Problems: From Regularization to Bayesian...

download Inverse Problems: From Regularization to Bayesian Problems: From Regularization to Bayesian Inference ... g(r,†) = −ln I I0 = Z Lr,† ... Inverse Problems: From Regularization

of 42

  • date post

    18-Mar-2018
  • Category

    Documents

  • view

    220
  • download

    3

Embed Size (px)

Transcript of Inverse Problems: From Regularization to Bayesian Problems: From Regularization to Bayesian...

  • Inverse Problems:From Regularization to Bayesian Inference

    An Overview onPrior Modeling and Bayesian Computation

    Application to Computed Tomography

    Ali Mohammad-Djafari

    Groupe Problemes InversesLaboratoire des Signaux et Systemes

    UMR 8506 CNRS - SUPELEC - Univ Paris Sud 11Supelec, Plateau de Moulon, 91192 Gif-sur-Yvette, FRANCE.

    djafari@lss.supelec.frhttp://djafari.free.fr

    http://www.lss.supelec.fr

    Applied Math Colloquium, UCLA, USA, July 17, 2009Organizer: Stanley Osher

    1 / 42

  • Content Image reconstruction in Computed Tomography :

    An ill posed invers problem Two main steps in Bayesian approach :

    Prior modeling and Bayesian computation Prior models for images :

    Separable Gaussian, GG, ... Gauss-Markov, General one layer Markovian models Hierarchical Markovian models with hidden variables

    (contours and regions) Gauss-Markov-Potts

    Bayesian computation MCMC Variational and Mean Field approximations (VBA, MFA)

    Application : Computed Tomography in NDT Conclusions and Work in Progress Questions and Discussion

    2 / 42

  • Computed Tomography :Making an image of the interior of a body

    f (x , y) a section of a real 3D body f (x , y , z) g(r) a line of observed radiographe g(r , z)

    Forward model :Line integrals or Radon Transform

    g(r) =

    Lr ,f (x , y) dl + (r)

    =

    f (x , y) (r x cos y sin ) dx dy + (r)

    Inverse problem : Image reconstruction

    Given the forward model H (Radon Transform) anda set of data gi (r), i = 1, , Mfind f (x , y)

    3 / 42

  • 2D and 3D Computed Tomography

    3D 2D

    80 60 40 20 0 20 40 60 80

    80

    60

    40

    20

    0

    20

    40

    60

    80

    f(x,y)

    x

    y

    Projections

    g(r1, r2) =

    Lr1,r2,

    f (x , y , z) dl g(r) =

    Lr ,

    f (x , y) dl

    Forward probelm : f (x , y) or f (x , y , z) g(r) or g(r1, r2)Inverse problem : g(r) or g(r1, r2) f (x , y) or f (x , y , z)

    4 / 42

  • X ray Tomography

    f(x,y)

    x

    y

    150 100 50 0 50 100 150

    150

    100

    50

    0

    50

    100

    150

    g(r , ) = ln(

    II0

    )=

    Lr,

    f (x , y) dl

    g(r , ) =

    Df (x , y) (r x cos y sin ) dx dy

    f (x , y)- RT -g(r , )

    phi

    r

    p(r,phi)

    0

    45

    90

    135

    180

    225

    270

    315

    IRT?

    =60 40 20 0 20 40 60

    60

    40

    20

    0

    20

    40

    60

    5 / 42

  • Analytical Inversion methods

    f (x , y)

    -x

    6y

    @@@

    @@

    HHH

    r

    Dg(r , ) =

    L f (x , y) dl

    S

    @@

    @@

    @@

    @@

    @@

    @@

    Radon :

    g(r , ) =

    Df (x , y) (r x cos y sin ) dx dy

    f (x , y) =(

    122

    )

    0

    +

    r g(r , )

    (r x cos y sin )dr d

    6 / 42

  • Filtered Backprojection method

    f (x , y) =(

    122

    )

    0

    +

    r g(r , )

    (r x cos y sin )dr d

    Derivation D : g(r , ) =g(r , )

    r

    Hilbert TransformH : g1(r, ) =

    1

    0

    g(r , )(r r )

    dr

    Backprojection B : f (x , y) =1

    2

    0g1(r

    = x cos + y sin , ) d

    f (x , y) = B HD g(r , ) = B F11 || F1 g(r , )

    Backprojection of filtered projections :

    g(r ,)

    FTF1

    Filter||

    IFTF11

    g1(r ,)

    BackprojectionB

    f (x,y)

    7 / 42

  • Limitations : Limited angle or noisy data

    60 40 20 0 20 40 60

    60

    40

    20

    0

    20

    40

    60

    60 40 20 0 20 40 60

    60

    40

    20

    0

    20

    40

    60

    60 40 20 0 20 40 60

    60

    40

    20

    0

    20

    40

    60

    60 40 20 0 20 40 60

    60

    40

    20

    0

    20

    40

    60

    Original 64 proj. 16 proj. 8 proj. [0, /2]

    Limited angle or noisy data Accounting for detector size Other measurement geometries : fan beam, ...

    8 / 42

  • CT as a linear inverse problem

    Source positions Detector positions

    Fan beam Xray Tomography

    1 0.5 0 0.5 1

    1

    0.5

    0

    0.5

    1

    g(si) =

    Lif (r) dli + (si) Discretization g = Hf +

    g, f and H are huge dimensional9 / 42

  • Inversion : Deterministic methodsData matching

    Observation modelgi = hi(f) + i , i = 1, . . . , M g = H(f ) +

    Misatch between data and output of the model (g,H(f ))

    f = arg minf

    {(g,H(f ))}

    Examples :

    LS (g,H(f )) = g H(f )2 =

    i

    |gi hi(f)|2

    Lp (g,H(f )) = g H(f )p =

    i

    |gi hi(f)|p , 1 < p < 2

    KL (g,H(f )) =

    i

    gi lngi

    hi(f)

    In general, does not give satisfactory results for inverseproblems.

    10 / 42

  • Inversion : Regularization theoryInverse problems = Ill posed problems

    Need for prior informationFunctional space (Tikhonov) :

    g = H(f ) + J(f ) = ||g H(f )||22 + ||Df ||22

    Finite dimensional space (Philips & Towmey) : g = H(f) + Minimum norme LS (MNLS) : J(f ) = ||g H(f )||2 + ||f ||2

    Classical regularization : J(f ) = ||g H(f )||2 + ||Df ||2

    More general regularization :

    J(f ) = Q(g H(f )) + (Df)or

    J(f) = 1(g,H(f )) + 2(f ,f)Limitations : Errors are considered implicitly white and Gaussian Limited prior information on the solution Lack of tools for the determination of the hyperparameters

    11 / 42

  • Bayesian estimation approachM : g = Hf +

    Observation model M + Hypothesis on the noise p(g|f ;M) = p(g Hf)

    A priori information p(f |M)

    Bayes : p(f |g;M) =p(g|f ;M) p(f |M)

    p(g|M)

    Link with regularization :

    Maximum A Posteriori (MAP) :

    f = arg maxf

    {p(f |g)} = arg maxf

    {p(g|f) p(f)}

    = arg minf

    { ln p(g|f) ln p(f)}

    with Q(g,Hf ) = ln p(g|f) and (f) = ln p(f)But, Bayesian inference is not only limited to MAP

    12 / 42

  • Case of linear models and Gaussian priorsg = Hf +

    Hypothesis on the noise : N (0, 2 I)

    p(g|f) exp{ 122

    g Hf2}

    Hypothesis on f : f N (0, 2f (DtD)1)

    p(f) exp{ 1

    22fDf2

    }

    A posteriori :

    p(f |g) exp{ 122

    g Hf2 122f

    Df2}

    MAP : f = arg maxf {p(f |g)} = arg minf {J(f )}

    with J(f ) = g Hf2 + Df2, = 2

    2f

    Advantage : characterization of the solution

    f |g N (f , P ) with f = PH tg, P =(H tH + DtD

    )1

    13 / 42

  • MAP estimation with other priors :

    f = arg minf

    {J(f )} avec J(f ) = g Hf2 + (f)

    Separable priors : Gaussian : p(fj) exp

    {|fj |2

    } (f) =

    j |fj |

    2

    Gamma : p(fj) f j exp{fj

    } (f) =

    j ln fj + fj

    Beta :p(fj) f j (1 fj)

    (f) =

    j ln fj +

    j ln(1 fj) Generalized Gaussian :

    p(fj) exp{|fj |p

    }, 1 < p < 2 (f) =

    j |fj |

    p,

    Markovian models :

    p(fj |f) exp

    iNj

    (fj , fi )

    (f) =

    j

    iNj

    (fj , fi),

    14 / 42

  • MAP estimation with markovien priors

    f = arg minf

    {J(f )} with J(f) = g Hf2 + (f)

    (f) =

    j

    (fj fj1)

    with (t) :

    Convex functions :

    |t |,

    1 + t2 1, log(cosh(t)),{

    t2 |t | T2T |t | T 2 |t | > T

    or Non convex functions :

    log(1 + t2),t2

    1 + t2, arctan(t2),

    {t2 |t | TT 2 |t | > T

    15 / 42

  • MAP estimation with different prior models

    g = Hf + , N (0, 2 I)f = Cf + z, z N (0, 2f I)f = (I C)1z = Dz

    p(g|f) = N (Hf , 2 I)p(f) = N (0, 2f (D

    tD)1)

    p(f |g) = N (f , Pf )f = arg minf {J(f )}J(f ) = g Hf2 + Df2

    f = PH tg

    Pf =(H tH + DtD

    )1

    g = Hf + , N (0, 2 I)f = Wz, z N (0, 2zI)g = HWz +

    p(g|z) = N (HWz, 2 I)p(z) = N (0, 2zI)

    p(z|g) = N (z, Pz)z = arg minz {J(z)}J(z) = g HWz2 + z2

    z = PzHtW tg

    Pz =(H tW tH + I

    )1

    f = Wz

    Pf = WPzWt

    16 / 42

  • MAP estimation with different prior models

    {g = Hf + f N

    (0, 2f (D

    tD)1)) =

    g = Hf +

    f = Cf + z with z N (0, 2f I)Df = z with D = (I C)

    f |g N (f , Pf ) with f = Pf Htg, Pf =

    (H tH + DtD

    )1

    J(f ) = ln p(f |g) = g Hf2 + Df2

    {

    g = Hf + f N

    (0, 2f (WW

    t)) =

    {g = Hf + f = Wz with z N (0, 2f I)

    z|g N (z, Pz) with z = PzW tH tg, Pz =(W tH tHW + I

    )1

    J(z) = ln p(z|g) = g HWz2 + z2 f = Wz

    z decomposqition coefficients

    17 / 42

  • MAP estimation and Compressed Sensing

    {g = Hf + f = Wz

    W a code book matrix, z coefficients Gaussian :

    p(z) = N (0, 2zI) exp{ 1

    22z

    j |z j |

    2}

    J(z) = ln p(z|g) = g HWz2 +

    j |z j |2

    Generalized Gaussian (sparsity, = 1) :

    p(z) exp{

    j |z j |

    }

    J(z) = ln p(z|g) = g HWz2 +

    j |z j |

    z = arg minz {J(z)} f = Wz

    18 / 42

  • Main advantages of the Bayesian approach

    MAP = Regularization Posterior mean ? Marginal MAP ? More information in the posterior law than only its mode or

    its mean Meaning and tools for estimating hyper parameters Meaning and tools for model selection More specific and specialized priors, particularly through

    the hidden variables More computational tools :

    Expectation-Maximization for computing the maximumlikelihood parameters

    MCMC for posterior exploration Variational Bayes for analytical computation of the posterior

    marginals ...

    19 / 42

  • Full Bayesian approachM : g = Hf +