     • date post

18-Mar-2018
• Category

## Documents

• view

220

3

Embed Size (px)

### Transcript of Inverse Problems: From Regularization to Bayesian Problems: From Regularization to Bayesian...

• Inverse Problems:From Regularization to Bayesian Inference

An Overview onPrior Modeling and Bayesian Computation

Application to Computed Tomography

Groupe Problemes InversesLaboratoire des Signaux et Systemes

UMR 8506 CNRS - SUPELEC - Univ Paris Sud 11Supelec, Plateau de Moulon, 91192 Gif-sur-Yvette, FRANCE.

djafari@lss.supelec.frhttp://djafari.free.fr

http://www.lss.supelec.fr

Applied Math Colloquium, UCLA, USA, July 17, 2009Organizer: Stanley Osher

1 / 42

• Content Image reconstruction in Computed Tomography :

An ill posed invers problem Two main steps in Bayesian approach :

Prior modeling and Bayesian computation Prior models for images :

Separable Gaussian, GG, ... Gauss-Markov, General one layer Markovian models Hierarchical Markovian models with hidden variables

(contours and regions) Gauss-Markov-Potts

Bayesian computation MCMC Variational and Mean Field approximations (VBA, MFA)

Application : Computed Tomography in NDT Conclusions and Work in Progress Questions and Discussion

2 / 42

• Computed Tomography :Making an image of the interior of a body

f (x , y) a section of a real 3D body f (x , y , z) g(r) a line of observed radiographe g(r , z)

Forward model :Line integrals or Radon Transform

g(r) =

Lr ,f (x , y) dl + (r)

=

f (x , y) (r x cos y sin ) dx dy + (r)

Inverse problem : Image reconstruction

Given the forward model H (Radon Transform) anda set of data gi (r), i = 1, , Mfind f (x , y)

3 / 42

• 2D and 3D Computed Tomography

3D 2D

80 60 40 20 0 20 40 60 80

80

60

40

20

0

20

40

60

80

f(x,y)

x

y

Projections

g(r1, r2) =

Lr1,r2,

f (x , y , z) dl g(r) =

Lr ,

f (x , y) dl

Forward probelm : f (x , y) or f (x , y , z) g(r) or g(r1, r2)Inverse problem : g(r) or g(r1, r2) f (x , y) or f (x , y , z)

4 / 42

• X ray Tomography

f(x,y)

x

y

150 100 50 0 50 100 150

150

100

50

0

50

100

150

g(r , ) = ln(

II0

)=

Lr,

f (x , y) dl

g(r , ) =

Df (x , y) (r x cos y sin ) dx dy

f (x , y)- RT -g(r , )

phi

r

p(r,phi)

0

45

90

135

180

225

270

315

IRT?

=60 40 20 0 20 40 60

60

40

20

0

20

40

60

5 / 42

• Analytical Inversion methods

f (x , y)

-x

6y

@@@

@@

HHH

r

Dg(r , ) =

L f (x , y) dl

S

@@

@@

@@

@@

@@

@@

g(r , ) =

Df (x , y) (r x cos y sin ) dx dy

f (x , y) =(

122

)

0

+

r g(r , )

(r x cos y sin )dr d

6 / 42

• Filtered Backprojection method

f (x , y) =(

122

)

0

+

r g(r , )

(r x cos y sin )dr d

Derivation D : g(r , ) =g(r , )

r

Hilbert TransformH : g1(r, ) =

1

0

g(r , )(r r )

dr

Backprojection B : f (x , y) =1

2

0g1(r

= x cos + y sin , ) d

f (x , y) = B HD g(r , ) = B F11 || F1 g(r , )

Backprojection of filtered projections :

g(r ,)

FTF1

Filter||

IFTF11

g1(r ,)

BackprojectionB

f (x,y)

7 / 42

• Limitations : Limited angle or noisy data

60 40 20 0 20 40 60

60

40

20

0

20

40

60

60 40 20 0 20 40 60

60

40

20

0

20

40

60

60 40 20 0 20 40 60

60

40

20

0

20

40

60

60 40 20 0 20 40 60

60

40

20

0

20

40

60

Original 64 proj. 16 proj. 8 proj. [0, /2]

Limited angle or noisy data Accounting for detector size Other measurement geometries : fan beam, ...

8 / 42

• CT as a linear inverse problem

Source positions Detector positions

Fan beam Xray Tomography

1 0.5 0 0.5 1

1

0.5

0

0.5

1

g(si) =

Lif (r) dli + (si) Discretization g = Hf +

g, f and H are huge dimensional9 / 42

• Inversion : Deterministic methodsData matching

Observation modelgi = hi(f) + i , i = 1, . . . , M g = H(f ) +

Misatch between data and output of the model (g,H(f ))

f = arg minf

{(g,H(f ))}

Examples :

LS (g,H(f )) = g H(f )2 =

i

|gi hi(f)|2

Lp (g,H(f )) = g H(f )p =

i

|gi hi(f)|p , 1 < p < 2

KL (g,H(f )) =

i

gi lngi

hi(f)

In general, does not give satisfactory results for inverseproblems.

10 / 42

• Inversion : Regularization theoryInverse problems = Ill posed problems

Need for prior informationFunctional space (Tikhonov) :

g = H(f ) + J(f ) = ||g H(f )||22 + ||Df ||22

Finite dimensional space (Philips & Towmey) : g = H(f) + Minimum norme LS (MNLS) : J(f ) = ||g H(f )||2 + ||f ||2

Classical regularization : J(f ) = ||g H(f )||2 + ||Df ||2

More general regularization :

J(f ) = Q(g H(f )) + (Df)or

J(f) = 1(g,H(f )) + 2(f ,f)Limitations : Errors are considered implicitly white and Gaussian Limited prior information on the solution Lack of tools for the determination of the hyperparameters

11 / 42

• Bayesian estimation approachM : g = Hf +

Observation model M + Hypothesis on the noise p(g|f ;M) = p(g Hf)

A priori information p(f |M)

Bayes : p(f |g;M) =p(g|f ;M) p(f |M)

p(g|M)

Maximum A Posteriori (MAP) :

f = arg maxf

{p(f |g)} = arg maxf

{p(g|f) p(f)}

= arg minf

{ ln p(g|f) ln p(f)}

with Q(g,Hf ) = ln p(g|f) and (f) = ln p(f)But, Bayesian inference is not only limited to MAP

12 / 42

• Case of linear models and Gaussian priorsg = Hf +

Hypothesis on the noise : N (0, 2 I)

p(g|f) exp{ 122

g Hf2}

Hypothesis on f : f N (0, 2f (DtD)1)

p(f) exp{ 1

22fDf2

}

A posteriori :

p(f |g) exp{ 122

g Hf2 122f

Df2}

MAP : f = arg maxf {p(f |g)} = arg minf {J(f )}

with J(f ) = g Hf2 + Df2, = 2

2f

Advantage : characterization of the solution

f |g N (f , P ) with f = PH tg, P =(H tH + DtD

)1

13 / 42

• MAP estimation with other priors :

f = arg minf

{J(f )} avec J(f ) = g Hf2 + (f)

Separable priors : Gaussian : p(fj) exp

{|fj |2

} (f) =

j |fj |

2

Gamma : p(fj) f j exp{fj

} (f) =

j ln fj + fj

Beta :p(fj) f j (1 fj)

(f) =

j ln fj +

j ln(1 fj) Generalized Gaussian :

p(fj) exp{|fj |p

}, 1 < p < 2 (f) =

j |fj |

p,

Markovian models :

p(fj |f) exp

iNj

(fj , fi )

(f) =

j

iNj

(fj , fi),

14 / 42

• MAP estimation with markovien priors

f = arg minf

{J(f )} with J(f) = g Hf2 + (f)

(f) =

j

(fj fj1)

with (t) :

Convex functions :

|t |,

1 + t2 1, log(cosh(t)),{

t2 |t | T2T |t | T 2 |t | > T

or Non convex functions :

log(1 + t2),t2

1 + t2, arctan(t2),

{t2 |t | TT 2 |t | > T

15 / 42

• MAP estimation with different prior models

g = Hf + , N (0, 2 I)f = Cf + z, z N (0, 2f I)f = (I C)1z = Dz

p(g|f) = N (Hf , 2 I)p(f) = N (0, 2f (D

tD)1)

p(f |g) = N (f , Pf )f = arg minf {J(f )}J(f ) = g Hf2 + Df2

f = PH tg

Pf =(H tH + DtD

)1

g = Hf + , N (0, 2 I)f = Wz, z N (0, 2zI)g = HWz +

p(g|z) = N (HWz, 2 I)p(z) = N (0, 2zI)

p(z|g) = N (z, Pz)z = arg minz {J(z)}J(z) = g HWz2 + z2

z = PzHtW tg

Pz =(H tW tH + I

)1

f = Wz

Pf = WPzWt

16 / 42

• MAP estimation with different prior models

{g = Hf + f N

(0, 2f (D

tD)1)) =

g = Hf +

f = Cf + z with z N (0, 2f I)Df = z with D = (I C)

f |g N (f , Pf ) with f = Pf Htg, Pf =

(H tH + DtD

)1

J(f ) = ln p(f |g) = g Hf2 + Df2

{

g = Hf + f N

(0, 2f (WW

t)) =

{g = Hf + f = Wz with z N (0, 2f I)

z|g N (z, Pz) with z = PzW tH tg, Pz =(W tH tHW + I

)1

J(z) = ln p(z|g) = g HWz2 + z2 f = Wz

z decomposqition coefficients

17 / 42

• MAP estimation and Compressed Sensing

{g = Hf + f = Wz

W a code book matrix, z coefficients Gaussian :

p(z) = N (0, 2zI) exp{ 1

22z

j |z j |

2}

J(z) = ln p(z|g) = g HWz2 +

j |z j |2

Generalized Gaussian (sparsity, = 1) :

p(z) exp{

j |z j |

}

J(z) = ln p(z|g) = g HWz2 +

j |z j |

z = arg minz {J(z)} f = Wz

18 / 42

• Main advantages of the Bayesian approach

MAP = Regularization Posterior mean ? Marginal MAP ? More information in the posterior law than only its mode or

its mean Meaning and tools for estimating hyper parameters Meaning and tools for model selection More specific and specialized priors, particularly through

the hidden variables More computational tools :

Expectation-Maximization for computing the maximumlikelihood parameters

MCMC for posterior exploration Variational Bayes for analytical computation of the posterior

marginals ...

19 / 42

• Full Bayesian approachM : g = Hf +