f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah...

128
What is an image ? Digital images are sampled 2-D analogue signals Black and white images f R 2 R f (x ) intensity level at that point, which varies from zero to 255 An image can be postulated as an L 2 (Ω) object (a) (b) Figure: (a) Image of Lenna and (b) Image of Lenna as a graph of a function

Transcript of f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah...

Page 1: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

What is an image ?

Digital images are sampled 2-D analogue signals

Black and white images ≡ f : Ω ⊂ R2 → Rf (x) ≡ intensity level at that point, which varies from zero to 255

An image can be postulated as an L2(Ω) object

(a) (b)

Figure: (a) Image of Lenna and (b) Image of Lenna as a graph of a function

Page 2: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

What is an image ?

Digital images are sampled 2-D analogue signals

Black and white images ≡ f : Ω ⊂ R2 → Rf (x) ≡ intensity level at that point, which varies from zero to 255

An image can be postulated as an L2(Ω) object

(a) (b)

Figure: (a) Image of Lenna and (b) Image of Lenna as a graph of a function

Page 3: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

What is an image ?

Digital images are sampled 2-D analogue signals

Black and white images ≡ f : Ω ⊂ R2 → Rf (x) ≡ intensity level at that point, which varies from zero to 255

An image can be postulated as an L2(Ω) object

(a) (b)

Figure: (a) Image of Lenna and (b) Image of Lenna as a graph of a function

Page 4: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

What is an image ?

Digital images are sampled 2-D analogue signals

Black and white images ≡ f : Ω ⊂ R2 → Rf (x) ≡ intensity level at that point, which varies from zero to 255

An image can be postulated as an L2(Ω) object

(a) (b)

Figure: (a) Image of Lenna and (b) Image of Lenna as a graph of a function

Page 5: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Problems in image processing

Image deblurringf = TU for a deblurring operator T : L2(Ω)→ L2(Ω)T may not be invertible : ill-posed problem.

Given f we need to get back the deblurred image U.

(a) (b)

Figure: Can we go from a blurred image (a) to a restored image in (b) ?

Page 6: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Problems in image processing

Image deblurringf = TU for a deblurring operator T : L2(Ω)→ L2(Ω)T may not be invertible : ill-posed problem.

Given f we need to get back the deblurred image U.

(a) (b)

Figure: Can we go from a blurred image (a) to a restored image in (b) ?

Page 7: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Problems in image processing...

Image denoising: f may have some noise η in it.

f = U + η, we need to get back the denoised image U.

(a) (b)

Figure: Can we go from a noisy image (a) to a restored image in (b) ?

f may be blurry and noisy f = TU + η

Page 8: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Problems in image processing...

Image segmentation ≡ identifying ‘components’ in f ≡ edge detection

(a) (b)

Figure: Can we identify components in (a) and get a segmented image as in (b) ?

Page 9: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Problems in image processing...

Multiscale image representation: Finding different level of ‘scales’ in f

(a) (b) (c)

Figure: Multiscale images of the city of Mumbai.

Multiscale representation: Family of images U(t) for a scalingparameter t

Forward marching: U(0) = 0,U(t)→ U

Backward marching: U(0) = f ,U(t)→ U

Page 10: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Problems in image processing...

Multiscale image representation: Finding different level of ‘scales’ in f

(a) (b) (c)

Figure: Multiscale images of the city of Mumbai.

Multiscale representation: Family of images U(t) for a scalingparameter t

Forward marching: U(0) = 0,U(t)→ U

Backward marching: U(0) = f ,U(t)→ U

Page 11: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Problems in image processing...

Multiscale image representation: Finding different level of ‘scales’ in f

(a) (b) (c)

Figure: Multiscale images of the city of Mumbai.

Multiscale representation: Family of images U(t) for a scalingparameter t

Forward marching: U(0) = 0,U(t)→ U

Backward marching: U(0) = f ,U(t)→ U

Page 12: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

There are two main approaches to solve above problems:

Variational approaches - Tikhonov regularization, greedy algorithms,wavelets shrinkage etc.

PDE based approaches - diffusion, Perona-Malik etc.

The approaches are related -

Page 13: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Variational methods in image processing: Tikhonov regularization

We need to solve the ill posed problem f = Tu :

Consider interpolation functional

infu∈X

(‖uλ‖X + λ‖f − Tuλ‖2

Y

)X ⊂ Y

‖u‖X : regularizing term

‖f − Tu‖2Y : fidelity term

(X ,Y ) ≡ (BV , L2): Rudin-Osher-Fatemi-Vese.

inff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − Tuλ|2)

Page 14: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Variational methods in image processing: Tikhonov regularization

We need to solve the ill posed problem f = Tu :

Consider interpolation functional

infu∈X

(‖uλ‖X + λ‖f − Tuλ‖2

Y

)X ⊂ Y

‖u‖X : regularizing term

‖f − Tu‖2Y : fidelity term

(X ,Y ) ≡ (BV , L2): Rudin-Osher-Fatemi-Vese.

inff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − Tuλ|2)

Page 15: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Variational methods in image processing

Rudin-Osher-Fatemi (ROF) decompositionf = uλ + vλ for scale parameter λ.

[uλ, vλ] = arginff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − uλ|2)

The BV norm∫

Ω|∇uλ| is a regularizing term∫

Ω|f − uλ|2: a fidelity term

λ : acts as an inverse scale of the uλ part ( smaller λ ≡ larger scale )

uλ := smooth parts and edges in fvλ := f − uλ texture, finer details, noise

Many other variational methods ...

Page 16: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Variational methods in image processing

Rudin-Osher-Fatemi (ROF) decompositionf = uλ + vλ for scale parameter λ.

[uλ, vλ] = arginff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − uλ|2)

The BV norm∫

Ω|∇uλ| is a regularizing term∫

Ω|f − uλ|2: a fidelity term

λ : acts as an inverse scale of the uλ part ( smaller λ ≡ larger scale )

uλ := smooth parts and edges in fvλ := f − uλ texture, finer details, noise

Many other variational methods ...

Page 17: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Variational methods in image processing

Rudin-Osher-Fatemi (ROF) decompositionf = uλ + vλ for scale parameter λ.

[uλ, vλ] = arginff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − uλ|2)

The BV norm∫

Ω|∇uλ| is a regularizing term∫

Ω|f − uλ|2: a fidelity term

λ : acts as an inverse scale of the uλ part ( smaller λ ≡ larger scale )

uλ := smooth parts and edges in fvλ := f − uλ texture, finer details, noise

Many other variational methods ...

Page 18: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Variational methods in image processing

Rudin-Osher-Fatemi (ROF) decompositionf = uλ + vλ for scale parameter λ.

[uλ, vλ] = arginff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − uλ|2)

The BV norm∫

Ω|∇uλ| is a regularizing term∫

Ω|f − uλ|2: a fidelity term

λ : acts as an inverse scale of the uλ part ( smaller λ ≡ larger scale )

uλ := smooth parts and edges in fvλ := f − uλ texture, finer details, noise

Many other variational methods ...

Page 19: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other variational methods in image processing...

Mumford-Shah segmentation (1985)

[u, v , C] = arginff =u+v,C

(∫Ω−C|f − u|2 + λ1

∫Ω−C|∇u|2 + λ2

∮C

dσ).

u : Ω→ R : piecewise smooth imageC ∈ Ω : the set of jump discontinuities

Ambrosio and Tortorelli approximation (1992)Kass-Witkin-Terzopoulos model (1988)

infc∈C

(∫ b

a|c′|2 + λ1

∫ b

a|c′′|2 + λ2

∫ b

ag2(|∇f (c)|)

)C : closed, piecewise regular, parametric curves (snakes)g : a decreasing function vanishing at infinity

Caselles, Kimmel, Sapiro: Geodesic active contours (1997)Osher, Sethian: Level set method (1988)... ...

Now we look at some PDE methods ...

Page 20: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other variational methods in image processing...

Mumford-Shah segmentation (1985)

[u, v , C] = arginff =u+v,C

(∫Ω−C|f − u|2 + λ1

∫Ω−C|∇u|2 + λ2

∮C

dσ).

u : Ω→ R : piecewise smooth imageC ∈ Ω : the set of jump discontinuities

Ambrosio and Tortorelli approximation (1992)Kass-Witkin-Terzopoulos model (1988)

infc∈C

(∫ b

a|c′|2 + λ1

∫ b

a|c′′|2 + λ2

∫ b

ag2(|∇f (c)|)

)C : closed, piecewise regular, parametric curves (snakes)g : a decreasing function vanishing at infinity

Caselles, Kimmel, Sapiro: Geodesic active contours (1997)Osher, Sethian: Level set method (1988)... ...

Now we look at some PDE methods ...

Page 21: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other variational methods in image processing...

Mumford-Shah segmentation (1985)

[u, v , C] = arginff =u+v,C

(∫Ω−C|f − u|2 + λ1

∫Ω−C|∇u|2 + λ2

∮C

dσ).

u : Ω→ R : piecewise smooth imageC ∈ Ω : the set of jump discontinuities

Ambrosio and Tortorelli approximation (1992)Kass-Witkin-Terzopoulos model (1988)

infc∈C

(∫ b

a|c′|2 + λ1

∫ b

a|c′′|2 + λ2

∫ b

ag2(|∇f (c)|)

)C : closed, piecewise regular, parametric curves (snakes)g : a decreasing function vanishing at infinity

Caselles, Kimmel, Sapiro: Geodesic active contours (1997)Osher, Sethian: Level set method (1988)... ...

Now we look at some PDE methods ...

Page 22: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other variational methods in image processing...

Mumford-Shah segmentation (1985)

[u, v , C] = arginff =u+v,C

(∫Ω−C|f − u|2 + λ1

∫Ω−C|∇u|2 + λ2

∮C

dσ).

u : Ω→ R : piecewise smooth imageC ∈ Ω : the set of jump discontinuities

Ambrosio and Tortorelli approximation (1992)Kass-Witkin-Terzopoulos model (1988)

infc∈C

(∫ b

a|c′|2 + λ1

∫ b

a|c′′|2 + λ2

∫ b

ag2(|∇f (c)|)

)C : closed, piecewise regular, parametric curves (snakes)g : a decreasing function vanishing at infinity

Caselles, Kimmel, Sapiro: Geodesic active contours (1997)Osher, Sethian: Level set method (1988)... ...

Now we look at some PDE methods ...

Page 23: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Heat equation

Heat equation ∂U∂t = ∆U. (Koenderink 1984, Witkin 1983)

Backward marching: U(0) = f and U(t)→ U.

This gives us Gaussian smoothing with variance=t

Forward scaling: Anything with scale smaller than t is smoothed outThus t acts as the scale parameter

Figure: Different scales in a carpet obtained by heat equation

Page 24: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Heat equation

Heat equation ∂U∂t = ∆U. (Koenderink 1984, Witkin 1983)

Backward marching: U(0) = f and U(t)→ U.

This gives us Gaussian smoothing with variance=t

Forward scaling: Anything with scale smaller than t is smoothed outThus t acts as the scale parameter

Figure: Different scales in a carpet obtained by heat equation

Page 25: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Heat equation

Heat equation ∂U∂t = ∆U. (Koenderink 1984, Witkin 1983)

Backward marching: U(0) = f and U(t)→ U.

This gives us Gaussian smoothing with variance=t

Forward scaling: Anything with scale smaller than t is smoothed outThus t acts as the scale parameter

Figure: Different scales in a carpet obtained by heat equation

Page 26: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Heat equation...

Denoising with heat equation:

(a) (b)

Figure: Result of isotropic diffusion: reduction of noise at the expense of losinginformation at the edges

Problem 1: cannot distinguish between noise and boundaries of regions

Problem 2: where to stop ?

Page 27: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Heat equation...

Denoising with heat equation:

(a) (b)

Figure: Result of isotropic diffusion: reduction of noise at the expense of losinginformation at the edges

Problem 1: cannot distinguish between noise and boundaries of regions

Problem 2: where to stop ?

Page 28: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Heat equation...

Denoising with heat equation:

(a) (b)

Figure: Result of isotropic diffusion: reduction of noise at the expense of losinginformation at the edges

Problem 1: cannot distinguish between noise and boundaries of regions

Problem 2: where to stop ?

Page 29: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Perona-Malik model (1988)

Heat equation ≡ isotropic diffusion⇒ we lose information about edges

Perona-Malik proposed an anisotropic diffusion method

∂U∂t

= div (g(|∇U|)∇U), U(0) = f

The idea: preserve the edgesSmooth regions ≡ |∇U| is weak⇒ we need an isotropic smoothingNear the edges ≡ |∇U| is large⇒ we need to control the diffusionExamples of suitable function g(s) : e−s, 1

1+s2 , 1√1+s

Perona-Malik is not well posed ! Catté et.al. modification2 :

∂U∂t

= div (g(|∇G ? U|)∇U),

G is Gaussian kernel.

2F. Catté, P-L. Lions, J-M. Morel, T. Coll (1992)

Page 30: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Perona-Malik model (1988)

Heat equation ≡ isotropic diffusion⇒ we lose information about edges

Perona-Malik proposed an anisotropic diffusion method

∂U∂t

= div (g(|∇U|)∇U), U(0) = f

The idea: preserve the edgesSmooth regions ≡ |∇U| is weak⇒ we need an isotropic smoothingNear the edges ≡ |∇U| is large⇒ we need to control the diffusionExamples of suitable function g(s) : e−s, 1

1+s2 , 1√1+s

Perona-Malik is not well posed ! Catté et.al. modification2 :

∂U∂t

= div (g(|∇G ? U|)∇U),

G is Gaussian kernel.

2F. Catté, P-L. Lions, J-M. Morel, T. Coll (1992)

Page 31: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Perona-Malik model (1988)

Heat equation ≡ isotropic diffusion⇒ we lose information about edges

Perona-Malik proposed an anisotropic diffusion method

∂U∂t

= div (g(|∇U|)∇U), U(0) = f

The idea: preserve the edgesSmooth regions ≡ |∇U| is weak⇒ we need an isotropic smoothingNear the edges ≡ |∇U| is large⇒ we need to control the diffusionExamples of suitable function g(s) : e−s, 1

1+s2 , 1√1+s

Perona-Malik is not well posed ! Catté et.al. modification2 :

∂U∂t

= div (g(|∇G ? U|)∇U),

G is Gaussian kernel.

2F. Catté, P-L. Lions, J-M. Morel, T. Coll (1992)

Page 32: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Perona-Malik model (1988)

Heat equation ≡ isotropic diffusion⇒ we lose information about edges

Perona-Malik proposed an anisotropic diffusion method

∂U∂t

= div (g(|∇U|)∇U), U(0) = f

The idea: preserve the edgesSmooth regions ≡ |∇U| is weak⇒ we need an isotropic smoothingNear the edges ≡ |∇U| is large⇒ we need to control the diffusionExamples of suitable function g(s) : e−s, 1

1+s2 , 1√1+s

Perona-Malik is not well posed ! Catté et.al. modification2 :

∂U∂t

= div (g(|∇G ? U|)∇U),

G is Gaussian kernel.

2F. Catté, P-L. Lions, J-M. Morel, T. Coll (1992)

Page 33: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Perona-Malik model (1988)

Heat equation ≡ isotropic diffusion⇒ we lose information about edges

Perona-Malik proposed an anisotropic diffusion method

∂U∂t

= div (g(|∇U|)∇U), U(0) = f

The idea: preserve the edgesSmooth regions ≡ |∇U| is weak⇒ we need an isotropic smoothingNear the edges ≡ |∇U| is large⇒ we need to control the diffusionExamples of suitable function g(s) : e−s, 1

1+s2 , 1√1+s

Perona-Malik is not well posed ! Catté et.al. modification2 :

∂U∂t

= div (g(|∇G ? U|)∇U),

G is Gaussian kernel.

2F. Catté, P-L. Lions, J-M. Morel, T. Coll (1992)

Page 34: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Perona-Malik model (1988)

Heat equation ≡ isotropic diffusion⇒ we lose information about edges

Perona-Malik proposed an anisotropic diffusion method

∂U∂t

= div (g(|∇U|)∇U), U(0) = f

The idea: preserve the edgesSmooth regions ≡ |∇U| is weak⇒ we need an isotropic smoothingNear the edges ≡ |∇U| is large⇒ we need to control the diffusionExamples of suitable function g(s) : e−s, 1

1+s2 , 1√1+s

Perona-Malik is not well posed ! Catté et.al. modification2 :

∂U∂t

= div (g(|∇G ? U|)∇U),

G is Gaussian kernel.

2F. Catté, P-L. Lions, J-M. Morel, T. Coll (1992)

Page 35: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Alvarez et. al.

L. Alvarez P-L. Lions and J-M Morel’s model (1992)

∂U∂t

= g(|G ?∇U|)|∇U| div(∇U|∇U|

), U(0) = f

Idea: Diffuse U only in the direction orthogonal to its gradient ∇U.

The term |∇U| div(∇U|∇U|

)does exactly this.

g is a diffusion controlling function as before.

(a) (b)

Figure: Result of anisotropic diffusion: edges are preserved.

Page 36: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Alvarez et. al.

L. Alvarez P-L. Lions and J-M Morel’s model (1992)

∂U∂t

= g(|G ?∇U|)|∇U| div(∇U|∇U|

), U(0) = f

Idea: Diffuse U only in the direction orthogonal to its gradient ∇U.

The term |∇U| div(∇U|∇U|

)does exactly this.

g is a diffusion controlling function as before.

(a) (b)

Figure: Result of anisotropic diffusion: edges are preserved.

Page 37: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Alvarez et. al.

L. Alvarez P-L. Lions and J-M Morel’s model (1992)

∂U∂t

= g(|G ?∇U|)|∇U| div(∇U|∇U|

), U(0) = f

Idea: Diffuse U only in the direction orthogonal to its gradient ∇U.

The term |∇U| div(∇U|∇U|

)does exactly this.

g is a diffusion controlling function as before.

(a) (b)

Figure: Result of anisotropic diffusion: edges are preserved.

Page 38: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Nordström’s model

Problem: As t →∞ the models discussed before diffuse completely.... so where to stop ?Solution: Nordström modified Perona-Malik model.

∂U∂t

= f − U + div (g(|∇U|)∇U), U(0) = 0.

This equation has non-trivial steady state.

Forward marching: U(0) = 0 and U(t)→ U.

Page 39: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Nordström’s model

Problem: As t →∞ the models discussed before diffuse completely.... so where to stop ?Solution: Nordström modified Perona-Malik model.

∂U∂t

= f − U + div (g(|∇U|)∇U), U(0) = 0.

This equation has non-trivial steady state.

Forward marching: U(0) = 0 and U(t)→ U.

Page 40: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE methods in image processing: Nordström’s model

Problem: As t →∞ the models discussed before diffuse completely.... so where to stop ?Solution: Nordström modified Perona-Malik model.

∂U∂t

= f − U + div (g(|∇U|)∇U), U(0) = 0.

This equation has non-trivial steady state.

Forward marching: U(0) = 0 and U(t)→ U.

Page 41: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE approach ! variational approach

Rudin-Osher-Fatemi decomposition

[uλ, vλ] = arginff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − uλ|2)

The Euler-Lagrange equation:

f − u +1

2λdiv(∇u|∇u|

)= 0.

Nordström’s modification of Perona-Malik:

∂u∂t

= f − u + div (g(|∇u|)∇u).

g(s) = 1λs ⇒ steady-state of Nordström ≡ Euler-Lagrange of ROF !

Let us look at our model now ...

Page 42: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE approach ! variational approach

Rudin-Osher-Fatemi decomposition

[uλ, vλ] = arginff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − uλ|2)

The Euler-Lagrange equation:

f − u +1

2λdiv(∇u|∇u|

)= 0.

Nordström’s modification of Perona-Malik:

∂u∂t

= f − u + div (g(|∇u|)∇u).

g(s) = 1λs ⇒ steady-state of Nordström ≡ Euler-Lagrange of ROF !

Let us look at our model now ...

Page 43: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE approach ! variational approach

Rudin-Osher-Fatemi decomposition

[uλ, vλ] = arginff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − uλ|2)

The Euler-Lagrange equation:

f − u +1

2λdiv(∇u|∇u|

)= 0.

Nordström’s modification of Perona-Malik:

∂u∂t

= f − u + div (g(|∇u|)∇u).

g(s) = 1λs ⇒ steady-state of Nordström ≡ Euler-Lagrange of ROF !

Let us look at our model now ...

Page 44: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

PDE approach ! variational approach

Rudin-Osher-Fatemi decomposition

[uλ, vλ] = arginff =uλ+vλ

(∫Ω

|∇uλ|+ λ

∫Ω

|f − uλ|2)

The Euler-Lagrange equation:

f − u +1

2λdiv(∇u|∇u|

)= 0.

Nordström’s modification of Perona-Malik:

∂u∂t

= f − u + div (g(|∇u|)∇u).

g(s) = 1λs ⇒ steady-state of Nordström ≡ Euler-Lagrange of ROF !

Let us look at our model now ...

Page 45: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 46: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 47: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 48: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 49: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 50: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 51: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 52: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 53: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A novel integro-differential model

We propose a novel model.

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

An Integro-differential equation.The scaling function λ(t) : increasing function at our disposal.This model gives an inverse scale representation.Compare this with Nordström’s model:

U(t) +∂U(t)∂t

= f +1

2λdiv(∇U(t)|∇U(t)|

).

? ? ? QUESTIONS ? ? ?

What is the motivation ?

Where to start ?

Where to stop ?

What does the scaling function λ(t) mean ??

Page 54: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Starting point: the idea of Tadmor-Nezzar-Vese (TNV) 2004, 2008

Recall ROF decomposition: f = uλ0 + vλ0 , where λ0 dictates the scale.

vλ0 can be decomposed with a scaling parameter λ1 > λ0.

vλ0 = uλ1 + vλ1 , [uλ1 , vλ1 ] = arginfvλ0

=uλ1+vλ1

(∫Ω

|∇uλ1 |+ λ1

∫Ω

|vλ0 − uλ1 |2).

TNV multiscale decomposition

vλk−1 = uλk + vλk ,[uλk , vλk

]= arginfvλk−1

=uλk+vλk

(∫Ω|∇uλk |+ λk

∫Ω|vk−1 − uk |2

).

With this scheme after N + 1 steps we get:

f = uλ0 + vλ0

= uλ0 + uλ1 + vλ1

= ...

= uλ0 + uλ1 + ...+ uλN + vλN .

i.e. a nonlinear multiscale decomposition: f ∼∑N

k=0 uλk + vλN .

Page 55: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Starting point: the idea of Tadmor-Nezzar-Vese (TNV) 2004, 2008

Recall ROF decomposition: f = uλ0 + vλ0 , where λ0 dictates the scale.

vλ0 can be decomposed with a scaling parameter λ1 > λ0.

vλ0 = uλ1 + vλ1 , [uλ1 , vλ1 ] = arginfvλ0

=uλ1+vλ1

(∫Ω

|∇uλ1 |+ λ1

∫Ω

|vλ0 − uλ1 |2).

TNV multiscale decomposition

vλk−1 = uλk + vλk ,[uλk , vλk

]= arginfvλk−1

=uλk+vλk

(∫Ω|∇uλk |+ λk

∫Ω|vk−1 − uk |2

).

With this scheme after N + 1 steps we get:

f = uλ0 + vλ0

= uλ0 + uλ1 + vλ1

= ...

= uλ0 + uλ1 + ...+ uλN + vλN .

i.e. a nonlinear multiscale decomposition: f ∼∑N

k=0 uλk + vλN .

Page 56: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Starting point: the idea of Tadmor-Nezzar-Vese (TNV) 2004, 2008

Recall ROF decomposition: f = uλ0 + vλ0 , where λ0 dictates the scale.

vλ0 can be decomposed with a scaling parameter λ1 > λ0.

vλ0 = uλ1 + vλ1 , [uλ1 , vλ1 ] = arginfvλ0

=uλ1+vλ1

(∫Ω

|∇uλ1 |+ λ1

∫Ω

|vλ0 − uλ1 |2).

TNV multiscale decomposition

vλk−1 = uλk + vλk ,[uλk , vλk

]= arginfvλk−1

=uλk+vλk

(∫Ω|∇uλk |+ λk

∫Ω|vk−1 − uk |2

).

With this scheme after N + 1 steps we get:

f = uλ0 + vλ0

= uλ0 + uλ1 + vλ1

= ...

= uλ0 + uλ1 + ...+ uλN + vλN .

i.e. a nonlinear multiscale decomposition: f ∼∑N

k=0 uλk + vλN .

Page 57: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Starting point: the idea of Tadmor-Nezzar-Vese (TNV) 2004, 2008

Recall ROF decomposition: f = uλ0 + vλ0 , where λ0 dictates the scale.

vλ0 can be decomposed with a scaling parameter λ1 > λ0.

vλ0 = uλ1 + vλ1 , [uλ1 , vλ1 ] = arginfvλ0

=uλ1+vλ1

(∫Ω

|∇uλ1 |+ λ1

∫Ω

|vλ0 − uλ1 |2).

TNV multiscale decomposition

vλk−1 = uλk + vλk ,[uλk , vλk

]= arginfvλk−1

=uλk+vλk

(∫Ω|∇uλk |+ λk

∫Ω|vk−1 − uk |2

).

With this scheme after N + 1 steps we get:

f = uλ0 + vλ0

= uλ0 + uλ1 + vλ1

= ...

= uλ0 + uλ1 + ...+ uλN + vλN .

i.e. a nonlinear multiscale decomposition: f ∼∑N

k=0 uλk + vλN .

Page 58: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

TNV scheme

k th step in TNV scheme: uλk + vλk = vλk−1

[uλk , vλk ] = arginfvλk−1

=uλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − uλk |2)

uλk −1

2λkdiv(∇uλk

|∇uλk |

)︸ ︷︷ ︸

vλk

= vλk−1

TNV iteration:uλk + vλk = vλk−1

Telescopic sum of the above gives us:N∑

k=0

uλk + vλN = f

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)

Page 59: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

TNV scheme

k th step in TNV scheme: uλk + vλk = vλk−1

[uλk , vλk ] = arginfvλk−1

=uλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − uλk |2)

uλk −1

2λkdiv(∇uλk

|∇uλk |

)︸ ︷︷ ︸

vλk

= vλk−1

TNV iteration:uλk + vλk = vλk−1

Telescopic sum of the above gives us:N∑

k=0

uλk + vλN = f

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)

Page 60: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

TNV scheme

k th step in TNV scheme: uλk + vλk = vλk−1

[uλk , vλk ] = arginfvλk−1

=uλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − uλk |2)

uλk −1

2λkdiv(∇uλk

|∇uλk |

)︸ ︷︷ ︸

vλk

= vλk−1

TNV iteration:uλk + vλk = vλk−1

Telescopic sum of the above gives us:N∑

k=0

uλk + vλN = f

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)

Page 61: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

TNV scheme

k th step in TNV scheme: uλk + vλk = vλk−1

[uλk , vλk ] = arginfvλk−1

=uλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − uλk |2)

uλk −1

2λkdiv(∇uλk

|∇uλk |

)︸ ︷︷ ︸

vλk

= vλk−1

TNV iteration:uλk + vλk = vλk−1

Telescopic sum of the above gives us:N∑

k=0

uλk + vλN = f

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)

Page 62: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

TNV scheme

k th step in TNV scheme: uλk + vλk = vλk−1

[uλk , vλk ] = arginfvλk−1

=uλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − uλk |2)

uλk −1

2λkdiv(∇uλk

|∇uλk |

)︸ ︷︷ ︸

vλk

= vλk−1

TNV iteration:uλk + vλk = vλk−1

Telescopic sum of the above gives us:N∑

k=0

uλk + vλN = f

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)

Page 63: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

TNV scheme

k th step in TNV scheme: uλk + vλk = vλk−1

[uλk , vλk ] = arginfvλk−1

=uλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − uλk |2)

uλk −1

2λkdiv(∇uλk

|∇uλk |

)︸ ︷︷ ︸

vλk

= vλk−1

TNV iteration:uλk + vλk = vλk−1

Telescopic sum of the above gives us:N∑

k=0

uλk + vλN = f

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)

Page 64: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

TNV scheme

k th step in TNV scheme: uλk + vλk = vλk−1

[uλk , vλk ] = arginfvλk−1

=uλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − uλk |2)

uλk −1

2λkdiv(∇uλk

|∇uλk |

)︸ ︷︷ ︸

vλk

= vλk−1

TNV iteration:uλk + vλk = vλk−1

Telescopic sum of the above gives us:N∑

k=0

uλk + vλN = f

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)

Page 65: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A new formulation of the TNV scheme

We have the TNV scheme as follows:

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

).

Define UN as the sum UN =∑N

k=0 uk ⇒ uN = UN − UN−1, we get :

New formulation of TNV

UN = f +1

2λNdiv(∇(UN − UN−1)

|∇(UN − UN−1)|

)

Question: How do we solve this numerically ?

Page 66: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A new formulation of the TNV scheme

We have the TNV scheme as follows:

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

).

Define UN as the sum UN =∑N

k=0 uk ⇒ uN = UN − UN−1, we get :

New formulation of TNV

UN = f +1

2λNdiv(∇(UN − UN−1)

|∇(UN − UN−1)|

)

Question: How do we solve this numerically ?

Page 67: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

A new formulation of the TNV scheme

We have the TNV scheme as follows:

N∑k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

).

Define UN as the sum UN =∑N

k=0 uk ⇒ uN = UN − UN−1, we get :

New formulation of TNV

UN = f +1

2λNdiv(∇(UN − UN−1)

|∇(UN − UN−1)|

)

Question: How do we solve this numerically ?

Page 68: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

How to solve TNV numerically ?

We need to solve this : UN = f + 12λN

div(∇(UN−UN−1)

|∇(UN−UN−1)|

).

cE (U) ≡1√

(εh)2 + (∆+x Ui,j )2 + (∆0y Ui,j )2, cW (U) ≡

1√(εh)2 + (∆−x Ui,j )2 + (∆0y Ui−1,j )2

,

cS (U) ≡1√

(εh)2 + (∆0x Ui,j )2 + (∆+y Ui,j )2, cN (U) ≡

1√(εh)2 + (∆0x Ui,j−1)2 + (∆−y Ui,j )2

.

Given UN−1 we get UN by solving the following fixed point iteration.

Uni,j =

2λhfi,j + cE (Un−1i+1,j − UN−1

i+1,j ) + cW (Un−1i−1,j − UN−1

i−1,j ) + cS (Un−1i,j+1 − UN−1

i,j+1 ) + cN (Un−1i,j−1 − UN−1

i,j−1) + (∑

c )UN−1

2λhfi,j +∑

c,

where cE ≡ cE (Un−1 − UN−1) etc. and∑

c ≡ cE + cW + cN + cS .

Page 69: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

How to solve TNV numerically ?

We need to solve this : UN = f + 12λN

div(∇(UN−UN−1)

|∇(UN−UN−1)|

).

cE (U) ≡1√

(εh)2 + (∆+x Ui,j )2 + (∆0y Ui,j )2, cW (U) ≡

1√(εh)2 + (∆−x Ui,j )2 + (∆0y Ui−1,j )2

,

cS (U) ≡1√

(εh)2 + (∆0x Ui,j )2 + (∆+y Ui,j )2, cN (U) ≡

1√(εh)2 + (∆0x Ui,j−1)2 + (∆−y Ui,j )2

.

Given UN−1 we get UN by solving the following fixed point iteration.

Uni,j =

2λhfi,j + cE (Un−1i+1,j − UN−1

i+1,j ) + cW (Un−1i−1,j − UN−1

i−1,j ) + cS (Un−1i,j+1 − UN−1

i,j+1 ) + cN (Un−1i,j−1 − UN−1

i,j−1) + (∑

c )UN−1

2λhfi,j +∑

c,

where cE ≡ cE (Un−1 − UN−1) etc. and∑

c ≡ cE + cW + cN + cS .

Page 70: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

How to solve TNV numerically ?

We need to solve this : UN = f + 12λN

div(∇(UN−UN−1)

|∇(UN−UN−1)|

).

cE (U) ≡1√

(εh)2 + (∆+x Ui,j )2 + (∆0y Ui,j )2, cW (U) ≡

1√(εh)2 + (∆−x Ui,j )2 + (∆0y Ui−1,j )2

,

cS (U) ≡1√

(εh)2 + (∆0x Ui,j )2 + (∆+y Ui,j )2, cN (U) ≡

1√(εh)2 + (∆0x Ui,j−1)2 + (∆−y Ui,j )2

.

Given UN−1 we get UN by solving the following fixed point iteration.

Uni,j =

2λhfi,j + cE (Un−1i+1,j − UN−1

i+1,j ) + cW (Un−1i−1,j − UN−1

i−1,j ) + cS (Un−1i,j+1 − UN−1

i,j+1 ) + cN (Un−1i,j−1 − UN−1

i,j−1) + (∑

c )UN−1

2λhfi,j +∑

c,

where cE ≡ cE (Un−1 − UN−1) etc. and∑

c ≡ cE + cW + cN + cS .

Page 71: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Numerical results of standard TNV continued ...

Numerical results of UN = f + 12λN

div(∇(UN−UN−1)

|∇(UN−UN−1)|

).

Figure: Numerical results with standard TNV model. We used λk = (0.0005)× 2k .

Page 72: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Going from TNV to a novel integro-differential equation

Recall for TNV formulation: UN =∑N

k=0 uk and uN = UN − UN−1.

UN = f +1

2λNdiv(∇(UN − UN−1)

|∇(UN − UN−1)|

)N∑

k=0

uk = f +1

2λNdiv(∇uN

|∇uN |

).

This motivates us to write the following model.

The novel integro-differential model

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where λ(t) > 0 is an increasing scaling function at our disposal.

Page 73: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Going from TNV to a novel integro-differential equation

Recall for TNV formulation: UN =∑N

k=0 uk and uN = UN − UN−1.

UN = f +1

2λNdiv(∇(UN − UN−1)

|∇(UN − UN−1)|

)N∑

k=0

uk = f +1

2λNdiv(∇uN

|∇uN |

).

This motivates us to write the following model.

The novel integro-differential model

∫ t

0u(x , s)ds = f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where λ(t) > 0 is an increasing scaling function at our disposal.

Page 74: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

How to solve it numerically ?

Let ∆t be the time interval step. Thus, after N steps:

U(t) :=

∫ t

0u(x , s)ds =

N−1∑k=0

∫ (k+1)∆t

k∆tu(x , s)ds

UN :=∫ N∆t

0 u(x , s)ds and uk+1 := u((k + 1)∆t), with this we have

UN ≈ UN−1 + uN ∆t .

Thus, we have the following fixed point iteration.

uni,j =

2λNh(fi,j − UN−1i,j ) + cE un−1

i+1,j + cW un−1i−1,j + cSun−1

i,j+1 + cNun−1i,j−1

2λNh∆t + cE + cW + cS + cN.

This fixed point implementation gives us uN and thus UN = UN−1 + uN∆t

Page 75: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

How to solve it numerically ?

Let ∆t be the time interval step. Thus, after N steps:

U(t) :=

∫ t

0u(x , s)ds =

N−1∑k=0

∫ (k+1)∆t

k∆tu(x , s)ds

UN :=∫ N∆t

0 u(x , s)ds and uk+1 := u((k + 1)∆t), with this we have

UN ≈ UN−1 + uN ∆t .

Thus, we have the following fixed point iteration.

uni,j =

2λNh(fi,j − UN−1i,j ) + cE un−1

i+1,j + cW un−1i−1,j + cSun−1

i,j+1 + cNun−1i,j−1

2λNh∆t + cE + cW + cS + cN.

This fixed point implementation gives us uN and thus UN = UN−1 + uN∆t

Page 76: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

How to solve it numerically ?

Let ∆t be the time interval step. Thus, after N steps:

U(t) :=

∫ t

0u(x , s)ds =

N−1∑k=0

∫ (k+1)∆t

k∆tu(x , s)ds

UN :=∫ N∆t

0 u(x , s)ds and uk+1 := u((k + 1)∆t), with this we have

UN ≈ UN−1 + uN ∆t .

Thus, we have the following fixed point iteration.

uni,j =

2λNh(fi,j − UN−1i,j ) + cE un−1

i+1,j + cW un−1i−1,j + cSun−1

i,j+1 + cNun−1i,j−1

2λNh∆t + cE + cW + cS + cN.

This fixed point implementation gives us uN and thus UN = UN−1 + uN∆t

Page 77: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

How to solve it numerically ?

Let ∆t be the time interval step. Thus, after N steps:

U(t) :=

∫ t

0u(x , s)ds =

N−1∑k=0

∫ (k+1)∆t

k∆tu(x , s)ds

UN :=∫ N∆t

0 u(x , s)ds and uk+1 := u((k + 1)∆t), with this we have

UN ≈ UN−1 + uN ∆t .

Thus, we have the following fixed point iteration.

uni,j =

2λNh(fi,j − UN−1i,j ) + cE un−1

i+1,j + cW un−1i−1,j + cSun−1

i,j+1 + cNun−1i,j−1

2λNh∆t + cE + cW + cS + cN.

This fixed point implementation gives us uN and thus UN = UN−1 + uN∆t

Page 78: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Proposed model λ(t) = (0.02)2t , on 256× 256 image of Lenna.

Numerical result for∫ t

0 u(x , s)ds = f (x) + 12λ(t) div

(∇u(x,t)|∇u(x,t)|

).

Figure: As λ(t)→∞, the image∫ t

0 u(x , s)ds approaches the given image f .

Page 79: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Modified integro-differential model

We propose a modified version of our model

Modified integro differential model

∫ t

0u(x , s)ds = f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where g is diffusion controlling function. (Recall Perona-Malik.)

The motivation: numerical implementation of the ROF model.

Euler-Lagrange differential equation for ROF:

u = f +1

2λdiv(∇u|∇u|

).

There are two problems here.Problem 1 : |∇u| ≈ 0.Problem 2 : |∇u| is undefined at the sharp discontinuities.

Page 80: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Modified integro-differential model

We propose a modified version of our model

Modified integro differential model

∫ t

0u(x , s)ds = f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where g is diffusion controlling function. (Recall Perona-Malik.)

The motivation: numerical implementation of the ROF model.

Euler-Lagrange differential equation for ROF:

u = f +1

2λdiv(∇u|∇u|

).

There are two problems here.Problem 1 : |∇u| ≈ 0.Problem 2 : |∇u| is undefined at the sharp discontinuities.

Page 81: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Modified integro-differential model

We propose a modified version of our model

Modified integro differential model

∫ t

0u(x , s)ds = f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where g is diffusion controlling function. (Recall Perona-Malik.)

The motivation: numerical implementation of the ROF model.

Euler-Lagrange differential equation for ROF:

u = f +1

2λdiv(∇u|∇u|

).

There are two problems here.Problem 1 : |∇u| ≈ 0.Problem 2 : |∇u| is undefined at the sharp discontinuities.

Page 82: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Modified integro-differential model

We propose a modified version of our model

Modified integro differential model

∫ t

0u(x , s)ds = f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where g is diffusion controlling function. (Recall Perona-Malik.)

The motivation: numerical implementation of the ROF model.

Euler-Lagrange differential equation for ROF:

u = f +1

2λdiv(∇u|∇u|

).

There are two problems here.Problem 1 : |∇u| ≈ 0.Problem 2 : |∇u| is undefined at the sharp discontinuities.

Page 83: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

The motivation: numerical implementation of the ROF model.

Problem 1 : |∇u| ≈ 0 : u = f + 12λ div

(∇u√

ε2+|∇u|2

)Problem 2 : How to deal with the sharp discontinuities ?

Idea : increase the cell size h if |∇u| is large: non-uniform grid.

Let h = hg(|G?∇u|) with g(0) = 1 and vanishing at infinity⇒

u = f +g(|G ?∇u|)

2λdiv

∇u√(εh)2 + |∇u|2

.

This motivates us to look at :

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

).

Numerical results ...

Page 84: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

The motivation: numerical implementation of the ROF model.

Problem 1 : |∇u| ≈ 0 : u = f + 12λ div

(∇u√

ε2+|∇u|2

)Problem 2 : How to deal with the sharp discontinuities ?

Idea : increase the cell size h if |∇u| is large: non-uniform grid.

Let h = hg(|G?∇u|) with g(0) = 1 and vanishing at infinity⇒

u = f +g(|G ?∇u|)

2λdiv

∇u√(εh)2 + |∇u|2

.

This motivates us to look at :

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

).

Numerical results ...

Page 85: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

The motivation: numerical implementation of the ROF model.

Problem 1 : |∇u| ≈ 0 : u = f + 12λ div

(∇u√

ε2+|∇u|2

)Problem 2 : How to deal with the sharp discontinuities ?

Idea : increase the cell size h if |∇u| is large: non-uniform grid.

Let h = hg(|G?∇u|) with g(0) = 1 and vanishing at infinity⇒

u = f +g(|G ?∇u|)

2λdiv

∇u√(εh)2 + |∇u|2

.

This motivates us to look at :

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

).

Numerical results ...

Page 86: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

The motivation: numerical implementation of the ROF model.

Problem 1 : |∇u| ≈ 0 : u = f + 12λ div

(∇u√

ε2+|∇u|2

)Problem 2 : How to deal with the sharp discontinuities ?

Idea : increase the cell size h if |∇u| is large: non-uniform grid.

Let h = hg(|G?∇u|) with g(0) = 1 and vanishing at infinity⇒

u = f +g(|G ?∇u|)

2λdiv

∇u√(εh)2 + |∇u|2

.

This motivates us to look at :

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

).

Numerical results ...

Page 87: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

The motivation: numerical implementation of the ROF model.

Problem 1 : |∇u| ≈ 0 : u = f + 12λ div

(∇u√

ε2+|∇u|2

)Problem 2 : How to deal with the sharp discontinuities ?

Idea : increase the cell size h if |∇u| is large: non-uniform grid.

Let h = hg(|G?∇u|) with g(0) = 1 and vanishing at infinity⇒

u = f +g(|G ?∇u|)

2λdiv

∇u√(εh)2 + |∇u|2

.

This motivates us to look at :

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

).

Numerical results ...

Page 88: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

The motivation: numerical implementation of the ROF model.

Problem 1 : |∇u| ≈ 0 : u = f + 12λ div

(∇u√

ε2+|∇u|2

)Problem 2 : How to deal with the sharp discontinuities ?

Idea : increase the cell size h if |∇u| is large: non-uniform grid.

Let h = hg(|G?∇u|) with g(0) = 1 and vanishing at infinity⇒

u = f +g(|G ?∇u|)

2λdiv

∇u√(εh)2 + |∇u|2

.

This motivates us to look at :

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

).

Numerical results ...

Page 89: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

The motivation: numerical implementation of the ROF model.

Problem 1 : |∇u| ≈ 0 : u = f + 12λ div

(∇u√

ε2+|∇u|2

)Problem 2 : How to deal with the sharp discontinuities ?

Idea : increase the cell size h if |∇u| is large: non-uniform grid.

Let h = hg(|G?∇u|) with g(0) = 1 and vanishing at infinity⇒

u = f +g(|G ?∇u|)

2λdiv

∇u√(εh)2 + |∇u|2

.

This motivates us to look at :

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

).

Numerical results ...

Page 90: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Comparison between standard ROF and modified ROF

(a) (b)

Figure: (a) Result of standard ROF and (b) result of the modified ROF withg(|G ?∇u|); both for the same λ = 0.0001.

Page 91: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Modified TNV and the proposed model

TNV scheme:N∑

k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)We looked at modified ROF:

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

)Can we modify TNV ?

Modified TNV scheme

N∑k=0

uλk = f +g(|G ?∇uλN |)

2λNdiv(∇uλN

|∇uλN |

)

Numerical results ...

Page 92: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Modified TNV and the proposed model

TNV scheme:N∑

k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)We looked at modified ROF:

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

)Can we modify TNV ?

Modified TNV scheme

N∑k=0

uλk = f +g(|G ?∇uλN |)

2λNdiv(∇uλN

|∇uλN |

)

Numerical results ...

Page 93: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Modified TNV and the proposed model

TNV scheme:N∑

k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)We looked at modified ROF:

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

)Can we modify TNV ?

Modified TNV scheme

N∑k=0

uλk = f +g(|G ?∇uλN |)

2λNdiv(∇uλN

|∇uλN |

)

Numerical results ...

Page 94: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Modified TNV and the proposed model

TNV scheme:N∑

k=0

uλk = f +1

2λNdiv(∇uλN

|∇uλN |

)We looked at modified ROF:

u = f +g(|G ?∇u|)

2λdiv(∇u|∇u|

)Can we modify TNV ?

Modified TNV scheme

N∑k=0

uλk = f +g(|G ?∇uλN |)

2λNdiv(∇uλN

|∇uλN |

)

Numerical results ...

Page 95: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Numerical results of modified TNV.

Numerical results of∑N

k=0 uλk = f +g(|G?∇uλN

|)2λN

div( ∇uλN|∇uλN

|

).

Figure: Numerical results of TNV with diffusion controlling function g(s) = 11+s2 ,

with initial λk = (0.0005)× 2k .

Can we modify the integro-differential model ?

Page 96: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Numerical results of modified TNV.

Numerical results of∑N

k=0 uλk = f +g(|G?∇uλN

|)2λN

div( ∇uλN|∇uλN

|

).

Figure: Numerical results of TNV with diffusion controlling function g(s) = 11+s2 ,

with initial λk = (0.0005)× 2k .

Can we modify the integro-differential model ?

Page 97: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

We looked at modified TNV:

N∑k=0

uλk = f +g(|G ?∇uλN |)

2λNdiv(∇uλN

|∇uλN |

)This leads to modified Integro-differential equation:

∫ t

0u(x , s)ds = f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Numerical results ...

Page 98: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

We looked at modified TNV:

N∑k=0

uλk = f +g(|G ?∇uλN |)

2λNdiv(∇uλN

|∇uλN |

)This leads to modified Integro-differential equation:

∫ t

0u(x , s)ds = f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Numerical results ...

Page 99: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

We looked at modified TNV:

N∑k=0

uλk = f +g(|G ?∇uλN |)

2λNdiv(∇uλN

|∇uλN |

)This leads to modified Integro-differential equation:

∫ t

0u(x , s)ds = f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Numerical results ...

Page 100: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Numerical results of the modified integro-differential model

Numerical results of∫ t

0 u(x , s)ds = f (x) + g(|G?∇u(x,t)|)2λ(t) div

(∇u(x,t)|∇u(x,t)|

).

Figure: Numerical results of modified integro-differential mode with diffusioncontrolling function g(s) = 1

1+s2 . The function λ(t) = 10−4e2t × 2t .

Page 101: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

What does the scaling function λ(t) mean ?

Recall the integro-differential model∫ t

0u(x , s)ds − f (x) =

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where λ(t) is an increasing function at our disposal.

Star-norm is the dual of the BV norm w.r.t. the L2 scalar product

‖v‖∗ := supφ 6=0

〈v , φ〉∫Ω|∇φ|

.

Theorem: Let us define for the integro-differential equation the errorterm as

∫ t0 u(x , s)ds − f (x), then

‖∫ t

0u(x , s)ds − f (x)‖∗ =

12λ(t)

.

Proof: The proof follows from Meyer’s theorem: for ROF decompositionf = u + v with scale λ, we get

∫Ω

uv = ‖u‖BV‖v‖∗ and ‖v‖∗ = 12λ .

Page 102: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

What does the scaling function λ(t) mean ?

Recall the integro-differential model∫ t

0u(x , s)ds − f (x) =

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where λ(t) is an increasing function at our disposal.

Star-norm is the dual of the BV norm w.r.t. the L2 scalar product

‖v‖∗ := supφ 6=0

〈v , φ〉∫Ω|∇φ|

.

Theorem: Let us define for the integro-differential equation the errorterm as

∫ t0 u(x , s)ds − f (x), then

‖∫ t

0u(x , s)ds − f (x)‖∗ =

12λ(t)

.

Proof: The proof follows from Meyer’s theorem: for ROF decompositionf = u + v with scale λ, we get

∫Ω

uv = ‖u‖BV‖v‖∗ and ‖v‖∗ = 12λ .

Page 103: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

What does the scaling function λ(t) mean ?

Recall the integro-differential model∫ t

0u(x , s)ds − f (x) =

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where λ(t) is an increasing function at our disposal.

Star-norm is the dual of the BV norm w.r.t. the L2 scalar product

‖v‖∗ := supφ 6=0

〈v , φ〉∫Ω|∇φ|

.

Theorem: Let us define for the integro-differential equation the errorterm as

∫ t0 u(x , s)ds − f (x), then

‖∫ t

0u(x , s)ds − f (x)‖∗ =

12λ(t)

.

Proof: The proof follows from Meyer’s theorem: for ROF decompositionf = u + v with scale λ, we get

∫Ω

uv = ‖u‖BV‖v‖∗ and ‖v‖∗ = 12λ .

Page 104: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

What does the scaling function λ(t) mean ?

Recall the integro-differential model∫ t

0u(x , s)ds − f (x) =

12λ(t)

div(∇u(x , t)|∇u(x , t)|

).

where λ(t) is an increasing function at our disposal.

Star-norm is the dual of the BV norm w.r.t. the L2 scalar product

‖v‖∗ := supφ 6=0

〈v , φ〉∫Ω|∇φ|

.

Theorem: Let us define for the integro-differential equation the errorterm as

∫ t0 u(x , s)ds − f (x), then

‖∫ t

0u(x , s)ds − f (x)‖∗ =

12λ(t)

.

Proof: The proof follows from Meyer’s theorem: for ROF decompositionf = u + v with scale λ, we get

∫Ω

uv = ‖u‖BV‖v‖∗ and ‖v‖∗ = 12λ .

Page 105: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Remark: This theorem is important, in the sense that it dictates the star-normof the residual

∫ t0 u(x , s)ds − f (x) at any time. We get the following result

using this property.

Corollary: The star-norm of the residual vanishes as t →∞

Proof: In our model the function limt→∞ λ(t) =∞. Thus, the result followsfrom the previous theorem :

‖∫ t

0u(x , s)ds − f (x)‖∗ =

12λ(t)

.

Question: What happens if limt→∞ λ(t) = λ∞ <∞ ?

Page 106: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Remark: This theorem is important, in the sense that it dictates the star-normof the residual

∫ t0 u(x , s)ds − f (x) at any time. We get the following result

using this property.

Corollary: The star-norm of the residual vanishes as t →∞

Proof: In our model the function limt→∞ λ(t) =∞. Thus, the result followsfrom the previous theorem :

‖∫ t

0u(x , s)ds − f (x)‖∗ =

12λ(t)

.

Question: What happens if limt→∞ λ(t) = λ∞ <∞ ?

Page 107: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Remark: This theorem is important, in the sense that it dictates the star-normof the residual

∫ t0 u(x , s)ds − f (x) at any time. We get the following result

using this property.

Corollary: The star-norm of the residual vanishes as t →∞

Proof: In our model the function limt→∞ λ(t) =∞. Thus, the result followsfrom the previous theorem :

‖∫ t

0u(x , s)ds − f (x)‖∗ =

12λ(t)

.

Question: What happens if limt→∞ λ(t) = λ∞ <∞ ?

Page 108: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Remark: This theorem is important, in the sense that it dictates the star-normof the residual

∫ t0 u(x , s)ds − f (x) at any time. We get the following result

using this property.

Corollary: The star-norm of the residual vanishes as t →∞

Proof: In our model the function limt→∞ λ(t) =∞. Thus, the result followsfrom the previous theorem :

‖∫ t

0u(x , s)ds − f (x)‖∗ =

12λ(t)

.

Question: What happens if limt→∞ λ(t) = λ∞ <∞ ?

Page 109: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Where to start and stop ?

Starting : We know that for ROF decomposition if ‖f‖∗ < 12λ then v = f ,

thus we start with a very small value of λ(t).

Stopping : This is an open problem.We know : ‖

∫ t0 u(x , s)ds − f (x)‖∗ = 1

2λ(t) .

Question : What does the star-norm really mean ?

Page 110: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Where to start and stop ?

Starting : We know that for ROF decomposition if ‖f‖∗ < 12λ then v = f ,

thus we start with a very small value of λ(t).

Stopping : This is an open problem.We know : ‖

∫ t0 u(x , s)ds − f (x)‖∗ = 1

2λ(t) .

Question : What does the star-norm really mean ?

Page 111: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Observation: Let f and g be in (X , (·, ·)), an inner product space.

Let f ∼ uk , vk∞k=0 and g ∼ Uk ,Vk∞k=0 be any hierarchical decompositions :

v−1 = f and vk−1 = uk + vk for k = 0, 1, 2 . . .∞ andV−1 = g and Vk−1 = Uk + Vk for k = 0, 1, 2 . . .∞.

Let us define an inner product

〈f , g〉 =∞∑

k=0

(uk ,Uk ) + (vk ,Uk ) + (uk ,Vk ) +(vk ,Vk )

Then 〈f , g〉 = (f , g) if and only if limk→∞(vk ,Vk ) = 0.

Indeed for (BV , L2) multiscale decompositions: ‖vλk ‖L2 → 0 ...

Page 112: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Observation: Let f and g be in (X , (·, ·)), an inner product space.

Let f ∼ uk , vk∞k=0 and g ∼ Uk ,Vk∞k=0 be any hierarchical decompositions :

v−1 = f and vk−1 = uk + vk for k = 0, 1, 2 . . .∞ andV−1 = g and Vk−1 = Uk + Vk for k = 0, 1, 2 . . .∞.

Let us define an inner product

〈f , g〉 =∞∑

k=0

(uk ,Uk ) + (vk ,Uk ) + (uk ,Vk ) +(vk ,Vk )

Then 〈f , g〉 = (f , g) if and only if limk→∞(vk ,Vk ) = 0.

Indeed for (BV , L2) multiscale decompositions: ‖vλk ‖L2 → 0 ...

Page 113: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Observation: Let f and g be in (X , (·, ·)), an inner product space.

Let f ∼ uk , vk∞k=0 and g ∼ Uk ,Vk∞k=0 be any hierarchical decompositions :

v−1 = f and vk−1 = uk + vk for k = 0, 1, 2 . . .∞ andV−1 = g and Vk−1 = Uk + Vk for k = 0, 1, 2 . . .∞.

Let us define an inner product

〈f , g〉 =∞∑

k=0

(uk ,Uk ) + (vk ,Uk ) + (uk ,Vk ) +(vk ,Vk )

Then 〈f , g〉 = (f , g) if and only if limk→∞(vk ,Vk ) = 0.

Indeed for (BV , L2) multiscale decompositions: ‖vλk ‖L2 → 0 ...

Page 114: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Observation: Let f and g be in (X , (·, ·)), an inner product space.

Let f ∼ uk , vk∞k=0 and g ∼ Uk ,Vk∞k=0 be any hierarchical decompositions :

v−1 = f and vk−1 = uk + vk for k = 0, 1, 2 . . .∞ andV−1 = g and Vk−1 = Uk + Vk for k = 0, 1, 2 . . .∞.

Let us define an inner product

〈f , g〉 =∞∑

k=0

(uk ,Uk ) + (vk ,Uk ) + (uk ,Vk ) +(vk ,Vk )

Then 〈f , g〉 = (f , g) if and only if limk→∞(vk ,Vk ) = 0.

Indeed for (BV , L2) multiscale decompositions: ‖vλk ‖L2 → 0 ...

Page 115: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Observation: Let f and g be in (X , (·, ·)), an inner product space.

Let f ∼ uk , vk∞k=0 and g ∼ Uk ,Vk∞k=0 be any hierarchical decompositions :

v−1 = f and vk−1 = uk + vk for k = 0, 1, 2 . . .∞ andV−1 = g and Vk−1 = Uk + Vk for k = 0, 1, 2 . . .∞.

Let us define an inner product

〈f , g〉 =∞∑

k=0

(uk ,Uk ) + (vk ,Uk ) + (uk ,Vk ) +(vk ,Vk )

Then 〈f , g〉 = (f , g) if and only if limk→∞(vk ,Vk ) = 0.

Indeed for (BV , L2) multiscale decompositions: ‖vλk ‖L2 → 0 ...

Page 116: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Consequences of this observation

For (BV , L2) hierarchical decomposition of f we get

(f , f )L2 = 〈f , f 〉 =∞∑

k=0

(uλk , uλk )L2 + (vλk , uλk )L2 + (uλk , vλk )L2

‖f‖2L2 =

∞∑k=0

‖uλk ‖2L2 + 2(uλk , vλk )L2 .

Meyer’s theorem: (uλk , vλk )L2 = 12λk‖uλk ‖BV we get:

Energy decomposition (Tadmor et. al. 2004)

‖f‖2L2 =

∞∑k=0

‖uλk ‖2L2 +

1λk‖uλk ‖BV .

Page 117: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Consequences of this observation

For (BV , L2) hierarchical decomposition of f we get

(f , f )L2 = 〈f , f 〉 =∞∑

k=0

(uλk , uλk )L2 + (vλk , uλk )L2 + (uλk , vλk )L2

‖f‖2L2 =

∞∑k=0

‖uλk ‖2L2 + 2(uλk , vλk )L2 .

Meyer’s theorem: (uλk , vλk )L2 = 12λk‖uλk ‖BV we get:

Energy decomposition (Tadmor et. al. 2004)

‖f‖2L2 =

∞∑k=0

‖uλk ‖2L2 +

1λk‖uλk ‖BV .

Page 118: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Consequences of this observation

For (BV , L2) hierarchical decomposition of f we get

(f , f )L2 = 〈f , f 〉 =∞∑

k=0

(uλk , uλk )L2 + (vλk , uλk )L2 + (uλk , vλk )L2

‖f‖2L2 =

∞∑k=0

‖uλk ‖2L2 + 2(uλk , vλk )L2 .

Meyer’s theorem: (uλk , vλk )L2 = 12λk‖uλk ‖BV we get:

Energy decomposition (Tadmor et. al. 2004)

‖f‖2L2 =

∞∑k=0

‖uλk ‖2L2 +

1λk‖uλk ‖BV .

Page 119: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works

Deblurring: Tadmor et. al. (2008) consider hierarchical decomposition:

[uλk , vλk ] = arginfvλk−1

=Tuλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − Tuλk |2)

T ∗Tuλk = T ∗vλk−1 +1

2λkdiv(∇uλk

|∇uλk |

)

(a) (b)

Figure: A blurred image (a) is deblurred as shown in (b).

Novel ‘deblurring’ integro-differential equation

∫ t

0T ∗Tu(x , s)ds = T ∗f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Page 120: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works

Deblurring: Tadmor et. al. (2008) consider hierarchical decomposition:

[uλk , vλk ] = arginfvλk−1

=Tuλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − Tuλk |2)

T ∗Tuλk = T ∗vλk−1 +1

2λkdiv(∇uλk

|∇uλk |

)

(a) (b)

Figure: A blurred image (a) is deblurred as shown in (b).

Novel ‘deblurring’ integro-differential equation

∫ t

0T ∗Tu(x , s)ds = T ∗f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Page 121: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works

Deblurring: Tadmor et. al. (2008) consider hierarchical decomposition:

[uλk , vλk ] = arginfvλk−1

=Tuλk+vλk

(∫Ω

|∇uλk |+ λk

∫Ω

|vλk−1 − Tuλk |2)

T ∗Tuλk = T ∗vλk−1 +1

2λkdiv(∇uλk

|∇uλk |

)

(a) (b)

Figure: A blurred image (a) is deblurred as shown in (b).

Novel ‘deblurring’ integro-differential equation

∫ t

0T ∗Tu(x , s)ds = T ∗f (x) +

12λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Page 122: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works ...

Modified TNV deblurring

T ∗Tuλk = T ∗vλk−1 +g(|G ?∇uλk |)

2λkdiv(∇uλk

|∇uλk |

)

(a) (b)

Figure: A blurred image (a) is deblurred with modified TNV deblurring as shown in (b).

Modified ‘deblurring’ integro-differential equation

∫ t

0T ∗Tu(x , s)ds = T ∗f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Page 123: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works ...

Modified TNV deblurring

T ∗Tuλk = T ∗vλk−1 +g(|G ?∇uλk |)

2λkdiv(∇uλk

|∇uλk |

)

(a) (b)

Figure: A blurred image (a) is deblurred with modified TNV deblurring as shown in (b).

Modified ‘deblurring’ integro-differential equation

∫ t

0T ∗Tu(x , s)ds = T ∗f (x) +

g(|G ?∇u(x , t)|)2λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Page 124: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works ...

Modified TNV deblurring with selective diffusion:

T ∗Tuλk = T ∗vλk−1 +g(|G ?∇uλk |)|∇uλk |

2λkdiv(∇uλk

|∇uλk |

)

(a) (b)

Figure: A blurred image (a) is denoised with modified TNV deblurring withselective diffusion as shown in (b).

Modified ‘deblurring’ integro-differential equation

∫ t

0T ∗Tu(x , s)ds = T ∗f (x) +

g(|G ?∇u(x , t)|)|∇u(x , t)|2λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Page 125: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works ...

Modified TNV deblurring with selective diffusion:

T ∗Tuλk = T ∗vλk−1 +g(|G ?∇uλk |)|∇uλk |

2λkdiv(∇uλk

|∇uλk |

)

(a) (b)

Figure: A blurred image (a) is denoised with modified TNV deblurring withselective diffusion as shown in (b).

Modified ‘deblurring’ integro-differential equation

∫ t

0T ∗Tu(x , s)ds = T ∗f (x) +

g(|G ?∇u(x , t)|)|∇u(x , t)|2λ(t)

div(∇u(x , t)|∇u(x , t)|

)

Page 126: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works ...

multiscale (BV , L1)

multiscale (BV , (L1)2)

multiscale (BV , L1), (BV , (L1)2) with g(|G ?∇u|) and g(|G ?∇u|)|∇u|

(a) (b)

Figure: A blurred image (a) is denoised with multiscale (BV , (L1)2) withg(|G ?∇u|)|∇u| as shown in (b).

Page 127: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works ...

multiscale (BV , L1)

multiscale (BV , (L1)2)

multiscale (BV , L1), (BV , (L1)2) with g(|G ?∇u|) and g(|G ?∇u|)|∇u|

(a) (b)

Figure: A blurred image (a) is denoised with multiscale (BV , (L1)2) withg(|G ?∇u|)|∇u| as shown in (b).

Page 128: f R L () object - CSCAMM€¦ · Other variational methods in image processing... Mumford-Shah segmentation (1985) [u;v;C] = arginf ff=u+v;Cg Z C jf uj2 + 1 C jruj2 + 2 I C d˙ :

Other works ...

multiscale (BV , L1)

multiscale (BV , (L1)2)

multiscale (BV , L1), (BV , (L1)2) with g(|G ?∇u|) and g(|G ?∇u|)|∇u|

(a) (b)

Figure: A blurred image (a) is denoised with multiscale (BV , (L1)2) withg(|G ?∇u|)|∇u| as shown in (b).