Degraded digital image restoration – Spatial domain ...

Post on 23-Feb-2022

9 views 0 download

Transcript of Degraded digital image restoration – Spatial domain ...

Image Comm. Lab EE/NTHU 1

Chapter 5Image Restoration

• Degraded digital image restoration– Spatial domain processing

Additive noise– Frequency domain

Blurred image

Image Comm. Lab EE/NTHU 2

5. 1 Model of Image degradation and restoration5. 1 Model of Image degradation and restoration

•g(x, y)=h(x, y)∗f(x, y)+η(x, y)•G(u, v)=H(u, v)F(u, v)+N(u, v)

Image Comm. Lab EE/NTHU 3

5.2 Noise Model

• Spatial and frequency property of noise– White noise (random noise)– Gaussian noise

– μ: mean ; σ :variance– 70% [(μ-σ), (μ+σ)]– 95 % [(μ-2σ), (μ+2σ)]

22 2/)(

21)( σμ

σπ−−= zezp

Image Comm. Lab EE/NTHU 4

5.2 Noise Model

• Rayleigh noiseThe PDF is

• μ=a+(πb/4)1/2

• σ =b(4-π)/4

⎪⎩

⎪⎨⎧

<≥−=

−−

azforazforeaz

bzpbaz

0

)(2)(

/)( 2

Image Comm. Lab EE/NTHU 5

5.2 Noise Model

• Erlang (gamma) noise

• μ=b/a; σ =b/a2

• Exponential noise

• μ=1/a; σ =1/a2

⎪⎩

⎪⎨⎧

<

≥−=

−−

00

0)!1()(

1

zfor

zforeb

zazp

azbb

⎩⎨⎧

<≥

=−

000

)(zforzforae

zpaz

Image Comm. Lab EE/NTHU 6

5.2 Noise Model

• Uniform noise

• μ=(a+b)/2; σ =(b-a)2/12• Impulse noise (salt and pepper noise)

⎪⎩

⎪⎨⎧ ≤≤

−=otherwise

bzaforabzp

0

1)(

⎪⎩

⎪⎨

⎧==

=otherwise

bzforazfor

PP

zp b

a

0)(

Image Comm. Lab EE/NTHU 7

5.2 Noise Model5.2 Noise Model

Image Comm. Lab EE/NTHU 8

5.2 Noise Model5.2 Noise Model

Image Comm. Lab EE/NTHU 9

5.2 Noise Model5.2 Noise Model

Image Comm. Lab EE/NTHU 10

5.2 Noise Model5.2 Noise Model

Image Comm. Lab EE/NTHU 11

5.2.3 Noise Model-periodic noise

• Periodic noise is due to the electrical or electromechanical interference during image acquisition.

• Can be estimated through the inspection of the Fourier spectrum of the image.

Image Comm. Lab EE/NTHU 12

5.2.3 Noise Model-periodic noise5.2.3 Noise Model-periodic noise

Image Comm. Lab EE/NTHU 13

5.2.4 Noise Model- Noise parameters estimation

• Use a optical sensor to capture the image of a solid gray board that is illuminated uniformly.

• The resulting image is a good indicator of system noise.

• Find the mean μ and standard deviation σ of the histogram of the resulting image.

• From the μ and σ we can calculate the a and b, the parameter of that specific noise distribution.

Image Comm. Lab EE/NTHU 14

5.2 Noise Model5.2 Noise Model

Image Comm. Lab EE/NTHU 15

5.3 Restoration using spatial filter

• Mean filters– Arithmetic mean filter

– Geometric mean filter

∑∈

=xySts

tsgmn

yxf),(

),(1),(ˆ

mn

Sts xy

tsgyxf/1

),(

),(),(ˆ⎥⎥⎦

⎢⎢⎣

⎡= ∏

Image Comm. Lab EE/NTHU 16

5.3 Restoration using spatial filter

– Harmonic mean filter is good for salt noise not for pepper noise.

– Contra-harmonic filter: Q>0 reduce pepper noise Q<0 reduce salt noise.

∑∈

=

xySts tsg

mnyxf

),( ),(

1),(

ˆ

+

=

xy

xy

Sts

QSts

Q

tsg

tsgyxf

),(

),(

1

),(

),(),(ˆ

Image Comm. Lab EE/NTHU 17

5.3 Restoration using spatial filter5.3 Restoration using spatial filter

Image Comm. Lab EE/NTHU 18

5.3 Restoration using spatial filter5.3 Restoration using spatial filter

Image Comm. Lab EE/NTHU 19

5.3 Restoration using spatial filter5.3 Restoration using spatial filter

Image Comm. Lab EE/NTHU 20

5.3 Restoration using spatial filter

• Order-Statistics filtersMedian filterMax filter: find the brightest points to reduce the pepper noise

Min filter: find the darkest point to reduce the salt noise

Midpoint filter

{ }),(),(ˆ),(

tsgmedianyxfxySts ∈

=

{ }),(max),(ˆ),(

tsgyxfxySts ∈

=

{ }),(min),(ˆ),(

tsgyxfxySts ∈

=

{ } { }⎥⎦⎤

⎢⎣⎡ +=

∈∈),(max),(min

21),(ˆ

),(),(tsgtsgyxf

xyxy StsSts

Image Comm. Lab EE/NTHU 21

5.3 Restoration using spatial filter

• Alpha-trimmed mean filterSuppose delete d/2 lowest and d/2 highest gray-level value in the neighborhood of (s, t) and average the rest.

where d=0 ~ mn-1

∑∈−

=xySts

r tsgdmn

yxf),(

),(1),(ˆ

Image Comm. Lab EE/NTHU 22

5.3 Restoration using spatial filter5.3 Restoration using spatial filter

Image Comm. Lab EE/NTHU 23

5.3 Restoration using spatial filter5.3 Restoration using spatial filter

Image Comm. Lab EE/NTHU 24

5.3 Restoration using spatial filter

5.3 Restoration using spatial filter

Image Comm. Lab EE/NTHU 25

5.3 Restoration using spatial filter-Adaptive local noise reduction filter

Measurements for local region Sxy centered at (x, y)(1) noisy image at (x, y): g(x, y) with variance σ2

η(2) The local mean mL in Sxy(3) The local variance σ2

L(4) The local variance of noise is high σ2

L≥ σ2η

Conditions: Assume additive noise(a) Zero-noise case: If σ2

η =0 then f(x, y)=g(x, y)(b) Edges: If σ2

L > σ2η then f(x, y)≈g(x, y)

(c) If σ2L = σ2

η then f(x,y)= mL, the arithmetic mean

In practice, σ2η is unknown, so a test is built before filtering

]),([),(),(ˆ2

2

LL

myxgyxgyxf −−=σση

Image Comm. Lab EE/NTHU 26

5.3 Restoration using spatial filter5.3 Restoration using spatial filter

Image Comm. Lab EE/NTHU 27

5.3 Restoration using spatial filter

• Adaptive median filter can handle impulse noise with larger probability (Pa and Pb are large).Increasing(change) window size during operation

• Notations:zmin=minimum gray-level value in Sxyzmax=maximum gray-level value in Sxyzmed=median gray-level value in Sxyzxy= gray-level at (x,y)Smax=maximum allowed size of Sxy

Image Comm. Lab EE/NTHU 28

5.3 Restoration using spatial filter

• The adaptive median filter algorithm works in two levels: A and B

• Level A: A1=zmed-zmin, A2=zmed-zmaxIf A1>0 AND A2<0 goto level Belse increase the window sizeIf window size ≤ Smax repeat level Aelse output zxy

Level B: B1=zxy-zmin, B2=zxy-zmaxIf B1>0 AND B2<0, output zxyElse output zmed

Image Comm. Lab EE/NTHU 29

5 .3 Restoration using spatial filter5 .3 Restoration using spatial filter

Image Comm. Lab EE/NTHU 30

5.4 Restoration using frequency-domain filter5.4.1 band-reject filter

• Ideal band reject filter

• Butterworth band reject filter

⎪⎩

⎪⎨

+>+≤≤−

−<=

2/),(2/),(2/

2/),(

101

),(

0

00

0

WDvuDifWDvuDWDif

WDvuDifvuH

n

DvuDWvuD

vuH 2

20

2 ),(),(1

1),(

⎥⎦

⎤⎢⎣

⎡−

+

=

Image Comm. Lab EE/NTHU 31

5.4.1 Restoration using frequency-domain filter

• Gaussian band reject filter22

02

),(),(

21

1),( ⎥⎥⎦

⎢⎢⎣

⎡ −−

−= WvuDDvuD

evuH

Image Comm. Lab EE/NTHU 32

5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter

Image Comm. Lab EE/NTHU 33

5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter

Image Comm. Lab EE/NTHU 34

5.4.2 Bandpass filter

• Obtained form band reject filterHbp(u, v)=1-Hbr(u, v)

• To isolated an image of certain frequency band.

Image Comm. Lab EE/NTHU 35

5.4 Restoration using frequency-domain filterBandpass filter

5.4 Restoration using frequency-domain filterBandpass filter

Image Comm. Lab EE/NTHU 36

5.4.3 Notch filter

• Notch filter rejects (passes) frequencies in predefined neighborhoods about a center frequency. For notch rejects filter as

where⎩⎨⎧ ≤≤

=otherwise

DvuDorDvuDifvuH

1),(),(0

),( 0201

[ ] 2/120

201 )2/()2/(),( vNvuMuvuD −−+−−=

[ ] 2/120

202 )2/()2/(),( vNvuMuvuD +−++−=

Image Comm. Lab EE/NTHU 37

5.4 Restoration using frequency-domain filter-Notch filter

• Butterworth notch reject filter

• Gaussian notch reject filter

• Hnp(u, v)=1-Hnr(u, v)• It becomes a low-pass filter when u0=v0=0

n

vuDvuDD

vuH

⎥⎦

⎤⎢⎣

⎡+

=

),(),(1

1),(

21

20

⎥⎥⎦

⎢⎢⎣

⎡−

−=20

21 ),(),(21

1),( DvuDvuD

evuH

Image Comm. Lab EE/NTHU 38

5.4 Restoration using frequency-domain filterNotch filter

5.4 Restoration using frequency-domain filterNotch filter

Image Comm. Lab EE/NTHU 39

5.4 Restoration using frequency-domain filter

Notch filter

5.4 Restoration using frequency-domain filter

Notch filter

Use 1-D Notch pass filter to find the horizontal ripple noise

Image Comm. Lab EE/NTHU 40

5.4.4 Optimal Notch filter

• Clearly defined interference is not common.• Images from electro-optical scanner are corrupted by

periodic degradation.• Several interference components are present.• Place a notch pass filter H(u, v) at the location of

each spike, i.e., N(u, v)=H(u, v)G(u, v), where G(u, v)is the corrupted image.

• η(x, y)=F-1{H(u, v)G(u, v)}.• The effect of components not present in the estimate

of η(x, y) can be minimized.

Image Comm. Lab EE/NTHU 41

5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter

Image Comm. Lab EE/NTHU 42

5.4.4 Optimal Notch filter

• The restored image is • The weighting function w(x, y) is found so that the

variance of is minimized for a selected neighborhood.• One way is to select w(x, y) so that the variance of the

estimate is minimized over a small local neighborhood of size (2a+1, 2b+1).

• ∂σ2 (x, y) /∂ w(x, y)=0 → to select w(x, y)

),(),(),(),(ˆ yxyxwyxgyxf η−=

),(ˆ yxf

[ ]22 ),(ˆ),(ˆ)12)(12(

1),( ∑∑−= −=

−++++

=b

bt

a

as

yxftysxfba

yxσ

Image Comm. Lab EE/NTHU 43

5.4.4 Restoration using frequency-domain filter5.4.4 Restoration using frequency-domain filter

a=b=15

Image Comm. Lab EE/NTHU 44

5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter

Image Comm. Lab EE/NTHU 45

5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter

),(),(),(),(ˆ yxyxwyxgyxf η−=

Image Comm. Lab EE/NTHU 46

5.5 Linear Position Invariant Degradation

• The System Model

Hf(x, y)

η(x, y)

g(x, y)

g(x, y): the degraded image

f(x, y) : the original image

η(x, y): additive noise

H: System function which is a linear operator

Image Comm. Lab EE/NTHU 47

5.5 Linear Position Invariant Degradation

g(x, y)=H[f(x, y)]+η(x, y)Assume H is linear and η(x, y)=0If g(x, y)=H[f(x, y)] is position invariant then

H[f(x-α, y-β)]=g(x-α, y-β)g(x, y)=H[f(x,y)]+ η(x, y)

= +η(x, y)=h(x, y)*f(x, y)+ η(x, y)∫∫ −− βαβαβα ddyxhf ),(),(

Image Comm. Lab EE/NTHU 48

5.6 Estimating the Degradation Function

• To estimate the degradation functionby Observationby Experimentationby Mathematical modeling

(Blind deconvolution)

Image Comm. Lab EE/NTHU 49

5.6 Estimating the Degradation Function

• By observation: construct an unblurredimage of some strong signal content.

• Let the observed image is gs(x, y)• The constructed image is• Assume the noise is negligible• Find the degradation function H(u, v) which

is similar to

),(~),(),(

vuFvuGvuH

s

ss =

),(~ yxfs

Image Comm. Lab EE/NTHU 50

5.6 Estimating the Degradation Function

• By experimentation: to obtain the impulse response of the degradation by imaging an impulse (small dot of light) using the same system setting.

• The Four Transform of an impulse is a constant A

• Let g(x, y) be the observed image.• Find the degradation function as

H(u, v)=G(u, v)/A

Image Comm. Lab EE/NTHU 51

5.6 Estimating the Degradation Function5.6 Estimating the Degradation Function

Image Comm. Lab EE/NTHU 52

5.6 Estimating the Degradation Function -by modeling

• By Modeling: through experience• The physical characteristic of turbulence as

H(u, v)=e-k(u2+v2)5/6

• Where k is a constant which depend on the nature of turbulence.

• It is similar to the Gaussian Low pass. k=0.0025 (severe turbulence)k=0.001k=0.00025 (low turbulence)

Image Comm. Lab EE/NTHU 53

5.6 Estimating the Degradation Function5.6 Estimating the Degradation Function

Image Comm. Lab EE/NTHU 54

5.6 Estimating the Degradation Function -by modeling

• Through mathematical modelImage f(x, y) is blurred by uniform motion.

• x0(t) and y0(t) are the time varying component in x, y directions.

• The total exposure at any point of the film is obtained by integrating the instantaneous exposure over the time interval during which the shutter is opened.

Image Comm. Lab EE/NTHU 55

5.6 Estimating the Degradation Function-by modeling

• Assume T is duration of exposure, the blurred image g(x, y) is

• Fourier Transform of g(x, y) is

∫ −−=T

dttyytxxfyxg0

00 )](),([),(

dxdyvyuxjxpedttyytxxf

dxdyvyuxjyxgvuG

T

)](2[)](),([

)](2exp[),(),(

00 +−⎥⎥⎤

⎢⎢⎡

−−=

+−=

∫∫ ∫

∫ ∫

π

π

x-5 x-4 x-3 x-2 x-1 x

0 ⎦⎣

Horizontal Motion direction y0=0

Exposure pixel

x0(t)=at/T, T=constant a=velocity

Image Comm. Lab EE/NTHU 56

5.6 Estimating the Degradation Function-by modeling

• Reverse the order of integration

• The term inside the outer brackets is Fourier transform of the displacement

• Therefore

∫ ∫ ∫ ⎥⎥

⎢⎢

⎡+−−−==

∞−

T

dtdxdyvyuxjtyytxxfvuG0

00 )](2exp[)](),([),( π

),(),())]()((2exp[),(),(

))]()((2exp[),(),(

00

000

vuHvuFdttvytuxjvuFvuG

dttvytuxjvuFvuG

T

T

=+−=

+−=

π

π

0

Image Comm. Lab EE/NTHU 57

5.6 Estimating the Degradation Function-by modeling

• Assume uniform motion in x direction only, i.e.,x0(t)=at/T, y0(t)=0, a is velocity, T=exposure duration

• Simplify as

• Problem: when u=n/a, H(u, v)=0

∫ +−=T

dttvytuxjvuH0

00 ))]()((2exp[),( π

uajT

T

euauaTdtTuatj

dttuxjvuH

πππ

π

π

−=−=

−=

)sin()]/2exp[

))](2exp[),(

0

00

Image Comm. Lab EE/NTHU 58

5.6 Estimating the Degradation Function-by modeling

• If we allow y-component movement, with the motion given by y0=bt/T then the degradation function is

)())(sin()(

),( vbuajevbuavbua

TvuH +−++

= πππ

Image Comm. Lab EE/NTHU 59

5.6 Estimating the Degradation Function-by modeling

5.6 Estimating the Degradation Function-by modeling

Image Comm. Lab EE/NTHU 60

5.6 Estimating the Degradation Function-by spatial modeling

• Assume horizontal motion only (y0=0), then the blurred image is

where 0≤x≤L• Substitute τ=x–at/T

we have• Differentiation with respect to x

g’(x)=f(x)–f(x-a) or f(x)=g’(x)+f(x-a)

∫∫ −=−=TT

dtTatxfdttxxfxg00 0 )/()]([)(

ττ dfxgx

ax∫ −= )()(

a=displacement /unit time

Image Comm. Lab EE/NTHU 61

5.6 Estimating the Degradation Function-by modeling

• Let L=Ka, where K is integer, then x=z+ma where 0≦z≦a, m=Integer[x/a], and m=0,1,…K-1.

• Substitute x with z+ma

• Denote ψ(z) as the portion of the scene moves into the range 0≦z≦a during exposure, i.e., ψ(z)=f(z-a)

))1(()()( ' amzfmazgmazf −+++=+

x-5z

x-4z+1

x-3z+2

x-2z+3

x-1 z+4

xz+5

a=5φ(z)= f(x-5) f(x)= φ(z+5)

Image Comm. Lab EE/NTHU 62

5.6 Estimating the Degradation Function-by modeling

• For m=0

• For m=1

• For m=2

• Finally we have

)()()()()( '' zzgazfzgzf φ+=−+=

)()()()()()( ''' zzgazgzfazgazf φ+++=++=+

)()()()2()2( ''' zzgazgazgazf φ+++++=+

)()()(0

' zkazgmazfm

k

φ++=+ ∑=

Image Comm. Lab EE/NTHU 63

5.6 Estimating the Degradation Function-by modeling

• For x=z+ma and 0≦x≦L, we have

g(x) is known, we may estimate ψ(x) to find f(x).• Since 0≦x-ma≦a, so ψ(x-ma) is repeated K times

during the evaluation of f(x) for 0≦x≦L• Define

• So we have

)()()(0

' maxkaxgxfm

k

−+−=∑=

φ

∑=

−=m

j

jazgxf0

' )()(ˆ

)(ˆ)()( xfxfmax −=−φ

Image Comm. Lab EE/NTHU 64

5.6 Estimating the Degradation Function-by modeling

• For ka≦x≦(k+1)a, and adding the results for k=1,….K-1

where m=0 and 0≦x≦a• For large value of K (small a)

where constant A is the average of f

∑∑−

=

=

+−+=1

0

1

0

)(ˆ)()(K

k

K

k

kaxfkaxfxKφ

∑∑∑−

=

=

=

+−≈+−+=1

0

1

0

1

0

(ˆ1)(ˆ1)(1)K

k

K

k

K

k

kaxfK

AkaxfK

kaxfK

x )(φ

Image Comm. Lab EE/NTHU 65

5.6 Estimating the Degradation Function-by modeling

• For φ(x-ma)

• The restored image is

∑∑

∑−

= =

=

−−+−≈

−+−≈−

1

0 0

'

1

0

)(1

)(ˆ1)(

K

k

k

j

K

k

jamakaxgK

A

makaxfK

Amaxφ

)()()(0

' maxkaxgxfm

k

−+−=∑=

φ

Image Comm. Lab EE/NTHU 66

5.6 Estimating the Degradation Function-by modeling

Image Comm. Lab EE/NTHU 67

5.7 Inverse filtering

• With degraded image:• Our goal is to find such that

approximate g in a least square sense

nfHg +⋅=

)fH(g)fH(gfHgn

fHgnT22 ˆˆˆ

ˆ

−−=−=

⋅−=

fHˆf̂

Image Comm. Lab EE/NTHU 68

5.7 Inverse filtering

• Minimizing by using

• Or we may have

)ˆ(fn 2 J=

gHgH)(HHgHH)(Hf 1T1T1T1T −−−− ===ˆ

),(),(),(ˆ

vuvuvu

HGF =

0ˆ/)ˆ( =∂∂ ffJ

0)ˆ(2ˆ/)ˆ( =−−=∂∂ fHgHff TJ

Image Comm. Lab EE/NTHU 69

5.7 Inverse filtering

• Without noise

• We have

• With noise

• We have

• Restored image

),(),(),(),(ˆ vu

vuvuvu F

HGF ==

),(),(),( vuvuvu FHG =

),(),(),(),( vuvuvuvu NFHG +=

),(),(),(),(ˆ

vuvuvuvu

HNFF +=

)},(ˆ{),(ˆ vuyxf F-1F=

Image Comm. Lab EE/NTHU 70

5.7 Inverse filtering

u, v

1/H(u,v)u, v

F(u,v)

Image Comm. Lab EE/NTHU 71

5.7 Inverse filtering5.7 Inverse filtering

Degradation function:

H(u,.v)=e-k(u2+v2)5/6

K=0.0025, M=N=480

Image Comm. Lab EE/NTHU 72

5.8 Minimum Mean Square Error (Wiener) Filtering

• Incorporates both the degradation function and statistical characteristics of noise into restoration

• Considering both the image and noise as random processes, and find an estimate of the uncorrupted image f such that the mean square error between them is minimized

• The error measure is e2=E{(f– )2}

Image Comm. Lab EE/NTHU 73

5.8 Minimum Mean Square Error (Wiener) Filtering

• Minimizing the error we haveE{[f(x, y)– ]g(x’, y’)}=0

• For image coordinate pair (x, y) and (x’, y’), we assume the restoration filter is hR(x, y), and have

• Assume the ideal image and observed image are jointly stationary, the expectation term can be expressed as covariance function as

),(ˆ yxf

{ } { }∫ ∫ −−= βαβα ddyxhyxgyxgEyxgyxfE R ),()','(),(),(),(

∫ ∫∞

∞−−−−−=−− βαβαβα ddyxhyxKyyxxK Rggfg ),()','()','(

Image Comm. Lab EE/NTHU 74

5.8 Minimum Mean Square Error (Wiener) Filtering

• Take Fourier Transform we haveHR(u, v)=Wfg(u, v)Wgg

-1(u, v)with additive noise

Wfg(u, v)=H*(u, v)Sf(u, v)Wgg(u, v)= |H(u, v)|2 Sf (u, v)+Sη(u, v)

Image Comm. Lab EE/NTHU 75

5.8 Minimum Mean Square Error (Wiener) Filtering5.8 Minimum Mean Square Error (Wiener) Filtering

),(),(/),(),(

),(),(

1

),(),(/),(),(

),(*

),(),(),(),(

),(),(*),(ˆ

2

2

2

2

vuGvuSvuSvuH

vuHvuH

vuGvuSvuSvuH

vuH

vuGvuSvuHvuS

vuSvuHvuF

f

f

f

f

⎥⎥⎦

⎢⎢⎣

+=

⎥⎥⎦

⎢⎢⎣

+=

⎥⎥⎦

⎢⎢⎣

+=

η

η

η

Assume the image and noise are uncorrelated

Image Comm. Lab EE/NTHU 76

5.8 Minimum Mean Square Error (Wiener) Filtering

• H(u, v): the degradation function• Sη(u, v)=|N(u, v)|2=power spectrum of the noise• Sf(u, v)=|F(u, v)|2=power spectrum of the

undegraded function• If noise is zero, then it becomes an inverse filter.• For a white noise |N(u, v)|2=constant then

),(),(

),(),(

1),(ˆ2

2

vuGKvuH

vuHvuH

vuF⎥⎥⎦

⎢⎢⎣

+=

Image Comm. Lab EE/NTHU 77

5.8 Minimum Mean Square Error (Wiener) Filtering5.8 Minimum Mean Square Error (Wiener) Filtering

Image Comm. Lab EE/NTHU 78

5.8 Minimum Mean Square Error

(Wiener) Filtering

5.8 Minimum Mean Square Error

(Wiener) Filtering

Image Comm. Lab EE/NTHU 79

5.9 Constraint Least Squares Filtering

• Difficulty for Wiener filter: the power spectra of the undegraded image and the noise must be known.

• Only the mean and variance of the noise are required• Given a noisy image in vector form

with g(x, y) has a size M×N, the matrix H has dimension MN × MN and H is highly sensitive to noise.

• Base optimality of restoration on a measure of smoothness, such as the 2nd derivative of an image, i.e.,

• Find the minimum of C subject to the constraint

nfHg +⋅=

[ ]∑∑−

=

=

∇=1

0

1

0

22 ),(M

x

N

yyxfC

22ˆ ηfHg =−

Image Comm. Lab EE/NTHU 80

5.9 Constraint Least Squares Filtering

• The frequency domain solution to this optimization problem is

• P(u,v) is Fourier transform of p(x, y)

),(),(),(

),(*),( 22 vuGvuPvuU

vuHvuF⎥⎥⎦

⎢⎢⎣

+=

γ

⎥⎥⎥

⎢⎢⎢

−−−

−=

010141

010),( yxp

Image Comm. Lab EE/NTHU 81

5.9 Constraint Least Squares Filtering5.9 Constraint Least Squares Filtering

Image Comm. Lab EE/NTHU 82

5.9 Constraint Least Squares Filtering

• To compute γ by iteration as follows.• Define the residue vector r:• is a function of γ , so is r, • φ (γ)=rT r=||r||2 is a monotonically increasing

function of γ• Adjust γ so that ||r||2 = || η ||2 ±a

where a is a accuracy factor• If ||r||2 = || η ||2 then the best solution is found.

fHgr ˆ−=),(ˆ vuF

Image Comm. Lab EE/NTHU 83

5.9 Constraint Least Squares Filtering

• Because φ (γ) is monotonic, finding γ is not difficult.

1. Specify an initial γ2. Compute ||r||2

3. if ||r||2 = || η ||2 ±a is satisfied Stop4. if ||r||2 < || η ||2 –a increase γ and go to step 2.5. if ||r||2 > || η ||2 –a decrease γ and go to step 2.

Image Comm. Lab EE/NTHU 84

5.9 Constraint Least Squares Filtering

• To compute the ||r||2 and || η ||2

• The vector from can also be rewritten as

• Compute the Inverse Fourier transform of R(u, v)

• Consider the variance of the noise over the entire image, using the sample-average method

• So we have || η ||2=MN[σ2η - mη]

),(ˆ),(),(),( vuFvuHvuGvuR −=

∑∑−

=

=

=1

0

1

0

22 ),(M

x

N

yyxrr

[ ]∑∑−

=

=

−=1

0

1

0

2 ),(1 M

x

N

ymyx

MN ηη ησ ∑∑−

=

=

=1

0

1

0),(1 M

x

N

yyx

MNm ηη

Image Comm. Lab EE/NTHU 85

5.9 Constraint Least Squares Filtering5.9 Constraint Least Squares Filtering

Image Comm. Lab EE/NTHU 86

5.10 Geometric Mean Filter5.10 Geometric Mean Filter

),(

),(),(

),(

),(*),(

),(*),(ˆ

1

22 vuG

vuSvuS

vuH

vuHvuH

vuHvuF

f

α

η

α

β

⎥⎥⎥⎥⎥

⎢⎢⎢⎢⎢

⎥⎥⎦

⎢⎢⎣

⎡+

⎥⎥⎦

⎢⎢⎣

⎡=

α, β are positive real constant

α=1 reduces to inverse filter

α=0 becomes parametric Wiener filter

α=0, β=1 standard Wiener filter

α=1/2, β=1 spectrum equalization filter

Generalized form of Wiener filter:

Image Comm. Lab EE/NTHU 87

5.11 Geometric Transformation

• It is also called Rubber sheet transform, which consists of two operations:

1) Spatial transformation: rearrangement of the pixels (locations) on the image.

2) Gray-level interpolation: assignment of gray levels to pixels in the spatially transformed image.

Image Comm. Lab EE/NTHU 88

5.11 Geometric Transformation-Geometric Distortion

Image Comm. Lab EE/NTHU 89

5.11 Geometric TransformationSpatial transformation

5.11 Geometric TransformationSpatial transformation

Mapping function T : maps a point from screen space to a point in texture space

Screen space Texture space

Image Comm. Lab EE/NTHU 90

5.11 Geometric TransformationSpatial transformation

• The transformation T may be expressed asx’=r(x, y)=c1x+c2y+c3xy+c4

y’=s(x, y)=c5x+c6y+c7xy+c8

Find the tiepoints, which are subset of pixels whose locations in the distorted image (in screen space) and the corrected images (in texture space) are known precisely.

Image Comm. Lab EE/NTHU 91

5.11 Geometric Transformation-Spatial transformation

• General Transformation:

where

x y w u v w T' ' ' = 1

⎥⎥⎥

⎢⎢⎢

⎡=

333231

232221

131211

1

SSSSSSSSS

T

Image Comm. Lab EE/NTHU 92

5.11 Geometric Transformation-Spatial transformation

• T1 is said to be one-to-one if:(a) The inverse transformation of T1 always exists.(b) With T1, one point in screen space produces only

one point in texture space(c) Through T1

-1, each point in texture space can find the corresponding point in screen space.

=> that is the determinate of T1 is nonzero.

Image Comm. Lab EE/NTHU 93

5.11 Geometric Transformation-Spatial transformation

• The 3×3 transformation matrix can be best understood by partitioning it into four separate sections.

• Rotation:

• Translation: • Perspective transformation: • Scaling:

⎥⎦

⎤⎢⎣

⎡=

2221

12112 SS

SST

S S31 32

S S T13 23

S33

Image Comm. Lab EE/NTHU 94

5.11 Geometric Transformation-Spatial transformation

• The general representation of an Affine transformation is :

⎥⎥⎥

⎢⎢⎢

⎡=

100

]1[]1[

3231

2221

1211

aaaaaa

vuyx

It is a planar mapping, that can be used to map an input triangle to any arbitrary triangle at the output

Image Comm. Lab EE/NTHU 95

5.11 Geometric Transformation-Spatial transformation

5.11 Geometric Transformation-Spatial transformation

Affine Transformation

Image Comm. Lab EE/NTHU 96

5.11 Geometric Transformation-Spatial transformation

5.11 Geometric Transformation-Spatial transformation

Perspective Transformation

⎥⎥⎥

⎢⎢⎢

⎡=′′′

333231

232221

131211

],,[],,[aaaaaaaaa

wvuwyx

332313

312111/avauaavaua

wxx++++

=′′=

and let w=1

332313

322212/avauaavaua

wyy++++

=′′=

If then it becomes Affine Transformation02313 == aa

Image Comm. Lab EE/NTHU 97

5.11 Geometric Transformation-Spatial transformation

• Perspective projection

X axis

Y axis

P(X,Y,Z)

Z

D

x

YZ axis

(X ,Y ,0)P P

Center of prjection

Projection Plane

Image Comm. Lab EE/NTHU 98

5.11 Geometric Transformation-Spatial transformation

• Any position along the projection line is (α,β,γ) is

where δ=Z/(Z+D), and 0<δ<1

α δβ δγ δ

= −= −= − +

X XY YZ Z D( )

)1

1(+

⋅=

DZ

XX p )1

1(+

⋅=

DZ

YY p

Image Comm. Lab EE/NTHU 99

5.11 Geometric Transformation-Spatial transformation

• Perspective Transformation

Image Comm. Lab EE/NTHU 100

5.11 Geometric Transformation-Spatial transformation

• Bilinear transformation is defined as

where W=[u, v, 1] and Z=[x, y, 1]

and are constant.

ZT → W: TWZ ⋅=

⎥⎥⎥

⎢⎢⎢

⎡+

+=

100

00

232

311

babuaa

vbbaT

3210,3210 and,,,,,, bbbbaaaa

Image Comm. Lab EE/NTHU 101

5.11 Geometric Transformation-Spatial transformation

Bilinear distortion for reference quadrangle and points.

P

P

P

P

1

2

3

4

P1

P2 P3

P4' '

'(0,1) (1,1)

(1,0)(0,0)

u

v

Y

X

T

T -1

Image Comm. Lab EE/NTHU 102

5.11 Geometric Transformation-gray-level interpolation

5.11 Geometric Transformation-gray-level interpolation

Image Comm. Lab EE/NTHU 103

5.11 Geometric Transformation-gray-level interpolation

• Zero-order interpolation: the simplest one.• Bilinear interpolation: using four neighbors to do

the interpolation for the gray level of non-integer point at (x’, y’):

v(x’, y’)=a+bx’+cy’+dx’y’• The four neighbors located at (xi, yi), with known

gray level vii i=1 and 2, then we have four different equations as

vii=a+bxi+cyi+d xi yi• Solving the above four equations for the four

parameters, a, b, c, and d.

Image Comm. Lab EE/NTHU 104

5.11 Geometric Transformation5.11 Geometric Transformation

Image Comm. Lab EE/NTHU 105

5.11 Geometric Transformation-gray-level interpolation

• resampling

Image Reconstruction

Spatial Transformation

Input Samples Output samples

Output Grid

Resampling Grid

Reconstructed Signal

Image Comm. Lab EE/NTHU 106

5.11 Geometric Transformation-gray-level interpolation

• resamplingf(u)

uDiscrete Input Discrete output

g(x)

x

f (u)c g (x)c

u xRecostructed Input

Warping Prefiltering

x=T(u) h(x)

Reconstructb(x)Sample s(x)

Warped Input Continuous Output

g (x)c'

x

Image Comm. Lab EE/NTHU 107

5.11 Geometric Transformation-gray-level interpolation

Ideal warping resampling

∫ ∑

−⎥⎥⎦

⎢⎢⎣

⎡−=

−=

∈∗=′=

Zj

Zj

c

cc

jxjf

dttxhjtTbjf

dttxhtTf

Zxh(x(x)gxgxg

),()(=

)())(()(

)())((

for ) )( )(

1

1

ψ

∫ −−= − dttxhjtTbjx )())((),( 1ψwhere

Image Comm. Lab EE/NTHU 108

5.11 Geometric Transformation-gray-level interpolation

Table : Elements of ideal resampling.

f u u Z( ), ∈

f u f u b u f j b u jcj Z

( ) ( )* ( ) ( ) ( )= = −∈∑

g x f T xc c( ) ( ( ))= −1

∫ −==′ dttxhtgxhxgxg ccc )()()(*)()(

g x g x s xc( ) ' ( ) ( )=

Stage Mathematical definition

Discrete Input

Reconstructed Input

Warped Signal

Continuous Output

Discrete Output

Image Comm. Lab EE/NTHU 109

5.11 Geometric Transformation-gray-level interpolation

• Cubic convolution interpolation:using much larger number of neighbors (i.e., 16) to obtain a smoother interpolation.

• Ideal Interpolation:

Reconstruction

Filter

Warped

PrefilterSampler

WarpResampling Filter

f(u) f (u)c g (u)c'

s(T (x))-1

s(x)

g(x)b(u) h(u)

Image Comm. Lab EE/NTHU 110

5.11 Geometric Transformation5.11 Geometric Transformation