Image Comm. Lab EE/NTHU 1
Chapter 5Image Restoration
• Degraded digital image restoration– Spatial domain processing
Additive noise– Frequency domain
Blurred image
Image Comm. Lab EE/NTHU 2
5. 1 Model of Image degradation and restoration5. 1 Model of Image degradation and restoration
•g(x, y)=h(x, y)∗f(x, y)+η(x, y)•G(u, v)=H(u, v)F(u, v)+N(u, v)
Image Comm. Lab EE/NTHU 3
5.2 Noise Model
• Spatial and frequency property of noise– White noise (random noise)– Gaussian noise
– μ: mean ; σ :variance– 70% [(μ-σ), (μ+σ)]– 95 % [(μ-2σ), (μ+2σ)]
22 2/)(
21)( σμ
σπ−−= zezp
Image Comm. Lab EE/NTHU 4
5.2 Noise Model
• Rayleigh noiseThe PDF is
• μ=a+(πb/4)1/2
• σ =b(4-π)/4
⎪⎩
⎪⎨⎧
<≥−=
−−
azforazforeaz
bzpbaz
0
)(2)(
/)( 2
Image Comm. Lab EE/NTHU 5
5.2 Noise Model
• Erlang (gamma) noise
• μ=b/a; σ =b/a2
• Exponential noise
• μ=1/a; σ =1/a2
⎪⎩
⎪⎨⎧
<
≥−=
−−
00
0)!1()(
1
zfor
zforeb
zazp
azbb
⎩⎨⎧
<≥
=−
000
)(zforzforae
zpaz
Image Comm. Lab EE/NTHU 6
5.2 Noise Model
• Uniform noise
• μ=(a+b)/2; σ =(b-a)2/12• Impulse noise (salt and pepper noise)
⎪⎩
⎪⎨⎧ ≤≤
−=otherwise
bzaforabzp
0
1)(
⎪⎩
⎪⎨
⎧==
=otherwise
bzforazfor
PP
zp b
a
0)(
Image Comm. Lab EE/NTHU 7
5.2 Noise Model5.2 Noise Model
Image Comm. Lab EE/NTHU 8
5.2 Noise Model5.2 Noise Model
Image Comm. Lab EE/NTHU 9
5.2 Noise Model5.2 Noise Model
Image Comm. Lab EE/NTHU 10
5.2 Noise Model5.2 Noise Model
Image Comm. Lab EE/NTHU 11
5.2.3 Noise Model-periodic noise
• Periodic noise is due to the electrical or electromechanical interference during image acquisition.
• Can be estimated through the inspection of the Fourier spectrum of the image.
Image Comm. Lab EE/NTHU 12
5.2.3 Noise Model-periodic noise5.2.3 Noise Model-periodic noise
Image Comm. Lab EE/NTHU 13
5.2.4 Noise Model- Noise parameters estimation
• Use a optical sensor to capture the image of a solid gray board that is illuminated uniformly.
• The resulting image is a good indicator of system noise.
• Find the mean μ and standard deviation σ of the histogram of the resulting image.
• From the μ and σ we can calculate the a and b, the parameter of that specific noise distribution.
Image Comm. Lab EE/NTHU 14
5.2 Noise Model5.2 Noise Model
Image Comm. Lab EE/NTHU 15
5.3 Restoration using spatial filter
• Mean filters– Arithmetic mean filter
– Geometric mean filter
∑∈
=xySts
tsgmn
yxf),(
),(1),(ˆ
mn
Sts xy
tsgyxf/1
),(
),(),(ˆ⎥⎥⎦
⎤
⎢⎢⎣
⎡= ∏
∈
Image Comm. Lab EE/NTHU 16
5.3 Restoration using spatial filter
– Harmonic mean filter is good for salt noise not for pepper noise.
– Contra-harmonic filter: Q>0 reduce pepper noise Q<0 reduce salt noise.
∑∈
=
xySts tsg
mnyxf
),( ),(
1),(
ˆ
∑
∑
∈
∈
+
=
xy
xy
Sts
QSts
Q
tsg
tsgyxf
),(
),(
1
),(
),(),(ˆ
Image Comm. Lab EE/NTHU 17
5.3 Restoration using spatial filter5.3 Restoration using spatial filter
Image Comm. Lab EE/NTHU 18
5.3 Restoration using spatial filter5.3 Restoration using spatial filter
Image Comm. Lab EE/NTHU 19
5.3 Restoration using spatial filter5.3 Restoration using spatial filter
Image Comm. Lab EE/NTHU 20
5.3 Restoration using spatial filter
• Order-Statistics filtersMedian filterMax filter: find the brightest points to reduce the pepper noise
Min filter: find the darkest point to reduce the salt noise
Midpoint filter
{ }),(),(ˆ),(
tsgmedianyxfxySts ∈
=
{ }),(max),(ˆ),(
tsgyxfxySts ∈
=
{ }),(min),(ˆ),(
tsgyxfxySts ∈
=
{ } { }⎥⎦⎤
⎢⎣⎡ +=
∈∈),(max),(min
21),(ˆ
),(),(tsgtsgyxf
xyxy StsSts
Image Comm. Lab EE/NTHU 21
5.3 Restoration using spatial filter
• Alpha-trimmed mean filterSuppose delete d/2 lowest and d/2 highest gray-level value in the neighborhood of (s, t) and average the rest.
where d=0 ~ mn-1
∑∈−
=xySts
r tsgdmn
yxf),(
),(1),(ˆ
Image Comm. Lab EE/NTHU 22
5.3 Restoration using spatial filter5.3 Restoration using spatial filter
Image Comm. Lab EE/NTHU 23
5.3 Restoration using spatial filter5.3 Restoration using spatial filter
Image Comm. Lab EE/NTHU 24
5.3 Restoration using spatial filter
5.3 Restoration using spatial filter
Image Comm. Lab EE/NTHU 25
5.3 Restoration using spatial filter-Adaptive local noise reduction filter
Measurements for local region Sxy centered at (x, y)(1) noisy image at (x, y): g(x, y) with variance σ2
η(2) The local mean mL in Sxy(3) The local variance σ2
L(4) The local variance of noise is high σ2
L≥ σ2η
Conditions: Assume additive noise(a) Zero-noise case: If σ2
η =0 then f(x, y)=g(x, y)(b) Edges: If σ2
L > σ2η then f(x, y)≈g(x, y)
(c) If σ2L = σ2
η then f(x,y)= mL, the arithmetic mean
In practice, σ2η is unknown, so a test is built before filtering
]),([),(),(ˆ2
2
LL
myxgyxgyxf −−=σση
Image Comm. Lab EE/NTHU 26
5.3 Restoration using spatial filter5.3 Restoration using spatial filter
Image Comm. Lab EE/NTHU 27
5.3 Restoration using spatial filter
• Adaptive median filter can handle impulse noise with larger probability (Pa and Pb are large).Increasing(change) window size during operation
• Notations:zmin=minimum gray-level value in Sxyzmax=maximum gray-level value in Sxyzmed=median gray-level value in Sxyzxy= gray-level at (x,y)Smax=maximum allowed size of Sxy
Image Comm. Lab EE/NTHU 28
5.3 Restoration using spatial filter
• The adaptive median filter algorithm works in two levels: A and B
• Level A: A1=zmed-zmin, A2=zmed-zmaxIf A1>0 AND A2<0 goto level Belse increase the window sizeIf window size ≤ Smax repeat level Aelse output zxy
Level B: B1=zxy-zmin, B2=zxy-zmaxIf B1>0 AND B2<0, output zxyElse output zmed
Image Comm. Lab EE/NTHU 29
5 .3 Restoration using spatial filter5 .3 Restoration using spatial filter
Image Comm. Lab EE/NTHU 30
5.4 Restoration using frequency-domain filter5.4.1 band-reject filter
• Ideal band reject filter
• Butterworth band reject filter
⎪⎩
⎪⎨
⎧
+>+≤≤−
−<=
2/),(2/),(2/
2/),(
101
),(
0
00
0
WDvuDifWDvuDWDif
WDvuDifvuH
n
DvuDWvuD
vuH 2
20
2 ),(),(1
1),(
⎥⎦
⎤⎢⎣
⎡−
+
=
Image Comm. Lab EE/NTHU 31
5.4.1 Restoration using frequency-domain filter
• Gaussian band reject filter22
02
),(),(
21
1),( ⎥⎥⎦
⎤
⎢⎢⎣
⎡ −−
−= WvuDDvuD
evuH
Image Comm. Lab EE/NTHU 32
5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter
Image Comm. Lab EE/NTHU 33
5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter
Image Comm. Lab EE/NTHU 34
5.4.2 Bandpass filter
• Obtained form band reject filterHbp(u, v)=1-Hbr(u, v)
• To isolated an image of certain frequency band.
Image Comm. Lab EE/NTHU 35
5.4 Restoration using frequency-domain filterBandpass filter
5.4 Restoration using frequency-domain filterBandpass filter
Image Comm. Lab EE/NTHU 36
5.4.3 Notch filter
• Notch filter rejects (passes) frequencies in predefined neighborhoods about a center frequency. For notch rejects filter as
where⎩⎨⎧ ≤≤
=otherwise
DvuDorDvuDifvuH
1),(),(0
),( 0201
[ ] 2/120
201 )2/()2/(),( vNvuMuvuD −−+−−=
[ ] 2/120
202 )2/()2/(),( vNvuMuvuD +−++−=
Image Comm. Lab EE/NTHU 37
5.4 Restoration using frequency-domain filter-Notch filter
• Butterworth notch reject filter
• Gaussian notch reject filter
• Hnp(u, v)=1-Hnr(u, v)• It becomes a low-pass filter when u0=v0=0
n
vuDvuDD
vuH
⎥⎦
⎤⎢⎣
⎡+
=
),(),(1
1),(
21
20
⎥⎥⎦
⎤
⎢⎢⎣
⎡−
−=20
21 ),(),(21
1),( DvuDvuD
evuH
Image Comm. Lab EE/NTHU 38
5.4 Restoration using frequency-domain filterNotch filter
5.4 Restoration using frequency-domain filterNotch filter
Image Comm. Lab EE/NTHU 39
5.4 Restoration using frequency-domain filter
Notch filter
5.4 Restoration using frequency-domain filter
Notch filter
Use 1-D Notch pass filter to find the horizontal ripple noise
Image Comm. Lab EE/NTHU 40
5.4.4 Optimal Notch filter
• Clearly defined interference is not common.• Images from electro-optical scanner are corrupted by
periodic degradation.• Several interference components are present.• Place a notch pass filter H(u, v) at the location of
each spike, i.e., N(u, v)=H(u, v)G(u, v), where G(u, v)is the corrupted image.
• η(x, y)=F-1{H(u, v)G(u, v)}.• The effect of components not present in the estimate
of η(x, y) can be minimized.
Image Comm. Lab EE/NTHU 41
5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter
Image Comm. Lab EE/NTHU 42
5.4.4 Optimal Notch filter
• The restored image is • The weighting function w(x, y) is found so that the
variance of is minimized for a selected neighborhood.• One way is to select w(x, y) so that the variance of the
estimate is minimized over a small local neighborhood of size (2a+1, 2b+1).
• ∂σ2 (x, y) /∂ w(x, y)=0 → to select w(x, y)
),(),(),(),(ˆ yxyxwyxgyxf η−=
),(ˆ yxf
[ ]22 ),(ˆ),(ˆ)12)(12(
1),( ∑∑−= −=
−++++
=b
bt
a
as
yxftysxfba
yxσ
Image Comm. Lab EE/NTHU 43
5.4.4 Restoration using frequency-domain filter5.4.4 Restoration using frequency-domain filter
a=b=15
Image Comm. Lab EE/NTHU 44
5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter
Image Comm. Lab EE/NTHU 45
5.4 Restoration using frequency-domain filter5.4 Restoration using frequency-domain filter
),(),(),(),(ˆ yxyxwyxgyxf η−=
Image Comm. Lab EE/NTHU 46
5.5 Linear Position Invariant Degradation
• The System Model
Hf(x, y)
η(x, y)
g(x, y)
g(x, y): the degraded image
f(x, y) : the original image
η(x, y): additive noise
H: System function which is a linear operator
Image Comm. Lab EE/NTHU 47
5.5 Linear Position Invariant Degradation
g(x, y)=H[f(x, y)]+η(x, y)Assume H is linear and η(x, y)=0If g(x, y)=H[f(x, y)] is position invariant then
H[f(x-α, y-β)]=g(x-α, y-β)g(x, y)=H[f(x,y)]+ η(x, y)
= +η(x, y)=h(x, y)*f(x, y)+ η(x, y)∫∫ −− βαβαβα ddyxhf ),(),(
Image Comm. Lab EE/NTHU 48
5.6 Estimating the Degradation Function
• To estimate the degradation functionby Observationby Experimentationby Mathematical modeling
(Blind deconvolution)
Image Comm. Lab EE/NTHU 49
5.6 Estimating the Degradation Function
• By observation: construct an unblurredimage of some strong signal content.
• Let the observed image is gs(x, y)• The constructed image is• Assume the noise is negligible• Find the degradation function H(u, v) which
is similar to
),(~),(),(
vuFvuGvuH
s
ss =
),(~ yxfs
Image Comm. Lab EE/NTHU 50
5.6 Estimating the Degradation Function
• By experimentation: to obtain the impulse response of the degradation by imaging an impulse (small dot of light) using the same system setting.
• The Four Transform of an impulse is a constant A
• Let g(x, y) be the observed image.• Find the degradation function as
H(u, v)=G(u, v)/A
Image Comm. Lab EE/NTHU 51
5.6 Estimating the Degradation Function5.6 Estimating the Degradation Function
Image Comm. Lab EE/NTHU 52
5.6 Estimating the Degradation Function -by modeling
• By Modeling: through experience• The physical characteristic of turbulence as
H(u, v)=e-k(u2+v2)5/6
• Where k is a constant which depend on the nature of turbulence.
• It is similar to the Gaussian Low pass. k=0.0025 (severe turbulence)k=0.001k=0.00025 (low turbulence)
Image Comm. Lab EE/NTHU 53
5.6 Estimating the Degradation Function5.6 Estimating the Degradation Function
Image Comm. Lab EE/NTHU 54
5.6 Estimating the Degradation Function -by modeling
• Through mathematical modelImage f(x, y) is blurred by uniform motion.
• x0(t) and y0(t) are the time varying component in x, y directions.
• The total exposure at any point of the film is obtained by integrating the instantaneous exposure over the time interval during which the shutter is opened.
Image Comm. Lab EE/NTHU 55
5.6 Estimating the Degradation Function-by modeling
• Assume T is duration of exposure, the blurred image g(x, y) is
• Fourier Transform of g(x, y) is
∫ −−=T
dttyytxxfyxg0
00 )](),([),(
dxdyvyuxjxpedttyytxxf
dxdyvyuxjyxgvuG
T
)](2[)](),([
)](2exp[),(),(
00 +−⎥⎥⎤
⎢⎢⎡
−−=
+−=
∫∫ ∫
∫ ∫
π
π
x-5 x-4 x-3 x-2 x-1 x
0 ⎦⎣
Horizontal Motion direction y0=0
Exposure pixel
x0(t)=at/T, T=constant a=velocity
Image Comm. Lab EE/NTHU 56
5.6 Estimating the Degradation Function-by modeling
• Reverse the order of integration
• The term inside the outer brackets is Fourier transform of the displacement
• Therefore
∫ ∫ ∫ ⎥⎥
⎦
⎤
⎢⎢
⎣
⎡+−−−==
∞
∞−
T
dtdxdyvyuxjtyytxxfvuG0
00 )](2exp[)](),([),( π
),(),())]()((2exp[),(),(
))]()((2exp[),(),(
00
000
vuHvuFdttvytuxjvuFvuG
dttvytuxjvuFvuG
T
T
=+−=
+−=
∫
∫
π
π
0
Image Comm. Lab EE/NTHU 57
5.6 Estimating the Degradation Function-by modeling
• Assume uniform motion in x direction only, i.e.,x0(t)=at/T, y0(t)=0, a is velocity, T=exposure duration
• Simplify as
• Problem: when u=n/a, H(u, v)=0
∫ +−=T
dttvytuxjvuH0
00 ))]()((2exp[),( π
uajT
T
euauaTdtTuatj
dttuxjvuH
πππ
π
π
−=−=
−=
∫
∫
)sin()]/2exp[
))](2exp[),(
0
00
Image Comm. Lab EE/NTHU 58
5.6 Estimating the Degradation Function-by modeling
• If we allow y-component movement, with the motion given by y0=bt/T then the degradation function is
)())(sin()(
),( vbuajevbuavbua
TvuH +−++
= πππ
Image Comm. Lab EE/NTHU 59
5.6 Estimating the Degradation Function-by modeling
5.6 Estimating the Degradation Function-by modeling
Image Comm. Lab EE/NTHU 60
5.6 Estimating the Degradation Function-by spatial modeling
• Assume horizontal motion only (y0=0), then the blurred image is
where 0≤x≤L• Substitute τ=x–at/T
we have• Differentiation with respect to x
g’(x)=f(x)–f(x-a) or f(x)=g’(x)+f(x-a)
∫∫ −=−=TT
dtTatxfdttxxfxg00 0 )/()]([)(
ττ dfxgx
ax∫ −= )()(
a=displacement /unit time
Image Comm. Lab EE/NTHU 61
5.6 Estimating the Degradation Function-by modeling
• Let L=Ka, where K is integer, then x=z+ma where 0≦z≦a, m=Integer[x/a], and m=0,1,…K-1.
• Substitute x with z+ma
• Denote ψ(z) as the portion of the scene moves into the range 0≦z≦a during exposure, i.e., ψ(z)=f(z-a)
))1(()()( ' amzfmazgmazf −+++=+
x-5z
x-4z+1
x-3z+2
x-2z+3
x-1 z+4
xz+5
a=5φ(z)= f(x-5) f(x)= φ(z+5)
Image Comm. Lab EE/NTHU 62
5.6 Estimating the Degradation Function-by modeling
• For m=0
• For m=1
• For m=2
• Finally we have
)()()()()( '' zzgazfzgzf φ+=−+=
)()()()()()( ''' zzgazgzfazgazf φ+++=++=+
)()()()2()2( ''' zzgazgazgazf φ+++++=+
)()()(0
' zkazgmazfm
k
φ++=+ ∑=
Image Comm. Lab EE/NTHU 63
5.6 Estimating the Degradation Function-by modeling
• For x=z+ma and 0≦x≦L, we have
g(x) is known, we may estimate ψ(x) to find f(x).• Since 0≦x-ma≦a, so ψ(x-ma) is repeated K times
during the evaluation of f(x) for 0≦x≦L• Define
• So we have
)()()(0
' maxkaxgxfm
k
−+−=∑=
φ
∑=
−=m
j
jazgxf0
' )()(ˆ
)(ˆ)()( xfxfmax −=−φ
Image Comm. Lab EE/NTHU 64
5.6 Estimating the Degradation Function-by modeling
• For ka≦x≦(k+1)a, and adding the results for k=1,….K-1
where m=0 and 0≦x≦a• For large value of K (small a)
where constant A is the average of f
∑∑−
=
−
=
+−+=1
0
1
0
)(ˆ)()(K
k
K
k
kaxfkaxfxKφ
∑∑∑−
=
−
=
−
=
+−≈+−+=1
0
1
0
1
0
(ˆ1)(ˆ1)(1)K
k
K
k
K
k
kaxfK
AkaxfK
kaxfK
x )(φ
Image Comm. Lab EE/NTHU 65
5.6 Estimating the Degradation Function-by modeling
• For φ(x-ma)
• The restored image is
∑∑
∑−
= =
−
=
−−+−≈
−+−≈−
1
0 0
'
1
0
)(1
)(ˆ1)(
K
k
k
j
K
k
jamakaxgK
A
makaxfK
Amaxφ
)()()(0
' maxkaxgxfm
k
−+−=∑=
φ
Image Comm. Lab EE/NTHU 66
5.6 Estimating the Degradation Function-by modeling
Image Comm. Lab EE/NTHU 67
5.7 Inverse filtering
• With degraded image:• Our goal is to find such that
approximate g in a least square sense
nfHg +⋅=
)fH(g)fH(gfHgn
fHgnT22 ˆˆˆ
ˆ
−−=−=
⋅−=
fHˆf̂
Image Comm. Lab EE/NTHU 68
5.7 Inverse filtering
• Minimizing by using
• Or we may have
)ˆ(fn 2 J=
gHgH)(HHgHH)(Hf 1T1T1T1T −−−− ===ˆ
),(),(),(ˆ
vuvuvu
HGF =
0ˆ/)ˆ( =∂∂ ffJ
0)ˆ(2ˆ/)ˆ( =−−=∂∂ fHgHff TJ
Image Comm. Lab EE/NTHU 69
5.7 Inverse filtering
• Without noise
• We have
• With noise
• We have
• Restored image
),(),(),(),(ˆ vu
vuvuvu F
HGF ==
),(),(),( vuvuvu FHG =
),(),(),(),( vuvuvuvu NFHG +=
),(),(),(),(ˆ
vuvuvuvu
HNFF +=
)},(ˆ{),(ˆ vuyxf F-1F=
Image Comm. Lab EE/NTHU 70
5.7 Inverse filtering
u, v
1/H(u,v)u, v
F(u,v)
Image Comm. Lab EE/NTHU 71
5.7 Inverse filtering5.7 Inverse filtering
Degradation function:
H(u,.v)=e-k(u2+v2)5/6
K=0.0025, M=N=480
Image Comm. Lab EE/NTHU 72
5.8 Minimum Mean Square Error (Wiener) Filtering
• Incorporates both the degradation function and statistical characteristics of noise into restoration
• Considering both the image and noise as random processes, and find an estimate of the uncorrupted image f such that the mean square error between them is minimized
• The error measure is e2=E{(f– )2}
f̂
f̂
Image Comm. Lab EE/NTHU 73
5.8 Minimum Mean Square Error (Wiener) Filtering
• Minimizing the error we haveE{[f(x, y)– ]g(x’, y’)}=0
• For image coordinate pair (x, y) and (x’, y’), we assume the restoration filter is hR(x, y), and have
• Assume the ideal image and observed image are jointly stationary, the expectation term can be expressed as covariance function as
),(ˆ yxf
{ } { }∫ ∫ −−= βαβα ddyxhyxgyxgEyxgyxfE R ),()','(),(),(),(
∫ ∫∞
∞−−−−−=−− βαβαβα ddyxhyxKyyxxK Rggfg ),()','()','(
Image Comm. Lab EE/NTHU 74
5.8 Minimum Mean Square Error (Wiener) Filtering
• Take Fourier Transform we haveHR(u, v)=Wfg(u, v)Wgg
-1(u, v)with additive noise
Wfg(u, v)=H*(u, v)Sf(u, v)Wgg(u, v)= |H(u, v)|2 Sf (u, v)+Sη(u, v)
Image Comm. Lab EE/NTHU 75
5.8 Minimum Mean Square Error (Wiener) Filtering5.8 Minimum Mean Square Error (Wiener) Filtering
),(),(/),(),(
),(),(
1
),(),(/),(),(
),(*
),(),(),(),(
),(),(*),(ˆ
2
2
2
2
vuGvuSvuSvuH
vuHvuH
vuGvuSvuSvuH
vuH
vuGvuSvuHvuS
vuSvuHvuF
f
f
f
f
⎥⎥⎦
⎤
⎢⎢⎣
⎡
+=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
+=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
+=
η
η
η
Assume the image and noise are uncorrelated
Image Comm. Lab EE/NTHU 76
5.8 Minimum Mean Square Error (Wiener) Filtering
• H(u, v): the degradation function• Sη(u, v)=|N(u, v)|2=power spectrum of the noise• Sf(u, v)=|F(u, v)|2=power spectrum of the
undegraded function• If noise is zero, then it becomes an inverse filter.• For a white noise |N(u, v)|2=constant then
),(),(
),(),(
1),(ˆ2
2
vuGKvuH
vuHvuH
vuF⎥⎥⎦
⎤
⎢⎢⎣
⎡
+=
Image Comm. Lab EE/NTHU 77
5.8 Minimum Mean Square Error (Wiener) Filtering5.8 Minimum Mean Square Error (Wiener) Filtering
Image Comm. Lab EE/NTHU 78
5.8 Minimum Mean Square Error
(Wiener) Filtering
5.8 Minimum Mean Square Error
(Wiener) Filtering
Image Comm. Lab EE/NTHU 79
5.9 Constraint Least Squares Filtering
• Difficulty for Wiener filter: the power spectra of the undegraded image and the noise must be known.
• Only the mean and variance of the noise are required• Given a noisy image in vector form
with g(x, y) has a size M×N, the matrix H has dimension MN × MN and H is highly sensitive to noise.
• Base optimality of restoration on a measure of smoothness, such as the 2nd derivative of an image, i.e.,
• Find the minimum of C subject to the constraint
nfHg +⋅=
[ ]∑∑−
=
−
=
∇=1
0
1
0
22 ),(M
x
N
yyxfC
22ˆ ηfHg =−
Image Comm. Lab EE/NTHU 80
5.9 Constraint Least Squares Filtering
• The frequency domain solution to this optimization problem is
• P(u,v) is Fourier transform of p(x, y)
),(),(),(
),(*),( 22 vuGvuPvuU
vuHvuF⎥⎥⎦
⎤
⎢⎢⎣
⎡
+=
γ
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−
−=
010141
010),( yxp
Image Comm. Lab EE/NTHU 81
5.9 Constraint Least Squares Filtering5.9 Constraint Least Squares Filtering
Image Comm. Lab EE/NTHU 82
5.9 Constraint Least Squares Filtering
• To compute γ by iteration as follows.• Define the residue vector r:• is a function of γ , so is r, • φ (γ)=rT r=||r||2 is a monotonically increasing
function of γ• Adjust γ so that ||r||2 = || η ||2 ±a
where a is a accuracy factor• If ||r||2 = || η ||2 then the best solution is found.
fHgr ˆ−=),(ˆ vuF
Image Comm. Lab EE/NTHU 83
5.9 Constraint Least Squares Filtering
• Because φ (γ) is monotonic, finding γ is not difficult.
1. Specify an initial γ2. Compute ||r||2
3. if ||r||2 = || η ||2 ±a is satisfied Stop4. if ||r||2 < || η ||2 –a increase γ and go to step 2.5. if ||r||2 > || η ||2 –a decrease γ and go to step 2.
Image Comm. Lab EE/NTHU 84
5.9 Constraint Least Squares Filtering
• To compute the ||r||2 and || η ||2
• The vector from can also be rewritten as
• Compute the Inverse Fourier transform of R(u, v)
• Consider the variance of the noise over the entire image, using the sample-average method
• So we have || η ||2=MN[σ2η - mη]
),(ˆ),(),(),( vuFvuHvuGvuR −=
∑∑−
=
−
=
=1
0
1
0
22 ),(M
x
N
yyxrr
[ ]∑∑−
=
−
=
−=1
0
1
0
2 ),(1 M
x
N
ymyx
MN ηη ησ ∑∑−
=
−
=
=1
0
1
0),(1 M
x
N
yyx
MNm ηη
Image Comm. Lab EE/NTHU 85
5.9 Constraint Least Squares Filtering5.9 Constraint Least Squares Filtering
Image Comm. Lab EE/NTHU 86
5.10 Geometric Mean Filter5.10 Geometric Mean Filter
),(
),(),(
),(
),(*),(
),(*),(ˆ
1
22 vuG
vuSvuS
vuH
vuHvuH
vuHvuF
f
α
η
α
β
−
⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡+
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
α, β are positive real constant
α=1 reduces to inverse filter
α=0 becomes parametric Wiener filter
α=0, β=1 standard Wiener filter
α=1/2, β=1 spectrum equalization filter
Generalized form of Wiener filter:
Image Comm. Lab EE/NTHU 87
5.11 Geometric Transformation
• It is also called Rubber sheet transform, which consists of two operations:
1) Spatial transformation: rearrangement of the pixels (locations) on the image.
2) Gray-level interpolation: assignment of gray levels to pixels in the spatially transformed image.
Image Comm. Lab EE/NTHU 88
5.11 Geometric Transformation-Geometric Distortion
Image Comm. Lab EE/NTHU 89
5.11 Geometric TransformationSpatial transformation
5.11 Geometric TransformationSpatial transformation
Mapping function T : maps a point from screen space to a point in texture space
Screen space Texture space
Image Comm. Lab EE/NTHU 90
5.11 Geometric TransformationSpatial transformation
• The transformation T may be expressed asx’=r(x, y)=c1x+c2y+c3xy+c4
y’=s(x, y)=c5x+c6y+c7xy+c8
Find the tiepoints, which are subset of pixels whose locations in the distorted image (in screen space) and the corrected images (in texture space) are known precisely.
Image Comm. Lab EE/NTHU 91
5.11 Geometric Transformation-Spatial transformation
• General Transformation:
where
x y w u v w T' ' ' = 1
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
333231
232221
131211
1
SSSSSSSSS
T
Image Comm. Lab EE/NTHU 92
5.11 Geometric Transformation-Spatial transformation
• T1 is said to be one-to-one if:(a) The inverse transformation of T1 always exists.(b) With T1, one point in screen space produces only
one point in texture space(c) Through T1
-1, each point in texture space can find the corresponding point in screen space.
=> that is the determinate of T1 is nonzero.
Image Comm. Lab EE/NTHU 93
5.11 Geometric Transformation-Spatial transformation
• The 3×3 transformation matrix can be best understood by partitioning it into four separate sections.
• Rotation:
• Translation: • Perspective transformation: • Scaling:
⎥⎦
⎤⎢⎣
⎡=
2221
12112 SS
SST
S S31 32
S S T13 23
S33
Image Comm. Lab EE/NTHU 94
5.11 Geometric Transformation-Spatial transformation
• The general representation of an Affine transformation is :
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
100
]1[]1[
3231
2221
1211
aaaaaa
vuyx
It is a planar mapping, that can be used to map an input triangle to any arbitrary triangle at the output
Image Comm. Lab EE/NTHU 95
5.11 Geometric Transformation-Spatial transformation
5.11 Geometric Transformation-Spatial transformation
Affine Transformation
Image Comm. Lab EE/NTHU 96
5.11 Geometric Transformation-Spatial transformation
5.11 Geometric Transformation-Spatial transformation
Perspective Transformation
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=′′′
333231
232221
131211
],,[],,[aaaaaaaaa
wvuwyx
332313
312111/avauaavaua
wxx++++
=′′=
and let w=1
332313
322212/avauaavaua
wyy++++
=′′=
If then it becomes Affine Transformation02313 == aa
Image Comm. Lab EE/NTHU 97
5.11 Geometric Transformation-Spatial transformation
• Perspective projection
X axis
Y axis
P(X,Y,Z)
Z
D
x
YZ axis
(X ,Y ,0)P P
Center of prjection
Projection Plane
Image Comm. Lab EE/NTHU 98
5.11 Geometric Transformation-Spatial transformation
• Any position along the projection line is (α,β,γ) is
where δ=Z/(Z+D), and 0<δ<1
α δβ δγ δ
= −= −= − +
X XY YZ Z D( )
)1
1(+
⋅=
DZ
XX p )1
1(+
⋅=
DZ
YY p
Image Comm. Lab EE/NTHU 99
5.11 Geometric Transformation-Spatial transformation
• Perspective Transformation
Image Comm. Lab EE/NTHU 100
5.11 Geometric Transformation-Spatial transformation
• Bilinear transformation is defined as
where W=[u, v, 1] and Z=[x, y, 1]
and are constant.
ZT → W: TWZ ⋅=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡+
+=
100
00
232
311
babuaa
vbbaT
3210,3210 and,,,,,, bbbbaaaa
Image Comm. Lab EE/NTHU 101
5.11 Geometric Transformation-Spatial transformation
Bilinear distortion for reference quadrangle and points.
P
P
P
P
1
2
3
4
P1
P2 P3
P4' '
'(0,1) (1,1)
(1,0)(0,0)
u
v
Y
X
T
T -1
Image Comm. Lab EE/NTHU 102
5.11 Geometric Transformation-gray-level interpolation
5.11 Geometric Transformation-gray-level interpolation
Image Comm. Lab EE/NTHU 103
5.11 Geometric Transformation-gray-level interpolation
• Zero-order interpolation: the simplest one.• Bilinear interpolation: using four neighbors to do
the interpolation for the gray level of non-integer point at (x’, y’):
v(x’, y’)=a+bx’+cy’+dx’y’• The four neighbors located at (xi, yi), with known
gray level vii i=1 and 2, then we have four different equations as
vii=a+bxi+cyi+d xi yi• Solving the above four equations for the four
parameters, a, b, c, and d.
Image Comm. Lab EE/NTHU 104
5.11 Geometric Transformation5.11 Geometric Transformation
Image Comm. Lab EE/NTHU 105
5.11 Geometric Transformation-gray-level interpolation
• resampling
Image Reconstruction
Spatial Transformation
Input Samples Output samples
Output Grid
Resampling Grid
Reconstructed Signal
Image Comm. Lab EE/NTHU 106
5.11 Geometric Transformation-gray-level interpolation
• resamplingf(u)
uDiscrete Input Discrete output
g(x)
x
f (u)c g (x)c
u xRecostructed Input
Warping Prefiltering
x=T(u) h(x)
Reconstructb(x)Sample s(x)
Warped Input Continuous Output
g (x)c'
x
Image Comm. Lab EE/NTHU 107
5.11 Geometric Transformation-gray-level interpolation
Ideal warping resampling
∑
∫ ∑
∫
∈
∈
−
−
−⎥⎥⎦
⎤
⎢⎢⎣
⎡−=
−=
∈∗=′=
Zj
Zj
c
cc
jxjf
dttxhjtTbjf
dttxhtTf
Zxh(x(x)gxgxg
),()(=
)())(()(
)())((
for ) )( )(
1
1
ψ
∫ −−= − dttxhjtTbjx )())((),( 1ψwhere
Image Comm. Lab EE/NTHU 108
5.11 Geometric Transformation-gray-level interpolation
Table : Elements of ideal resampling.
f u u Z( ), ∈
f u f u b u f j b u jcj Z
( ) ( )* ( ) ( ) ( )= = −∈∑
g x f T xc c( ) ( ( ))= −1
∫ −==′ dttxhtgxhxgxg ccc )()()(*)()(
g x g x s xc( ) ' ( ) ( )=
Stage Mathematical definition
Discrete Input
Reconstructed Input
Warped Signal
Continuous Output
Discrete Output
Image Comm. Lab EE/NTHU 109
5.11 Geometric Transformation-gray-level interpolation
• Cubic convolution interpolation:using much larger number of neighbors (i.e., 16) to obtain a smoother interpolation.
• Ideal Interpolation:
Reconstruction
Filter
Warped
PrefilterSampler
WarpResampling Filter
f(u) f (u)c g (u)c'
s(T (x))-1
s(x)
g(x)b(u) h(u)
Image Comm. Lab EE/NTHU 110
5.11 Geometric Transformation5.11 Geometric Transformation
Top Related