Chapter 5 Image Restoration - TUNImoncef/SGN-3016-DIP/Chap05.pdfImage Restoration: adaptive median...

11/16/2009

1

Chapter 5Image Restoration


)()()()(),(),(*),(),(

vuNvuFvuHvuGyxyxfyxhyxg

+=+= η

5.1

),(),(),(),( vuNvuFvuHvuG +

Goal of restoration: Find the restoration filter such that

possibleasyxftocloseasisyxf ),(),(ˆ

Chapter 5Image Restoration: common noise densities

Chapter 5Image Restoration: common noise densities

Basic assumptions:1. Noise is iid2. uncorrelated with the inputexcept in the case of periodic noise.

5.2

11/16/2009

2

Gaussian Noise• Gaussian or Normal (mathematical

bili i b h h i l d ftractability in both the spatial and frequency domains)

22 2/)(

21),;( σμ

σπσμ −−= zezp

where z represents the grey level, is the mean orμ

5.3

where z represents the grey level, is the mean or average value of z and is called the variance of z

μ2σ

Rayleigh Noise

• The PDF of Rayleigh noise is given by

⎪⎩

⎪⎨⎧ ≥=

−−

azfor

azforebzp

baz

p0

2)(

/)( 2

the mean and variance of this density are given by

4/b+ )4(2 π−b

5.4

4/ba πμ +=4

)4(2 πσ =b

11/16/2009

3

Uniform Noise

• The PDF of the uniform noise is given by⎧ 1

⎪⎪⎩

⎪⎪⎨

⎧ ≤≤−

=otherwise

bzaifab

zp0

1

)(

the mean and variance of this density are given by

ba +μ )( 22 ab −

5.5

2=μ

12)(2 ab

=σ

Impulse (Salt & Pepper) Noise• The PDF of bi-polar impulse noise is given by

⎪⎩

⎪⎨

⎧==

=otherwise

bzforPazforP

zp b

a

0)(

If b>>a, grey-level b will appear as a light dot in the image; while grey-level a will appear as a dark dot. If either P or P is zero the impulse noise is called

5.6

either Pa or Pb is zero, the impulse noise is called unipolar. If neither is zero and both are equal, the noise values will resemble salt and pepper granules randomly distributed over the image.

11/16/2009

4



E l t ill t t th h t i ti f th i PDFExample: to illustrate the characteristics of the noise PDFs, consider the following input image:

5.7



5.8

11/16/2009

5



Note the larger magnitude of the black grey-level due to the

5.9

level due to the additional black “pepper” dots!

Periodic Noise

Arises typically from electrical orArises typically from electrical or electromechanical interference during image acquisition,It can be effectively reduced by frequency

domain filtering.

5.10

11/16/2009

6



image is severely corruptedby spatial sinusoidal noiseby spatial sinusoidal noiseof various frequencies

notice the impulses (eachi f j

5.11

pair of conjugateimpulses corresponding to one sine wave)

Estimation of Noise Parameters• Periodic noise:

– inspection in the frequency domain or– inspection in the frequency domain, or– from image direcly (in simple cases only)

• Non-periodic noise:– noise PDF parameters may be known partially from sensor

specifications, or– can be estimated from relatively flat image areas. Mean and

variance are estimated as follows:

5.12

where the zi’s are the gray-level values of the pixels in S and p(zi) are the corresponding normalized histogram values.

∑∑∈∈

−==Sz

iiSz

iiii

zpzandzpz )()()( 22 μσμ

11/16/2009

7

Chapter 5Image Restoration: estimating the noise PDF parameters

Chapter 5Image Restoration: estimating the noise PDF parameters

5.13

Restoration in the Presense of Noise

• Assume that the only degradation present in the image is due to noise, i.e.,

• This brings us back to the enhancement

),(),(),(),(),(),(

vuNvuFvuGyxyxfyxg

+=+= η

5.14

problem encountered earlier.

11/16/2009

8

Mean Filters

1. Arithmetic mean filter

• which can be implemented with a convolution mask in which all coefficients have equal values of 1/mn. Sxy represents the set of coordinates in a

),(1),(),(

tsgmn

yxfxySts

∑∈

=

5.15

xy prectangular subimage windown of size m x n, centered at (x,y).

• Effects: smoothes local variations in an image and noise is reduced as a result of blurring.

Mean Filters

2. Geometric mean filter1

• Effects: achieves smoothing comparable to arithmetic mean, with less loss of image details.

mn

Sts

tsgyxfxy),(

),(),(ˆ⎥⎥⎦

⎤

⎢⎢⎣

⎡= ∏

∈

5.16

details.

11/16/2009

9

Chapter 5Image Restoration: arithmetic vs geometric mean

Chapter 5Image Restoration: arithmetic vs geometric mean

5.17

Arithmetic Mean:noise is reducedat the expenseof blurring

Geometric Mean:noise is alsoremoved, butless blurring

Mean Filters

3. Harmonic mean filter

• Effects: works well for salt noise, but fails for pepper noise. It performs well for other

∑∈

=

xySts tsg

mnyxf

),( ),(1),(ˆ

5.18

for pepper noise. It performs well for other types of noises, such as Gaussian.

11/16/2009

10

Mean Filters

4. Contraharmonic mean filter Q

Sts

tsg ),(ˆ

1

)(∑ +

∈

where Q is called the filter order.• Effects: well suited for salt&pepper noise:

– Q>0, it eliminates pepper noise, and– Q<0, it removes salt noise, but

Q

Sts

Sts

tsgyxf

xy

xy

),(),(ˆ

),(

),(

∑∈

∈=

5.19

Q 0, it removes salt noise, but– it cannot eliminate both types of noises simultaneously!

• Question: what happens when Q=0 and Q=-1?

Arithmetic mean and Harmonic mean, respectively!

Chapter 5Image Restoration: Contraharmonic filtering

Chapter 5Image Restoration: Contraharmonic filtering

the opposite

5.20

Q>0 filter did agood job in cleaningthe background atthe expense of someblur in dark areas

the oppositeis true forQ<0

11/16/2009

11

Chapter 5Image Restoration: contraharmonic filtering

Chapter 5Image Restoration: contraharmonic filtering

Be careful of wrong sign in contraharmonic filtering!g g g

5.21

Order Statistics Filters

Based on ranking the input samples in a local i d d l ti f th th t twindow and selecting one of them as the output.

1. Median filter

• the filter output is the median value of the input data inside the filter window

)},({),(),(

tsgmedianyxfxySts ∈

=

5.22

data inside the filter window. • Effects: for certain types of noises, it produces

excellent results with considerably less blurring than linear filters.

11/16/2009

12



result of 1st

pass of medianfilter

5.23

result of 2nd

pass of medianfilter

result of 3rd

pass of themedian filter

remaining spikes can be removed by 3rd pass, but notice that additional blur might result


2. Max and Min filters

• Max filter is useful for finding the brightest points in an image. It also reduces pepper noise

)},({min),()},({max),(),(),(

tsgyxfortsgyxfxyxy StsSts ∈∈

==

5.24

noise. • Min filter is useful for finding the darkest

points in an image. It also reduces salt noise

11/16/2009

13

Chapter 5Image Restoration: max and min filtering

Chapter 5Image Restoration: max and min filtering

result of filtering the image corrupted result of filtering the image corruptedg g pby pepper noise with a max filter

g g pby salt noise with a min filter

dilation erosion

5.25

note that some points on edges of dark objects became lighter!

note that some white points are removed around borders of light objects making light objects smaller and dark ones larger.


3. Midpoint filter

• Combines order statistics and averaging • Works best for randomly distributed noise,

⎥⎦⎤

⎢⎣⎡ +=

∈∈)},({min)},({max

21),(

),(),(tsgtsgyxf

xyxy StsSts

5.26

y ,like Gaussian and uniform noise.

11/16/2009

14


4. Alpha-trimmed mean filterSuppose we remove d/2 lowest and d/2 highestSuppose we remove d/2 lowest and d/2 highest

samples from the local input set and average the remaining samples:

• Combines order statistics and averaging

),(1),(),(

tsgdmn

yxfxySts

r∑∈−

=

5.27

Combines order statistics and averaging, • Works well for combination of noises, like

Gaussian and salt&pepper noises.Question: what happen when d=0 and d/2=(mn-1)/2?

Arithmetic mean and median, respectively!



(a) image corrupted (b) image corrupted i h b h ifwith uniform noise with both uniform

and s&p noise

(c) output ofarithmetic meanapplied to (b)

(d) output of geometricmean applied to (b)

5.28

(e) output of medianfilter applied to (b)

(f) output of alpha-trimmed meanapplied to (b)

11/16/2009

15

is the local variance of the pixels

i h i f h

Chapter 5Image Restoration: adaptive filtering

Chapter 5Image Restoration: adaptive filtering

[ ]LL

LL myxgyxgyxf −−= ),(),(),( 2

2

σση

2

2Lσ

is the variance of the noise corrupting f(x,y)mL is the local mean

Three cases:1. Noise variance is 0 (no noise)

No filtering should be done!2. Local variance is high

2ησ

5.29

relative to image varianceThis indicates presence of details and thus a value closeto g(x,y) should be returned.

3. Two variances are equalReduce noise by averaging.

Chapter 5Image Restoration: adaptive median


5.30

11/16/2009

16



5.31

The median filter can be adapted in several ways, e.g. selectinga direction for filtering inside a fixed window or varying thewindow size, according to some local statistics.

Chapter 5Image Restoration: noise reduction in frequency domain


Periodic noise reduction/removal in frequency domain

ideal bandreject Butterworth Gaussian

5.32

Main application: removal of periodic noise where the generallocation of their frequency components is approximately known

11/16/2009

17



Stop/Resume

5.33



5.34

note that it would not be possible to achieve this result using directspatial domain filtering with small convolution masks!

11/16/2009

18



Bandpass Filters:Perform the opposite operation of bandreject filtersPerform the opposite operation of bandreject filters.

),(1),( vuHvuH brbp −=

Using a bandpass filter, one can isolate the periodic noise in the previous example.

5.35

Chapter 5Image Restoration: Notch Filters

Chapter 5Image Restoration: Notch Filters

Notch filters reject (or pass) frequencies in predefined neighborhoodabout a center frequency.about a center frequency.

5.36

11/16/2009

19



Original satellite image corrupted byhorizontal scan lines, so noise is expected to occupy the vertical frequency spectrum.

Orig. spectrumnotice thatnoise is notd i t

Let´s apply an ideal notchpassfilter (vertically)

5.37

dominant

result ofnotch filter:noise patterns

result of notchreject filter,most scan lines have been

removed

Chapter 5Image Restoration: Optimum notch filtering


Consider the periodic interferences below. They are multiple intereference components and thus previous method does not work!intereference components and thus previous method does not work! They may remove too many details.

5.38

11/16/2009

20


Objective: optimal filtering by minimizing the local variancesIDEA: Isolate the principle contributions of the interference pattern and then subtract a

variable, weighted portion of the pattern from the corrupted image.Let H(u,v) be a notchpass filter placed at the location of each spike. Then the

interference noise pattern is :N(u,v)=H(u,v)G(u,v) Eq. (1)

where G(u,v) is the FT of the corrupted image.H(u,v) is iteratively selected by observing G(u,v) on a display.From Eq. (1) it follows that:

)},(),({),( 1 vuGvuHFTyx −=η

5.39

Let

where is the estimate of f(x,y) and w(x,y) is a weighting function to be determined.

)},(),({),( yη

),(),(),(),(ˆ yxyxwyxgyxf η−=

),(ˆ yxf


Optimization Problem:Select/find w(x,y) to minimize over a specific neighborhood of each pixel, i.e.,}ˆ{ fVar

where S(x,y) is fixed neighborhood at pixel (x,y).

Solution:Let S(x,y) be of size N=(2a+1)(2b+1) pixels around (x,y). Then

Wh i h f i S( ) i

),(]])ˆ[ˆ[(min 2 yxSoverfEfE −

∑∑−= −=

−++==a

as

b

btyxftysxf

NyxfVar 22 )],(ˆ),(ˆ[1),(}ˆ{ σ

)(f̂ ˆ

5.40

Where is the average of in S(x,y), i.e.,),( yxf ),( yxf

∑∑−= −=

++=a

as

b

bt

tysxfN

yxf ),(ˆ1),(ˆ

11/16/2009

21


Rewriting }ˆ{ fVar

22 }])()()([)]()()({[1)( ∑∑ −−++++−++=a b

yxyxwyxgtysxtysxwtysxgyx ηησ

Assume that w(x,y) remains constant over S(x,y), then

Then,

}]),(),(),([)],(),(),({[),( ∑∑−= −=

++++++=as bt

yxyxwyxgtysxtysxwtysxgN

yx ηησ

Sinsideyxyxwyxyxw

andyxwtysxw

),(),(),(),(

),(),(

ηη =

=++

22 })],(),(),([)],(),(),({[1),( ∑∑ −−++−++=a b

btyxyxwyxgtysxyxwtysxg

Nyx ηησ

5.41

To minimize , we solve for w(x,y).Result:

Finally,

−= −=as btN

),(2 yxσ 0),(),(2

=∂∂

yxwyxσ

),(),(),(),(),(),(),(

22 yxyxyxyxgyxyxgyxw

ηηηη

−

−=

),(),(),(),(ˆ yxyxwyxgyxf η−=



5.42

Fourier spectrum of original without shifting the origin.

11/16/2009

22



Spectrum of the interfering Interfering noisep gnoise pattern

Interfering noise pattern

5.43

Chapter 5Image Restoration Result of optimum notch filtering

Chapter 5Image Restoration Result of optimum notch filtering

Original Filtered

5.44

11/16/2009

23

Recall the image degration/restoration process

Chapter 5Image Restoration: Linear, Position-invariant Degradations

Chapter 5Image Restoration: Linear, Position-invariant Degradations

),()],([),( yxyxfHyxg η+=Before the restoration stage, we have

5.45

Assume that the noise term is absent and H is linear, i.e.][][][ 2121 fbHfaHbfafH +=+

that is H is additive and homogenous



Let’s also assume that H is position (or space) invariant:),()],([ byaxgbyaxfH −−=−− ),()],([ byaxgbyaxfH

for any f(x,y) and a and b, i.e. the response at any point in the imagedepends only on the value of the image at that point, not on its position.

Recall that an image can be expressed in terms of continous impulses:

∫∫ −−= βαβαδβα ddyxfyxf ),(),(),(

5.46

Then we get [ ]∫∫

∫∫−−=

−−==

βαβαδβα

βαβαδβα

ddyxfH

ddyxfHyxfHyxg

)],(),([

),(),()],([),(

assuming H is linear.

11/16/2009

24



Because f is independent of x and y and using the homogeneityit follows:

∫∫ −−= βαβαδβα ddyxHfyxg )],([),(),(

The term )],([),,,( βαδβα −−= yxHyxh

is called the impulse response of H.In optics, the impulse becomes a point of light and h(.) is called a

5.47

point spread function (PSF).Physical optical systems blur (spread) a point of light to someextent. So, we now get

∫∫= βαβαβα ddyxhfyxg ),,,(),(),(

which is called the superposition (or Fredholm) integral of the 1st kind



If H is position invariant, then

∫∫ −−= βαβαβα ddyxhfyxg ),(),(),(

This is called the convolution integral.

When noise is present, the degradation model becomes

∫∫

5.48

),(),,,(),(),( yxddyxhfyxg ηβαβαβα += ∫∫if H is position invariant, then

),(),(),(),( yxddyxhfyxg ηβαβαβα +−−= ∫∫the noise values are random and assumed to be position independent, thus

),(),(),(),(),(),(),(),( vuNvuFvuHvuGoryxyxfyxhyxg +=+∗= η

11/16/2009

25

Chapter 5Image Restoration: estimating the degradation function


There are three major techniques:1. Observation2. Experimentation3. Mathematical modeling

The process of Blind deconvolution refers to restoring an image where the degradation process has been estimated in some way, since the true degradation function is seldom known

5.49

exactly.



Estimation by image observation:Assuming no noise, we look at parts of the image, e.g. background g , p g , g gand part of an object and try to deblur it.

Denote by gs(x,y) the observed sub-image and fs(x,y) the constructedsub-image, then

),(),(),(

vuFvuGvuH

s

ss =

5.50

from the characteristics of this function, one can deduce the completetransfer function H(u,v).

11/16/2009

26



Estimation by experimentation:Assuming that similar equipment to the ones used in acquisition exist,g q p q ,then one can obtain an accurate estimation of the degradation.

Idea: try to get the impulse response of the degradation by imaging an impulse using the same system settings. Then

vuGH ),()(

5.51

AvuGvuH ),(),( =

where A is the FT of an impulse.



Example of estimating the degradation function by experimentation

5.52

11/16/2009

27



Estimation by modelingHufnagel and Stanley (atmospheric turbulence) (1964):g y ( p ) ( )

6/522 )(),( vukevuH +−=where k is constant that depends on the nature of the turbulence.

Note that this is almost a Gaussian! In fact, Gaussian LPF has beenused to model mild, uniform blurring.

5.53

Chapter 5Image Restoration: examples of blurring

Chapter 5Image Restoration: examples of blurring

5.54

11/16/2009

28



Another example of blurring using mathematical models derivedfrom basic principles. E.g. blurring due to uniform linear motionfrom basic principles. E.g. blurring due to uniform linear motionbetween the image and the sensor during image acquisition.

5.55

5.56

11/16/2009

29

5.57

5.58

11/16/2009

30

Chapter 5Image Restoration: Inverse Filtering

Chapter 5Image Restoration: Inverse Filtering

Assume that we have estimated H using any of the previous techniques. Let’s try now to restore the image.techniques. Let s try now to restore the image.

In the absence of any information concerning noise, we get

),(),(),(

),(),(),(~

vuHvuNvuF

vuHvuGvuF +==

Problem: even if we know H, still cannot recover f(x,y) because wedon’t know N(.,.)!

h h h i h ll

5.59

More problems: what happens when H is zero or has very smallvalues? The second part may dominate the restored image!

Can get around this by limiting the filter frequencies to values nearthe origin, i.e. pseudo-inverse filtering, see example next.

Inverse filtering vs Pseudo-inverseInverse filtering vs Pseudo-inverse

terrible results with full filterdue to low values of degradations!

result with H cut-off outsidea radius of 40

5.60

outside r = 70 outside r = 85

11/16/2009

31

Chapter 5Image Restoration: Wiener Filtering


Remark:d d i fil i h ff fInverse and pseudo-inverse filtering reverse the effects of system

only; but can do nothing about the random noise in the signal.

Alternative Solution: Wiener FilteringWiener filtering has been successfully used to filter images corrupted by noise and blurring. The idea of Wiener filtering isto find the “best” estimate of the true input u(m n) from the

5.61

to find the best estimate of the true input u(m,n) from the observed image v(m,n) by modeling the input and output images as random sequences.

“best” in the mean square error sense.



In mathematics, Wiener deconvolution is an application of the Wi filt t th i bl i h t i d l tiWiener filter to the noise problems inherent in deconvolution.

It works in the frequency domain, attempting to minimize the impact of deconvoluted noise at frequencies which have a poor signal to noise ratio.

The Wiener deconvolution method has widespread use in image

5.62

The Wiener deconvolution method has widespread use in imagedeconvolution applications, as the frequency spectrum of most visual images is fairly well behaved and may be estimated easily.

Wiener deconvolution is named after Norbert Wiener.

11/16/2009

32



u v ûG

The optimization problem is then stated as: find the estimate of u(m,n) which minimizes

})],(ˆ),({[ 22 nmunmuEe −=σ

G

5.63

(skip details)The best estimate is found to be the conditional mean of u(m,n)

given {v(m,n), for every (m,n)}, i.e.,

e

)],(),,(|),([),(ˆ lklkvnmuEnmu ∀=

• Problems: thi i li f ti d



– this is a nonlinear function, and– the conditional probability function needed to compute it is

very difficult to calculate. • Cure:

– Restrict the estimate to be linear, i.e., find g(.,.;.,.) in

∑∑ )()()(ˆ lklk

5.64

To minimize:∑∑= ),(),;,(),(ˆ lkvlknmgnmu

})],(ˆ),({[ 22 nmunmuEe −=σ

11/16/2009

33

Important Remark:If u(m n) and v(m n) are jointly Gaussian then the conditional



If u(m,n) and v(m,n) are jointly Gaussian, then the conditional mean (best solution in the mse sense) is linear!

Solution of the Wiener FilterAssume that u(m,n) and v(m,n) are jointly stationary, i.e., if we

denote the cross-correlation between u and v by)],(),([),;,( nmvnmuEnmnmRuv ′′=′′

5.65

Then

To minimize the following orthogonality condition must hold:

)','()',';,( nnmmRnmnmR uvuv −−=

2eσ

)','(),(0)]','()},(ˆ),([{ nmandnmnmvnmunmuE ∀=−

This condition leads to:



∑∑This is called the Wiener equation and its solution is the Wiener Filter.

Take the FT of both sides:

Now, suppose that the observation v(m,n) is modeled by a linear system plus

∑∑ =−− ),(),(),( nmRlkRlnkmg uvvv

),(),(),( 211

2121 wwSwwSwwG vvuv−=

5.66

pp ( ) y y psome additive noise, i.e.,

where the last term is a stationary noise sequence that is uncorrelated with u(m,n) and has power spectral density

∑∑ +−−= ),(),(),(),( nmlkulnkmhnmv η

ηηS

11/16/2009

34

Then



uuuvuuvv SHSandSSHS *2|| =+= ηη

The FT of the Wiener filter is then given by

This is also called the Fourier-Wiener filter and is ηηSSH

SHG

uu

uu

+= 2

*

||

5.67

completely specified by the power spectra of the object Suu, the noise and the imaging system H.ηηS



ηηSSHSH

Guu

uu

+= 2

*

||Interpretation of the Wiener Filter• In the absence of blur, H=1, the Wiener filter becomes a

smoothing filter (work it out)

• When there is no noise, the Wiener filter reduces to the inverse filter (work it out)

5.68

• When both noise and blur are present, the Wiener filter seeks a compromise between: lowpass noise smoothing, and highpass deblurring; the result is a bandpass filter. However, deblurring decreases rapidly as the noise power increases. Experiment with this!

11/16/2009

35



5.69



Example: Comparing Inverse and Wiener Filtering:(a) result of full inverse filtering(b) result of radially limited inverse filtering (c) result of Wiener filtering

5.70

11/16/2009

36

(a) image corrupted with motion blur and additive noise(b) result of inverse filtering(c) result of Wiener filtering(d)-(f) same sequence but with noise variance one order of magnitude less(g)-(j) same sequence but noise variance reduced by five orders of magnitude from (a)

5.71

Remark:The Wiener filter is not in general separable even if the imaging system is



g p g g yseparable! That is, the 2-D Wiener filter is not equivalent to row-by-row and column-by-column 1-DWiener filtering.

5.72

Chapter 5 Image Restoration - TUNImoncef/SGN-3016-DIP/Chap05.pdfImage Restoration: adaptive median...

Documents

Transcript of Chapter 5 Image Restoration - TUNImoncef/SGN-3016-DIP/Chap05.pdfImage Restoration: adaptive median...