Image Processing - Lesson 3

• Image Enhancement - Spatial Domain

• Smoothing filter

• Median filter

• Convolution

• 1D Discrete

• 1D Continuous

• 2D Discrete

• 2D Continuous

• Sharpening filter

Image Enhancement

Salt & Pepper Noise

Neighborhood Averaging

g(x,y) = (1/M) Σ f(n,m)

S = neighborhood of pixel (x,y)M = number of pixels in neighborhood S

Neighborhoods:

3 x 3

5 x 5

Sm)(n, ∈

Neighborhood Averaging - Example

Salt & PepperNoise

3 X 3 Average 5 X 5 Average

7 X 7 Average Median

Convolution

A, B = images B is typically smaller than A and is called the mask.

(A * B) (x) = Σ A(i)B(x-i)i

1 dimensional:

Example:

1 2 3

BA1 2 3 4 5

(A * B) (1)

1 2 3 4 5

3 2 1

x=0

1 10 4 5

* * *

+

x=0

x=1

3 4 3

(A * B) (2)

1 2 3 4 5

3 2 1

10 16 22

* * *

+

x=2

6 6 4

• Option 1: Zero padding

• Option 2: Wrap around

• Option 3: Reflection

What happens near the edges?

0 0 0 1 2 3 4 5 0 0 0

4 10 16 22 22

3 4 5 1 2 3 4 5 1 2 3 4 5

19 10 16 22 23 5

3 2 1 1 2 3 4 5 5 4 3 2

7 10 16 22 27 5

1 2 3Convolution with

Why one image is reflected in the convolution:

0 3 2 1 00 1 0 ........ 1 2 3 ........* ........=

0 3 2 1 01 2 3 ........ 0 1 0 ........* ........=

With reflection:

Without reflection:

0 1 2 3 00 1 0 ........ 1 2 3 ........* ........=

0 3 2 1 01 2 3 ........ 0 1 0 ........* ........=

Reflection is needed so that convolution is commutative:

A * B = B * A

Convolution - Continuous Case

1/2

1

g(α)

1

1

f(α)

α

α

1/2

-1

g(-α)

α

1/2

g(x-α)

αx

1/2

-1

f(α)g(x-α)

αx

0≤x≤11/2

-1

f(x-α)g(α)

αx

1≤x≤2

1/2

x1 20

( )( ) ( ) ( )∫∞

∞−

−=∗ aaa dxgfxgf

-1 1

0 1

Convolution - 2 Dimensions

A B*

(A * B) (x,y) = Σ Σ A(i,j) B(x-i,y-j)i j

10 5 20 20 20

10 5 20 20 20

10 5 20 20 20

10 5 20 20 20

10 5 20 20 20

*

-10 5 -15 0 0

-10 15 -10 20 20

-10 15 -10 20 20

-10 15 -10 20 20

-10 15 -10 20 20

=

Convolution - 2D Continuous Case:

(zero padding)

( ) ( ) ( ) dßdaßya,xgßa,fy)(x,gf −−=∗ ∫ ∫∞

∞−

∞

∞−

Grayscale Convolution - Example

*

A B

A * B

Convolution Properties

• Complexity:– Assume A is n x n and B is k x k then

A*B takes O(n2k2) operations.• A*B = B*A• (A*B)*C = A*(B*C)

– If B and C are k x k then(A*B)*C takes O(2n2k2) operations.However A*(B*C) takes O(k4+n2k2) operations, which is faster if k<<n .

• Separability– In some cases it is possible to

decompose B (k x k) into B=C*D where C is 1 x k and D is k x 1. In such a case A*B takes O(n2k2)while (A*C)*D takes O(2n2k).

1 -1 *1

-1=

1 -1

-1 1

• Image Average– In order to preserve the overall

average of A, the sum of B’s elements should equal 1

W1 W2 W3B

A

A*B

If W 1+W2+W3 =1 then Av(A)=Av(A*B)

( ) ==

=−−otherwise

yyandxxifyyxx

01

, 0000δ

0 0 00 1 00 0 0

A(x,y)*δ(x-x0,y-y0) = A(x-x0,y-y0)

),(d yx

Mask Constraints

Convolution Masks - Example: The Delta Kernel

A(x,y)*δ(x-x0,y-y0) = A(x-x0,y-y0)

A(x,y)*δ(x,y) = A(x,y)

0 0 00 1 00 0 0

0 0 10 0 00 0 0

δ(x,y) δ(x-1,y-1)

1 2 34 5 67 8 9

A

*0 0 10 0 00 0 0

δ(x-1,y-1)

=

6 4 59 7 83 1 2

0 4 50 7 80 0 0

A(x-1,y-1)

A(x-1,y-1)

(Zero padding)

(Wrap around)=

( ) ==

=−−otherwise

yyandxxifyyxx

01

, 0000δ

*

Grayscale Smoothing

Grayscale averaging = convolution with:

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

3 X 3

5 X 5

“Soft” Averaging: Convolution with a Gaussian

Discrete case:

0 1 0

1 2 1

0 1 0

(1/8) x

1 2 3 2 1

2 4 6 4 2

3 6 9 6 3

2 4 6 4 2

1 2 3 2 1

(1/81) x

A seperable kernel

2

22

2)y-(x

2 e2

1 σ

πσ

+

Normal vs Gaussian Grayscale Smoothing

OriginalNoisy image

3 X 3Average

3 X 3GaussianAverage

5 X 5GaussianAverage

7 X 7GaussianAverage

5 X 5Average

7 X 7Average

Median Filtering

S = neighborhood of pixel (x,y)

30 10 20

10 250 25

20 25 30

10, 10, 20, 20, 25, 25, 30, 30, 250

median

Median + Average: average the k central values.

10, 10, 20, 20, 25, 25, 30, 30, 250

24

medianSy)(x,

∈

New value at (x,y) = {I(x,y)}

Salt & PepperNoise

3 X 3 Average 5 X 5 Average

7 X 7 Average Median

Median vs Average Filtering

Large Noise

Median Median x 2

Median x 7

Median x 4

Median x 6Median x 8

Multiple Median Filtering

Original Salt & Pepper Noise

Median Filter

Median Filtering - Failure

Original Salt & Pepper Noise

Median Filter

Oriented Median Filtering

Oriented Median Filter

Salt & Pepper noise

4x4 Average 7x2 Average

Oriented Filters

Oriented 6x2Average

Original Noisy Image

4x4 Average

Oriented Filtering - Example

Directional Smoothing

Define oriented masks:

1/n1/n

1/n1/n

1/n1/n

1/n1/n

θ

Vθ

Choose neighborhood with smallest variance and replace pixel value with the average of that neighborhood.

Directional Smoothing - Example

Directional Smoothing(2x5, 5x2, diagonalx2)

Original + Noise 3x3 Average

Directional Smoothing - Example

Directional Smoothing(2x5, 5x2, diagonalx2)

Original + Noise 3x3 Average

Sharpening

• A sharpening filter is applied in order to enhance edges and fine details (high frequency) in an image:

– Assume A*G is a smoothing filtering.– A*(δ - G) contains the fine details of

the image.

– A+λA*(δ - G) = A*( (1+λ)δ-λG)=A*S(λ)amplifies fine details in the image.

– The parameter λ controls the amount of amplification.

G =

00

00

81

81

21

81

81

S(1) =

−

−−

−

00

00

81

81

23

81

81

A

δ-G

A*S

Ringing effect in edge enhancement

How can we enhance such an image?

Solution: Image Representation

2 1 3

5 8 7

0 3 5

= 1 0 0

0 0 0

0 0 0

2 + 1

0 1 0

0 0 0

0 0 0

0 0 1

0 0 0

0 0 0

+ 3 + 5

0 0 0

1 0 0

0 0 0

+

+ ...

= 3 + 5 +

+ 10 + 23 + ...

Frequency Domain

Map of “Sizes and Orientations”

Evaluating an Image in terms of “sizes”:

Large Small

Thin & Horizontal

Vertical