Convolution, Noise and Filters -...

41
Convolution, Noise and Filters T H E U N I V E R S I T Y of T E X A S Philip Baldwin, Ph.D. Department of Biochemistry

Transcript of Convolution, Noise and Filters -...

Page 1: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Convolution, Noise and Filters

T H E U N I V E R S I T Y of T E X A S

Philip Baldwin, Ph.D.Department of Biochemistry

Page 2: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Response to an Entire SignalThe response of a system with impulse response h(t) to input x(t)is simply the convolution of x(t) and h(t):

ττ)(τ dthxthtxtytx ∫∞

∞−

−=∗=→ )()()()()(

Page 3: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

One Way to Think of Convolution

Think of it this way:Shift a copy of h to each position t (or discrete position k)

Multiply by the value at that position x(t) (or discrete sample x[k])

Add shifted, multiplied copies for all t (or discrete k)

∑ −⋅=∗k

kjhkxjhjx ][][][][

ττ)(τ dthxthtx ∫∞

∞−

−=∗ )()()(

Page 4: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Convolutionx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]

x[0] h[j – 0] = [ __ __ __ __ __ __ __ __ __ ]x[1] h[j – 1] = [ __ __ __ __ __ __ __ __ __ ]x[2] h[j – 2] = [ __ __ __ __ __ __ __ __ __ ]x[3] h[j – 3] = [ __ __ __ __ __ __ __ __ __ ]x[4] h[j – 4] = [ __ __ __ __ __ __ __ __ __ ]

x[j] * h[j] = x[k] h[j – k]

= [ __ __ __ __ __ __ __ __ __ ]

Σk

Page 5: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Convolutionx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]

x[0] h[j – 0] = [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1] = [ __ __ __ __ __ __ __ __ __ ]x[2] h[j – 2] = [ __ __ __ __ __ __ __ __ __ ]x[3] h[j – 3] = [ __ __ __ __ __ __ __ __ __ ]x[4] h[j – 4] = [ __ __ __ __ __ __ __ __ __ ]

x[j] * h[j] = x[k] h[j – k]

= [ __ __ __ __ __ __ __ __ __ ]

Σk

Page 6: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Convolutionx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]

x[0] h[j – 0] = [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1] = [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2] = [ __ __ __ __ __ __ __ __ __ ]x[3] h[j – 3] = [ __ __ __ __ __ __ __ __ __ ]x[4] h[j – 4] = [ __ __ __ __ __ __ __ __ __ ]

x[j] * h[j] = x[k] h[j – k]

= [ __ __ __ __ __ __ __ __ __ ]

Σk

Page 7: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Convolutionx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]

x[0] h[j – 0] = [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1] = [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2] = [ __ __ 3 6 9 12 15 __ __ ]x[3] h[j – 3] = [ __ __ __ __ __ __ __ __ __ ]x[4] h[j – 4] = [ __ __ __ __ __ __ __ __ __ ]

x[j] * h[j] = x[k] h[j – k]

= [ __ __ __ __ __ __ __ __ __ ]

Σk

Page 8: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Convolutionx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]

x[0] h[j – 0] = [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1] = [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2] = [ __ __ 3 6 9 12 15 __ __ ]x[3] h[j – 3] = [ __ __ __ 1 2 3 4 5 __ ]x[4] h[j – 4] = [ __ __ __ __ __ __ __ __ __ ]

x[j] * h[j] = x[k] h[j – k]

= [ __ __ __ __ __ __ __ __ __ ]

Σk

Page 9: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Convolutionx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]

x[0] h[j – 0] = [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1] = [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2] = [ __ __ 3 6 9 12 15 __ __ ]x[3] h[j – 3] = [ __ __ __ 1 2 3 4 5 __ ]x[4] h[j – 4] = [ __ __ __ __ 2 4 6 8 10 ]

x[j] * h[j] = x[k] h[j – k]

= [ __ __ __ __ __ __ __ __ __ ]

Σk

Page 10: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Convolutionx[j] = [ 1 4 3 1 2 ]h[j] = [ 1 2 3 4 5 ]

x[0] h[j – 0] = [ 1 2 3 4 5 __ __ __ __ ]x[1] h[j – 1] = [ __ 4 8 12 16 20 __ __ __ ] x[2] h[j – 2] = [ __ __ 3 6 9 12 15 __ __ ]x[3] h[j – 3] = [ __ __ __ 1 2 3 4 5 __ ]x[4] h[j – 4] = [ __ __ __ __ 2 4 6 8 10 ]

x[j] * h[j] = x[k] h[j – k]

= [ 1 6 14 23 34 39 25 13 10 ]

Σk

Page 11: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Two-Dimensional Convolution

____ ____ ____ ____ ____ ____

____ ____ ____ ____ ____ ____

____ ____ ____ ____ ____ ____

____ ____ ____ ____ ____ ____

____ ____ ____ ____ ____ ____

____ ____ ____ ____ ____ ____

1 1 2 21 1 2 21 1 2 21 1 2 2

1 1 1* 1 2 1 =

1 1 1

Page 12: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Two-Dimensional Convolution

1 2 4 5 4 2

2 5 9 12 10 4

3 7 13 17 14 6

3 7 13 17 14 6

2 5 9 12 10 4

1 2 4 5 4 2

1 1 2 21 1 2 21 1 2 21 1 2 2

1 1 1* 1 2 1 =

1 1 1

Page 13: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Properties of Convolution

• Commutative: f * g = g * f

• Associative: f * (g * h) = (f * g) * h

• Distributive over addition: f * (g + h) = f * g + f * h

• Derivative:

Convolution has the same mathematical properties as multiplication(This is no coincidence, see Fourier convolution theorem!)

( )d f g f g f gdt

′ ′∗ = ∗ + ∗

Page 14: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Gaussian

Gaussian: maximum value = 1

Normalized Gaussian: area = 1

Convolving a Gaussian with another:

-σ σ

1

-σ σ

1

222( , )

tG t e σσ

−=

2221( , )

2

tG t e σσ

πσ−

=

),(),(),( 22

2121 σσσσ +=∗ tGtGtG

Page 15: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

image

+

noise

=

‘grainy’image

©www.cs.qub.ac.uk/~P.Miller/csc312/image/ presentations/csc312_4_02

What is Noise?

Page 16: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

What is Noise?• Anything that is NOT signal:

Signal is what carries information that we are interested inNoise is anything else

• Noise may beCompletely random (both spatially and temporally)StructuredStructured randomness

50 100 150 200 250

100

200

300

50 100 150 200 250

-100

0

100

50 100 150 200 2500

100

200

300

©www.cs.qub.ac.uk/~P.Miller/csc312/image/ presentations/csc312_4_02

Page 17: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Statistical ReviewMean: The average or expected value

Variance: The expected value of the squared error

Standard Deviation: The square root of the variance

∑== xN

xEµ 1}{

2 2 2 2{( ) } { }σ E x µ E x µ= − = −

2σσ =

Page 18: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Ensembles of ImagesConsider the picture Ĩ(x) as a random variable from which we sample an ensemble of images from the space of all possibilities

This ensemble (or collection) of images has a mean (average) image, Ī(x)

If we sample enough images, the ensemble mean approaches the noise-free original signal

Often not feasible

Page 19: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Signal-To-Noise RatioIf we compare the strength of a signal or image (the mean of the ensemble) to the variance between individual acquired images we get a signal-to-noise ratio:

The better (higher) the SNR, the better our ability to discern the signal information

Problem: How to measure m to compute the SNR?

σµ

=SNR

Page 20: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Noise and the Frequency DomainNoisy input:

Ĩ(x) = Ī(x) + ñ(x)

Spectrum of noisy input:

F(Ĩ(x)) = F(Ī(x)) + F(ñ(x))

White noise has equally random amounts of all frequencies

“Colored” noise has unequal amount for different frequencies

Since signals often have more low frequencies than high, the effect of white noise is usually greatest for high frequencies

Page 21: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Filters

Low pass filter- eliminate high frequencies and leave the low frequencies.

High pass filter- eliminate low frequencies and leave high frequencies.

Band pass filter- only a limited range of frequencies remains

Gaussian smoothing- has the effect of cutting off the high frequency components of the

frequency spectrum

Page 22: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Low-Pass Filter• Recall that quick changes in a signal/image require high

frequencies

• High frequency details are often “buried” in noise, which also requires high frequencies

• One method of reducing noise is pixel averaging:Average same pixel over multiple images of same sceneAverage multiple (neighboring) pixels in single image

Page 23: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Convolution Filtering: Averaging

Can use a square function (“box filter”) or Gaussian to locally average the signal/image

Square (box) function: uniform averagingGaussian: center-weighted averaging

Both of these blur the signal or image

Page 24: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Low-Pass Filtering = Spatial BlurringLow-pass filtering and spatial blurring are the same thing

Any convolution kernel with all positive (or all negative)weights does:

Weighted averagingSpatial blurringLow-pass filtering

They are all equivalent

Page 25: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Filtering and ConvolutionTwo ways to think of general filtering:

Spatial: Convolution by some spatial-domain kernel

Frequency: Multiplication by some frequency-domain filter

Can implement/analyze either way

Page 26: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Low-Pass FilteringTradeoff:

Reduces Noisebut

Blurs Image

The worse the noise, the more you need to blur to remove it

Original After Low-pass filtering

©www.cs.qub.ac.uk/~P.Miller/csc312/image/ presentations/csc312_4_02

Page 27: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

“Ideal” Low-Pass FilteringFor cutoff frequency uc:

What is the corresponding convolution kernel?

What problem does this cause?

What could you do differently?

⎪⎩

⎪⎨⎧ ≤

=Π=otherwise 0

if 1)/()( c

cuu

uuuH

Page 28: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Better (Smoother) Low-Pass FilteringGentler ways of cutting off high frequencies:

Hanning

Gaussian

Butterworth

n controls the sharpness of the cutoff

0 5 0 5cos / if ( ) 2

0 otherwise

c cπ. . ( u u ) u u

H u⎧ + ≤⎪= ⎨⎪⎩

22

2)( cuu

euH−

=

n

cuu

uH

⎟⎟⎠

⎞⎜⎜⎝

⎛+

=

22

1

1)(

Page 29: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Sharpening• Blurring is low-pass filtering, so de-blurring is high-pass

filtering:Explicit high-pass filteringUnsharp MaskingDeconvolutionEdge Detection

• Tradeoff:Reduces Blurbut

Increases Noise

Page 30: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

High-Pass Filtering• “Ideal”:

• Flipped Butterworth:

⎪⎩

⎪⎨⎧ ≤

=Π−=otherwise 1

if 0)/(1)( c

cuu

uuuH

ncuu

uH)/(1

11)( 22+−=

Page 31: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

High-Pass Filtering vs. Low-Pass Filtering

©exchange.manifold.net/manifold/manuals/5_userman/mfd50Image__Filter.htm

Original

After Low-pass filtering

After High-pass filtering

Page 32: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Convolution Filtering: Unsharp MaskingUnsharp masking is a technique for high-boost filtering. To sharpen a signal/image, subtract a little bit of the blurred input.

Procedure:Blur the image.Subtract from the original.Multiply by some weighting factor.Add back to the original.

I′ = I + α(I – I * g)

where I′ is the original image, g is the smoothing (blurring) kernel, and I is the final (sharpened) image

Page 33: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Unsharp Masking: Implementation

I + α(I – I * g)

⎥⎥⎥

⎢⎢⎢

α−α−α−α−α+α−α−α−α−

=

⎥⎥⎥⎥

⎢⎢⎢⎢

⎟⎟⎟⎟

⎜⎜⎜⎜

⎥⎥⎥

⎢⎢⎢

⎡−

⎥⎥⎥

⎢⎢⎢

⎡α+

⎥⎥⎥

⎢⎢⎢

8991

111111111

000090000

000090000

91

Page 34: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Unsharp Masking Image

Original Image After Unsharp Masking

©www.luminous-landscape.com/tutorials/understanding-series/understanding-usm.shtml

Page 35: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

DeconvolutionIf we want to “undo” low-pass filter H(u),

Problem 1: This assumes you know the point-spread functionProblem 2: H may have had small values at high frequencies, so Hinv has

large values (multipliers)

Small errors (noise, round-off, quantization, etc.) can get magnified greatly, especially at high frequencies

This is a common problem for all high-pass methods

)(1)( uHuHinv =

Page 36: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Example: Deconvolution

Before deconvolution After deconvolution

©www.reindeergraphics.com/tutorial/chap4/fourier12.html

Early Hubble space telescope image with precisely known optical aberrations

Page 37: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Band-Pass FilteringTradeoff: Blurring vs. Noise

Low-Pass: reduces noise but accentuates blurringHigh-Pass: reduces blurring but accentuates noise

A compromise:Band-pass filtering boosts certain midrange frequencies and partiallycorrects for blurring, but does not boost the very high (most noise corrupted) frequencies

Page 38: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Band-Pass Filtering vs. Low-Pass, High-Pass Filtering

Original Image

After Low-pass filter After High-pass filter After Band-pass filter

©astronomy.swin.edu.au/~pbourke/analysis/imagefilter/

Page 39: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Median “Filtering”Instead of a local neighborhood weighted average, compute the median of the neighborhood

• Advantages:Removes noise like low-pass filtering doesValue is from actual image valuesRemoves outliers – doesn’t average (blur) them into result (“despeckling”)Edge preserving

• Disadvantages:Not linearNot shift invariantSlower to compute

Page 40: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Median “Filtering”

©John C. RussRemoval of shot noise with a median filter

Original image

Image a with 10% of the pixels randomly selected and set to black, and another 10% randomly selected and set to white

Application of median filtering to image b using a 3x3 square region

Application of median filtering to image b using a 5x5 square region

Page 41: Convolution, Noise and Filters - biomachina.orgsitus.biomachina.org/hn06/talks/Baldwin/convolution...Unsharp masking is a technique for high-boost filtering. To sharpen a signal/image,

Figure and Text Credits

Text and figures for this lecture were adapted in part from the following source, in agreement with the listed copyright statements:

http://web.engr.oregonstate.edu/~enm/cs519© 2003 School of Electrical Engineering and Computer Science, Oregon State University. Dearborn, Corvallis, Oregon, 97331

Resources

Textbook:Kenneth R. Castleman, Digital Image Processing, Chapter 11