Image Processing Image Transform and Fourier/Wavelet Transform

Table of Content

Image ProcessingFourier Transform

O. Le [email protected]

Univ. of Rennes 1http://www.irisa.fr/temics/staff/lemeur/

October 12, 2015

1

mailto:[email protected]

http://www.irisa.fr/temics/staff/lemeur/

Table of Content

1 Reminder

2 Image transformation

3 Fourier transformation

4 Time sampling

5 Discrete Fourier Transform

6 Bi-dimensional Fouriertransformation

7 Fast Fourier transform

8 Discrete unitary transform

2

ReminderImage transformationFourier transformation

Time samplingDiscrete Fourier Transform

Bi-dimensional Fourier transformationFast Fourier transform

Discrete unitary transformConclusion

Reminder

1 Reminder



4 Time sampling





3





Notations

a continuous signal x(.);

a discrete signal x[.];

a matrix A.

4





Notations

f is the frequency (Hz); w = 2πf is the pulsation (radian/s); T = 1f is the period

(s);

x ∈ Lp(R) means that∫ +∞−∞ |x(t)|p dt < +∞. For instance, the space L2(R) is

composed of the functions x with a finite energy∫ +∞−∞ |x(t)|2 dt < +∞;

Euler’s formulas: exp(jθ) = cosθ + jsinθ;

From Wikepedia.

5





Reminder

sinc(x) = sinπxπx ;

(−1)n = exp (jπn) nth root of the unity;

Inner product:

〈x , y〉 =

∫ +∞

−∞x(t)× y∗(t)dt

〈x , y〉 =+∞∑

n=−∞x(n)× y∗(n)

6





Reminder

Condition of orthogonality for real basis function:∫ t2

t1φn(t)φm(t)dt =

{0 n 6= mλm n = m

where λm is a constant (λm = 1, orthonormal).

Condition of orthogonality for complex basis function:∫ t2

t1φn(t)φ∗m(t)dt =

{0 n 6= mλm n = m

where λm is a constant (λm = 1, orthonormal).

7





Presentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation

Image transformation

1 Reminder

2 Image transformationPresentation3 kinds of transformationPoint to point transformationLocal to point transformationGlobal to point transformation


4 Time sampling





8







What is a transformation?

im[x , y ] T−→ IM[u, v ]

im is the original image;

IM is the transformed image;

x , y (or u, v) represents the spatial coordinates of a pixel.

Goal of a transformation

The goal of a transformation is to get a new representation of the incoming picture.This new representation can be more convenient for a particular application or canease the extraction of particular properties of the picture.

9







There exist 3 types of transformation:

Point to point transformation:The output value at a specific coordinateis dependent only on one input value butnot necessarily at the same coordinate;

Local to point transformation:The output value at a specific coordinateis dependent on the input values in theneighborhood of that same coordinate;

Global to point transformation:The output value at a specific coordinateis dependent on all the values in the inputimage.

Note that the complexity increases with the size of the considered neighborhood...

10






Point to point transformation

Geometric transformation such as rotation, scaling...Example of rotation:

Scalar quantizationLUT (Look-Up-Table)

Remark: a point to point transformation doesn’t require extra memory...11






Example of local to point transformation

Examples of local to point transformation:

Linear transformation:For instance, a tranverse filter (FIR) IM[x , y ] =

∑i∑

j h(i , j)im[x − i , y − j],where h is the 2D impulse response.

Non-linear transformation such as adaptive linear filtering and rank filtering(median)...

Morphological filtering (erosion, dilatation...) based on the definition of astructuring element.

Note that local to point transformation is also called neighborhood operators.

12






Global to point transformation

One of the most important transformation is the Fourier transform that gives afrequential representation of the signal.

13





DefinitionTrivial propertiesParseval FormulaPropertiesSome examples

Fourier transformation

1 Reminder


3 Fourier transformationDefinitionTrivial propertiesParseval FormulaPropertiesSome examples

4 Time sampling





14







The Fourier transform decomposes a signal into the frequencies that make it up.

From Wikipedia.

15







Fourier transformation in L2(R)

x(t) F−→ X (f )

Inverse Fourier Transform:

x(t) =

∫ +∞

−∞X (f )exp(j2πft)df

Fourier Transform:

X (f ) =

∫ +∞

−∞x(t)exp(−j2πft)dt

provided that x ∈ L2(R) and X ∈ L2(R).

16







From the output of the Fourier transform, we define:

The frequency spectrum: Real (X (f )) + jImg ((X (f )))The Fourier transform of a function produces a frequency spectrum whichcontains all of the information about the original signal, but in a different form.

Magnitude spectrum: |Real (X (f )) + jImg (X (f ))|

Phase spectrum: Arctg(

Img(X (f ))Real(X (f ))

)Power spectrum: Real (X (f ))2 + Img (X (f ))2

17







Linearity:

ax(t)F−→ aX (f )

ax1(t) + bx2(t)F−→ aX1(f ) + bX2(f )

Complex conjugate: x∗(t)F−→ X∗(−f )

x∗(t) =

(∫ +∞


)∗x∗(t) =

∫ +∞

−∞X∗(f )exp(−j2πft)df

In the same way, x∗(−t)F−→ X∗(f ) and x(−t)

F−→ X (−f ).

18







Hermitian symmetry: if x(t) ∈ R, we deduce X (−f ) = X∗(f )

X (f ) =

∫ +∞


X∗(f ) =

(∫ +∞


)∗X∗(f ) =

∫ +∞

−∞x∗(t)exp(j2πft)dt

Given that x(t) ∈ R, we have x∗(t) = x(t) that implies X∗(f ) = X (−f ).

19







Note that, as we use the space L2(R), the inner product of x ∈ L2(R) and y ∈ L2(R)is given by:

〈x , y〉 =

∫ +∞

−∞x(t)y∗(t)dt

The norm is then given by:

‖x‖2 = 〈x , x〉 =∫ +∞−∞ |x(t)|2 dt∫ +∞

−∞|x(t)|2 dt =

∫ +∞

−∞x(t)x∗(t)dt

=

∫ +∞

−∞x(t)

[∫ +∞

−∞X∗(f )exp(−j2πft)df

]dt

=

∫ +∞

−∞X∗(f )

[∫ +∞


]df

=

∫ +∞

−∞X∗(f )X (f )df∫ +∞

−∞|x(t)|2 dt =

∫ +∞

−∞|X (f )|2 df

20







∫ +∞

−∞|x(t)|2 dt =

∫ +∞

−∞|X (f )|2 df

This formula (Parseval’s theorem or energy conservation) proves that theenergy is conserved by the Fourier transform.

Loosely, the sum (or integral) of the square of a function is equal to thesum (or integral) of the square of its transform.

21







Translation in time/space domain: x(t − t0)F−→ exp(−j2πt0f )X (f )

x(t − t0)F−→

∫ +∞

−∞x(t − t0)exp(−j2πtf )dt

k = t − t0

x(t − t0)F−→

∫ +∞

−∞x(k)exp(−j2π(k + t0)f )dk

x(t − t0)F−→ exp(−j2πt0f )

[∫ +∞

−∞x(k)exp(−j2πfk)dk

]Displacement in time or space induces a phase shift proportional to frequency and

to the amount of displacement.

Frequency shift: x(t)exp(±j2πf0t)F−→ X (f ± f0)

Displacement in frequency multiplies the time/space function by a unit phasorwhich has angle proportional to time/space and to the amount of displacement.

Amplitude modulation.

22







Convolution in time/space domain: x1(t) ∗ x2(t)F−→ X1(f )× X2(f )

x1(t) ∗ x2(t)F−→

∫ +∞

−∞

[∫ +∞

−∞x1(τ)x2(t − τ)dτ

]exp(−j2πft)dt

x1(t) ∗ x2(t)F−→

∫ +∞

−∞x1(τ)

[∫ +∞

−∞x2(t − τ)exp(−j2πft)dt

]dτ

Translation

x1(t) ∗ x2(t)F−→

[∫ +∞

−∞x1(τ)exp(−j2πf τ)dτ

]× X2(f )

Convolution in frequency domain: x1(t)× x2(t)F−→ X1(f ) ∗ X2(f )

These properties mean that if two functions are multiplied in one domain, thentheir Fourier transforms are convolved in the other domain.

23







Time and frequency scaling: x(λt)F−→ 1|λ|X ( f

λ)

x(λt)F−→

∫ +∞

−∞x(λt)exp(−j2πtf )dt

k = λt

x(λt)F−→

∫ +∞

−∞x(k)exp(−j2πk

fλ

)dk|λ|

x(λt)F−→

1|λ|

[∫ +∞

−∞x(k)exp(−j2πk

fλ

)dk]

Multiplication of the scale of the time/space reference frame changes by thefactor λ scales the frequency axis of the spectrum of the function.

24







Differentiation: dnx(t)dtn

F−→ (j2πf )nX (f )

dnx(t)

dtn=

dn

dtn

[∫ +∞

−∞X (f )exp(j2πtf )df

]dnx(t)

dtn=

∫ +∞

−∞[(j2πf )nX (f )] exp(j2πtf )df

Differentiation of a function induces a 90 degrees phase shift in the spectrum andscales the magnitude of the spectrum in proportion to frequency.

All the previous properties are what makes Fourier transforms so usefuland practical.

25







Translated Dirac, x(t) = δ(t − T )

X (f ) =

∫ +∞

−∞x(t)exp(−j2πtf )dt

X (f ) = exp(−j2πTf )

If T = 0, X (f ) = 1. The Fouriertransform is constant whatever thefrequency.

x(t) = 1, X (f ) = δ(f )

x(t) = Arect(t|T )

X (f ) =

∫ +∞

−∞x(t)exp(−j2πtf )dt

= A∫ + T

2

−T2

exp(−j2πtf )dt

= −A

j2πf[exp(−jfTπ)− exp(jfTπ)]

X (f ) = ATsinc(fT )

26







Ideal low-pass filter in frequency domain: X (f ) = rect(f |F2 ) selects low

frequencies over[−F

4 ,F4

]. The impluse response of the filter is calculated with

the inverse Fourier integral:

x(t) =

∫ +∞


=

∫ + F4

− F4

exp(j2πft)df

=1

j2πt

[exp(j2π

F4

t)− exp(−j2πF4

t)

]sinθ =

exp(jθ)− exp(−jθ)

2j

x(t) =F2

sinc(π

2tF )

27





What is the objective?Reminder on DiracSamplingSampling TheoremIn practice

Time sampling

1 Reminder



4 Time samplingWhat is the objective?Reminder on DiracSamplingSampling TheoremIn practice





28






Time sampling

To be more flexible and to gain in accuracy...

Digital algorithms are now ... everywhere from television, CD, embeddedelectronic (car...).

Whether sound recordings or images, most discrete signals are obtainedby sampling an analog signal. But, be careful, we have to follow some

constraints in order to be able to reconstruct the analog signal.

The simplest way to discretize an analog signal x is to record its samplevalues at intervals T : x(t) can be approximated by {x(nT )}n∈Z .

29






Definition

Diracs are essential to make the transition from functions of real variable to discretesequences.

A Dirac can be defined as: ∫ +∞

−∞δ(t)x(t)dt = x(0)

δ(t) =

{0 if t 6= 0

+∞ for t = 0∫ +∞

−∞δ(t)dt = 1.

Remark: {δτ : δ(t − τ)}τ∈Z is an orthonormal basis:

〈δ(t − n), δ(t −m)〉 =

{0 if m 6= n1 for m = n .

30






Properties of a Dirac

Scaling and symmetry: δ(λt) = δ(t)|λ|∫ +∞

−∞δ(λt)dt =

∫ +∞

−∞δ(k)

dk|λ|

δ(λt) =δ(t)

|λ|

Note that if λ = −1, δ(−t) = δ(t).Product: x(t)× δ(t − T ) = x(T )× δ(t − T )

Convolution: x(t) ∗ δ(t − T )

x(t) ∗ δ(t − T ) =

∫ +∞

−∞x(τ)δ(t − T − τ)dτ

=

∫ +∞

−∞x(τ)δ(τ − (t − T ))dτ

= x(t − T )

The effect of convolving with a translated Dirac is to time-delay x(t), by thesame amount (copy of the signal x(t) at the position T ).

31






Dirac comb: x(t) =∑

n∈Z δ(t − nT ). Notation: ΠT (t)

The product of a comb with a signal x(t) conducts to a sampling:

The convolution of a comb with a signal x(t) conducts to a periodization of x(t):

32






FT of a Dirac

Dirac: x(t) = δ(t), δ(t)F−→ 1, constant function whatever the frequency

X (f ) =

∫ +∞

−∞δ(t)exp(−j2πtf )dt

X (f ) = exp(−j2π0f ) = 1

Translated Dirac: x(t) = δ(t − T ), δ(t − T )F−→ exp(−j2πTf )

X (f ) =

∫ +∞

−∞δ(t − T )exp(−j2πtf )dt

X (f ) = exp(−j2πTf )

Constant signal: x(t) = 1, 1 F−→ δ(f ). The FT of a constant signal is a weight atthe null frequency.

33

Dirac comb: x(t) =∑

n∈Z δ(t − nT )This is a periodic function, since x(t + T ) = x(t), ∀t. Its Fourier serie is given by:

x(t) =+∞∑

n=−∞Cnexp(j2πn

tT

)

Cn =1T

∫ T2

−T2

δ(t)exp(−j2πnntT

)dt =1T

x(t) =1T

+∞∑n=−∞

exp(j2πntT

)

By using the Fourier Transform,

F(x(t)) = F

1T

+∞∑n=−∞

exp(j2πntT

)

F(x(t)) =

1T

+∞∑n=−∞

F(

exp(j2πntT

)

)

F(x(t)) =1T

+∞∑n=−∞

∫ +∞

−∞exp(−j2πt(f −

nT

))dt

∑n∈Z

δ(t − nT )F−→

1T

∑n∈Z

δ(f −nT

)






Sampling

Sampling a signal is equivalent to multiplying it by a grid of impulses.

35






Sampling

Let x(t) be a continous function, its sampled version y(t) is given by:

y(t) = x(t)× ΠT (t)

= x(t)×∑n∈Z

δ(t − nT )

=∑n∈Z

x(t)δ(t − nT )

y(t) =∑n∈Z

x(nT )δ(t − nT )

where ΠT (t) is the sampling function (sampling frequency 1T ).

36






Fourier Transform of a sampled signalLet x(t) be a continous function, its sampled version y(t), y(t) = x(t)× ΠT (t)

y(t) = x(t)×∑n∈Z

δ(t − nT )

F(y(t)) = F

x(t)×∑n∈Z

δ(t − nT )

F(y(t)) = X (f ) ∗ F

∑n∈Z

δ(t − nT )

F(y(t)) = X (f ) ∗

1T

∑n∈Z

δ(f −nT

)

F(y(t)) =

1T

∑n∈Z

X (f ) ∗ δ(f −nT

)

F(y(t)) =1T

∑n∈Z

X (f −nT

)

The Fourier representation of the sampled function is a superposition of the Fourier transformX (f ) and all its replicas at positions n

T . For n = 0, the spectrum is the primary spectrum andfor n 6= 0 we have copies of the spectrum.

37






Fourier Transform of a sampled signal

Let x(t) be a continous function, its sampled version y(t), y(t) = x(t)× ΠT (t)Let X (f ) be the Fourier transform of x(t) and Y (f ) the Fourier transform of itssampled version y(t).

Y (f ) =1T

∑n∈Z

X (f −nT

)

The sampling operation replicates the primary spectrum periodically in the Fourierdomain.

38







39







Aliasing effect...

We won’t be able to recover the originalsignal with a good quality. The

transformation is no more inversible!

The sampling frequency is too small.

40






Shannon-Nyquist theorem

To ensure a perfect reconstruction of a signal x(t) (to avoid aliasingeffect), the sampling frequency fe must verify:

fe > 2× fmax

where fmax is the maximal frequency of the spectrum X (f ).

The periodization doesn’t provoke an aliasing effet.

41






Whittaker-Shannon theorem

If x(t) is bandlimited to afrequency range [−fmax , fmax ],

that is, its Fourier transform X (f )is zero for |f | > fmax , then x(t)can be completely specified bysamples taken at the Nyquist

sample rate of 2fmax .

42






Whittaker-Shannon theorem

To recover the signal x(t), we can use alow-pass filter in frequency domain. Forinstance, the ideal low-pass filter in the

frequency domain is given byG(f ) = Te rect(f /fe)

X (f ) = Y (f )× rect(f /fe)

x(t) =+∞∑

k=−∞x(kTe)sinc((t − kTe)fe)

43

First summary in an ideal context

1 Sampling by an ideal sampler(Dirac). Do not forget to verify thevalue of the sampling frequency;

2 Low-pass filtering to retrieve thesignal






The sampling of a physical signal always induces an aliasing effect, even with an idealsampler. Paley-Wiener theorem indeed showed that there is no bandlimited signalhaving a finite energy...

In many cases, the spectrum of the signal is not perfectly known and can be corruptedby noise. It is thus required to filter the analog signal before the sampling operation.

1 Pre-filtering to obtain a bandlimited signal;2 Sampling by checking the value of the sampling frequency;3 Low-pass filtering to retrieve the signal.

45





DefinitionContinuous FT and Discrete TFFourier series and Discrete TFMatrix formulation of the DTFExamples

Discrete Fourier Transform

1 Reminder



4 Time sampling

5 Discrete Fourier TransformDefinitionContinuous FT and Discrete TFFourier series and Discrete TFMatrix formulation of the DTFExamples




46






Definition

The discrete Fourier transform (DFT) is the transform of a finite sequence. Since thetime-domain is discrete the spectrum is periodic

The DFT of a sequence of N complex numbers x0, . . . , xN−1 is given by:

X [k] = 1N∑N−1

n=0 x[n]exp(−j 2πN nk), k = 0, . . . ,N − 1

⇐⇒X [k] = 〈x , ek〉, where ek(n) = exp(j 2π

N nk).

The family{ek(n) = exp(j 2π

N nk)}

0≤k<N is an orthogonal basis of the space ofsignals of period N.

The IDFT of a sequence of N complex numbers X0, . . . ,XN−1 is given by:

x[n] =∑N−1

k=0 X [k]exp(j 2πN nk), n = 0, . . . ,N − 1

The DFT computes the X [k] from the x[n], while the IDFT shows how to computethe x[n] as a sum of sinusoidal components X [k]exp( j2π

N kn) with frequency k/Ncycles per sample.

47






Definition

X [k] =1N

N−1∑n=0

x[n]exp(−j2πN

nk)

The weighting coefficient 1N can be removed or replaced by 1√

N. In the first case, the

weighting coefficient of the IDFT is equal to 1N whereas in the second case it is equal

to 1√N.

What is important is that DFT ◦ IDFT = Id .

48






Continuous FT and Discrete TF

Let x [n] defined only for N samples, called x0, . . . , xN−1. Its continuousFourier transform is defined as:

X (f ) =

∫ +∞

−∞x [n]exp(j2πfn)dn

X (f ) =N−1∑n=0

x [n]exp(−j2πfn)

f =kN

X (kN) = N × X [k]

X [k] = 1N X ( k

N ), k = 0, . . . ,N − 1

49






Fourier series and Discrete TF

Let x [n] defined only for N samples, called x0, . . . , xN−1. Its Fourier serieis defined as:

x [n] =N−1∑k=0

C [k]exp(j2πknN

)

C [k] =1N

N−1∑n=0

x [n]exp(−j2πknN

)

C [k] = X [k]

X [k] = C [k], k = 0, . . . ,N − 1

50






Matrix formulation of the DTF

The DFT of a sequence of N complex numbers x0, . . . , xN−1 is given by:

X [k] =1N

N−1∑n=0

x[n]exp(−j2πN

nk)

k = 0, . . . ,N − 1

X [k] =

N−1∑n=0

AN(k, n)x[n]

X = AN × x

with,

x =

x0...

xN−1

, X =

X0...

XN−1

, and AN(k, n) = 1N exp(−j 2π

N kn)

51







X [k] =N−1∑n=0

AN(k, n)x [n]

X = AN × x

Let WN = exp(−j 2πN ), then AN(k, n) = 1N WN

kn and then

AN = 1N

1 1 1 · · · 11 WN WN

2 · · · WNN−1

1 WN2 WN

4 · · · WN2(N−1)

......

......

...1 WN

N−1 WN2(N−1) · · · WN

(N−1)(N−1)

52







Remark: WN is the Nth root of the unit.

A2 = 12

[1 11 W2

]= 1

2

[1 11 −1

]

A4 = 14

1 1 1 11 W4 W4

2 W43

1 W42 W4

4 W46

1 W43 W4

6 W49

= 14

1 1 1 11 −j −1 j1 −1 1 −11 j −1 j

53






Properties of AN

WNkn = WN

nk , then AN(n, k) = AN(k, n), ∀(k, n), AN is symmetric;As AN is symmetric, AN = AN

T ;WN

i = (WN−i )∗

WNi = exp(−j

2πN

i)

= cos(2πN

i)− jsin(2πN

i)

=

(cos(

2πN

i) + jsin(2πN

i))∗

WNi = (WN

−i )∗

If N is even, WNi+ N

2 = −WNi .

54






Inverse DFT

DFT: X = AN × x

AN = 1N

1 1 1 · · · 11 WN WN

2 · · · WNN−1

1 WN2 WN

4 · · · WN2(N−1)

......

......

...1 WN

N−1 WN2(N−1) · · · WN

(N−1)(N−1)

IDFT: x = AN

−1 × X

AN−1 =

1 1 1 · · · 11 WN

−1 WN−2 · · · WN

−(N−1)

1 WN−2 WN

−4 · · · WN−2(N−1)

......

......

...1 WN

−(N−1) WN−2(N−1) · · · WN

−(N−1)(N−1)

= N (AN∗)T

AN−1 = N (AN)∗

x = N (AN)∗ × X

55






Inverse DFT

Finally, the IDFT uses the DTF...

x = AN−1 × X

x∗ =(AN−1)∗ × X∗

x∗ = (NAN∗)∗ × X∗

x∗ = NAN × X∗

x = (NAN × X∗)∗

The computation of IDFT of the signal X consists of:1 Computation of the DFT of X ∗;2 Multiplying by N;3 Take the conjugate...

56






Examples

1 x = (1, 1, 1, 1), N = 4.

X = 14

1 1 1 11 −j −1 j1 −1 1 −11 j −1 −j

×1111

=

1000

2 x = (1, 0, 1, 0), N = 4.

X = 14

1 1 1 11 −j −1 j1 −1 1 −11 j −1 −j

×1010

=

120120

3 x = (1, 1, 0, 0), N = 4.

X = 14

1 1 1 11 −j −1 j1 −1 1 −11 j −1 −j

×1100

=

12

1−j401+j4

57






Examples

x = (1, 1, 0, 0), N = 4.

DFT:

X = ANx = 14

1 1 1 11 −j −1 j1 −1 1 −11 j −1 −j

×1100

=

12

1−j401+j4

IDFT:x = (AN)−1 X:

1 DFT of X∗: X∗ =

121+j40

1−j4

,

DFT (X∗) = 14

1 1 1 11 −j −1 j1 −1 1 −11 j −1 −j

×

121+j40

1−j4

= 14

1100

2 Multiplying by 4:

[1 1 0 0

]T3 Conjugate: xT =

[1 1 0 0

]

58





DefinitionRemarksPropertiesSeparable transformationExamples

Bi-dimensional Fourier transformation

1 Reminder



4 Time sampling


6 Bi-dimensional FouriertransformationDefinitionRemarksPropertiesSeparable transformationExamples



59






Continuous and bi-dimensional Fourier transformation

This is again a frequential decomposition in terms of exponential basisimages

Inverse Fourier Transform:

im(x , y) =

∫ +∞

−∞

∫ +∞

−∞IM(fx , fy )exp(j2π(fxx + fyy))dxdy

Fourier Transform:

IM(fx , fy ) =

∫ +∞

−∞

∫ +∞

−∞im(x , y)exp(−j2π(fxx + fyy))dxdy

fx is the x-direction spatial frequencies (m−1), Tx = 1fx;

fy is the y -direction spatial frequencies (m−1), Ty = 1fy.

60






Discrete and bi-dimensional Fourier transformation

im[k, l ] is a discrete picture having a size N ×M.Inverse Fourier Transform:

im[k, l ] =N−1∑u=0

M−1∑v=0

IM[u, v ]exp(j2π(kN

u +lM

v))

Fourier Transform:

IM[u, v ] =1√NM

N−1∑k=0

M−1∑l=0

im[k, l ]exp(−j2π(kN

u +lM

v))

u = 0, . . . ,N − 1, υk = uN , υk ∈ [0, 1];

v = 0, . . . ,M − 1, υl =vM , υl ∈ [0, 1].

The unit is cycles per unit length (pel−1).

61






Continuous and bi-dimensional Fourier transformation

A frequential decomposition in terms of exponential basis images

im[k, l ] = IM[0, 0]× exp(j2π(0× k + 0× l)) +IM[0, 1]× exp(j2π(0× k + l)) +IM[1, 0]× exp(j2π(k + 0× l)) + . . .

im[k, l ] = IM[0, 0] +IM[0, 1] (cos(2πl) + jsin(2πl)) +IM[1, 0] (cos(2πk) + jsin(2πk)) +IM[1, 1] (cos(2π(k + l)) + jsin(2π(k + l))) + . . .

IM[u, v ] are the weighting coefficients of the spatial frequencies presentin the signal.

62






Some remarks:

1 A picture having a size of N ×M is a linear combination of N ×Mexponential basis images!!!!

2 The DFT of a picture yields a complex picture of the same size;

63






Some remarks:

3 IM[u, v ] and im[k, l ] are periodic of infinite extent. Period = N for k, u. Period =M for l , v :

The image imt is bandlimited and is spatially sampled, for example, with arectangular array of N ×M pels:

im[k, l ] =

N−1∑i=0

M−1∑j=0

imt(k, l)δ(k − iTK )δ(l − jTL)

64






Some remarks:

65






Some remarks:

4 Sampling frequency: Te = 1 pel, Fe = 1 pel−1. To avoid aliasing, Shannon’stheorem has to be verified: Fe > 2Fmax . In our case, Fmax = 1

2

umax = N2 ;

vmax = M2 .

The sampling rate must be big enough to recover the signal.

66






Translation

Translation in the frequency domain

im[k, l ]exp(

j2πN

(u0k + v0l))

DFT−→ IM[u − u0, v − v0]

Translation in the spatial domain

im[k − k0, l − l0]DFT−→ IM[u, v ]exp

(−j

2πN

(k0u + l0v))

The multiplication by an exponential function in a representationmodifies the origin in the other representation.

67






Application of the translation’s property

Estimation of a translation between two consecutive pictures (Motion estimation):

imt+1[x + x0, y + y0] = imt [x , y ]

imt+1[x , y ] = imt [x − x0, y − y0]

DFT:

F(imt+1[x , y ]

)= F

(imt [x − x0, y − y0]

)IMt+1[u, v ] = IMt [u, v ]exp

(−j

2πN

(x0u + y0v)

)IMt+1[u, v ]IMt [u, v ]

∗= IMt [u, v ]IMt [u, v ]

∗exp(−j

2πN

(x0u + y0v)

)IMt+1[u, v ]IMt [u, v ]∗

|IMt |2= exp

(−j

2πN

(x0u + y0v)

)

F−1

(IMt+1[u, v ]IMt [u, v ]∗

|IMt |2

)= F−1

(exp

(−j

2πN

(x0u + y0v)

))

F−1

(IMt+1[u, v ]IMt [u, v ]∗

|IMt |2

)= δ(x − x0, y − y0)

68






Application of the translation’s property

Estimation of a translation between two consecutive pictures (Motion estimation):

imt+1[x , y ] = imt [x − x0, y − y0]

Finally, the algorithm consists of:

1 Computation of(

IMt+1[u,v ]IMt [u,v ]∗

|IMt |2

);

2 DFT−1;3 Detection of the most important peak (its spatial position indicates the amount

of translation).

69






Rotation

Let start by defining the polar coordinates in the spatial and frequencydomain:

Spatial:∣∣∣∣k = r × cosθl = r × sinθ

Frequency:∣∣∣∣u = w × cosθv = w × sinθ

We can show that:

im(r , θ + θ0)DFT−→ IM (w , φ+ θ0)

70






Average value

im =1

NM

N−1∑k=0

M−1∑l=0

im[k, l ]

IM[u, v ] =1√NM

N−1∑k=0

M−1∑l=0


u +lM

v))

IM[0, 0] =1√NM

N−1∑k=0

M−1∑l=0

im[k, l ]

im =1√NM

IM[0, 0]

IM[0, 0] is proportional to the average value of the picture.

71






Laplacian of im

∇2im[k, l ] =∂im∂k2 +

∂im∂l2

From the property of the differentiation (dn im[k,l ]dkn

F−→ ( j2πuN )nIM[u, v ]),

we obtain:

∇2im[k, l ] DFT−→ (2jπN

)2(u2 + v2)IM[u, v ]

F(∇2im[k, l ]

)=

∂

∂k2F (im[k, l ]) +∂

∂l2F (im[k, l ])

This property can be used to detect edges (Compute DFT of im,Multiplication and DFT−1).

72

Separable transformation = separable basis decomposition

The 2D DFT can be computed as the separable product of two one-dimensionaldiscrete Fourier transforms.

IM[u, v ] =1√

NM

N−1∑k=0

M−1∑l=0


u +lM

v))

Initialy the family{

ek,l [u, v ] = exp(−j2π( kN u + l

M v))}

0≤k<N,0≤l<Mis used, but we

can also use the family {ek(u)el (v)}0≤k<N,0≤l<M .

IM[u, v ] =1√

N

N−1∑k=0

exp(−j2πkN

u)

[1√

M

M−1∑l=0

im[k, l ]exp(−j2πlM

v)

]

IM[u, v ] =1√

N

N−1∑k=0

exp(−j2πkN

u)[√

MDFT1D(im[k, l ])]

IM[u, v ] =√

NDFT1D

[√MDFT1D(im[k, l ])

]We start by computing the Fourier coefficients on the image rows int the basis

{el}0≤l<M . A second transform is applied on the columns of the transformed imagesin the basis {ek}0≤k<N .







Matrix formulation:

IM[u, v ] =1√

NM

N−1∑k=0

M−1∑l=0


u +lM

v))

IM[u, v ] =1√

N

N−1∑k=0

exp(−j2πkN

u)

[1√

M

M−1∑l=0

im[k, l ]exp(−j2πlM

v)

]

IM[u, v ] =1√

N

N−1∑k=0

WNuk

[1√

M

M−1∑l=0

im[k, l ]WMvl

]

IM[u, v ] =1√

N

N−1∑k=0

NAN(u, k)

[1√

M

M−1∑l=0

im[k, l ]MAM(v , l)

]

IM[u, v ] =√

NMN−1∑k=0

AN(u, k)[imAT

M

]IM =

√NMAN imAT

M

74







75







For a picture having a size N × N, there are three ways to compute its DFT:

DFT 1D: N2 multiplications and N(N − 1) additions, O(N2)

X [k] =1N

N−1∑n=0

x[n]exp(−j2πN

nk)

DFT 2D: N2 × N2 multiplications and N2(N − 1)2 additions, O(N4)

IM[u, v ] =1N

N−1∑k=0

N−1∑l=0


u +lN

v))

DFT 2D separable transform: 2N × N2 multiplications and 2N × N(N − 1)2

additions, O(N3)

76






Examples

Remark: the spatial frequency amplitude spectra of natural images are well describedby a power law X (f ) = k

f β . Estimated average values of β are ranged over 0.9 and1.2. The power always falls with spatial frequency.

Power spectrum of a natural image (solid line) compared with 1f 2 (dashed line).

From E.P. Simoncelli and B.A.Olhausen, Natural Image Statistics and Neural Representation, Annun. Rev.

Neuroscience, 2001.77






Examples

The spectra may be more or lessanisotropic.

From R.M. Balboa, N.M. Grzywacz, Power spectra and distribution of contrasts of natural images from different

habitats, Vision Research, 43, pp. 2527-2537, 2003.

78






Examples

From left hand-side to right: original picture, spectrum, contour lines.

Remarks:

Fourier modulus of real images is even;

Fourier phase of real images is odd;

For display purpose, a logarithm is applied on the spectrum.

79






Examples

Geometric contours are mostly contained in the phase;Textures are mostly contained in the modulus.

Exchanging the modulus and the phase of two images:

A. V. Oppenheim and J. S. Lim, The importance of phase in signals, Proceedings of the IEEE, 1981 and adapted

from B. Galerne’s lecture.

80






Examples

Geometric contours are mostly contained in the phase;Textures are mostly contained in the modulus.

Exchanging the modulus and the phase of two images:

A. V. Oppenheim and J. S. Lim, The importance of phase in signals, Proceedings of the IEEE, 1981 and adapted

from B. Galerne’s lecture.

81






Examples

Forensic applications:

From left to right: original; Fourier modulus; Modified Fourier modulus; Result.

Adapted from http://www.robots.ox.ac.uk/~az/lectures/ia/lect2.pdf.

82

http://www.robots.ox.ac.uk/~az/lectures/ia/lect2.pdf





ContextCooley-Tukey algorithm

Fast Fourier transform

1 Reminder



4 Time sampling



7 Fast Fourier transformContextCooley-Tukey algorithm


83






Context

The Discrete Fourier Transform would normally require O(N2) operations to processN samples:

X [k] =1N

N−1∑n=0

x[n]exp(−j2πN

nk)

In practice, the previous formula is not used:

Fast Fourier Transform takes O(Nlog2N) operations;

Most common algorithm is the Cooley-Tukey algorithm (a divide and conqueralgorithm).

84






Cooley-Tukey algorithm

The Discrete Fourier Transform is defined by the formula (N = 2P)

X [k] =1N

N−1∑n=0

x[n]exp(−j2πN

nk)

X [k] =2N

N/2−1∑m=0

x[2m]exp(−2πjN

(2m)k) +2N

N/2−1∑m=0

x[2m + 1]exp(−2πjN

(2m + 1)k)

One can factor a common multiplier exp(− 2πjN k) out of the second sum:

X [k] =2N

N/2−1∑m=0

x[2m]exp(−2πjN/2

mk)︸︷︷︸DFT of even-indexed part of x[n]

+exp(−2πjN

k)2N

N/2−1∑m=0

x[2m + 1]exp(−2πjN/2

mk)︸︷︷︸DFT of odd-indexed part of x[n]

85







X [k] =2N

N/2−1∑m=0

x[2m]exp(−2πjN/2

mk)︸︷︷︸E [k]

+exp(−2πjN

k)2N

N/2−1∑m=0

x[2m + 1]exp(−2πjN/2

mk)︸︷︷︸O[k]

E [k] is the DFT of even samples whereas O[k] is the DFT of odd samples. TheseDFTs have a length N/2 and are periodic (period N/2).

X [k] = E [k] + W kN × O[k]

With, W kN = exp(− 2πj

N k), W kN = 1 for k even and W k

N = −1 for k odd.

The N-point DFT X [k] can be obtained from two N2 -point transforms: one on even

input data and one on add input data. Although the frequency index n ranges over Nvalues, only N

2 values need to be computed (periodic with period N2 ).

86






Cooley-Tukey algorithmExample for N = 8: X [k] = E [k] + W k

N × O[k]

Even input data: X [0], X [2], X [4], X [6].Odd input data: X [1], X [3], X [5], X [7].

X [0] = E [0] + W 08 × O[0]

X [1] = E [1] + W 18 × O[1]

X [2] = E [2] + W 28 × O[2]

X [3] = E [3] + W 38 × O[3]

X [4] = E [0] + W 48 × O[0] = E [0]−W 0

8 × O[0]

X [5] = E [1] + W 58 × O[1] = E [1]−W 1

8 × O[1]

X [6] = E [2] + W 68 × O[2] = E [2]−W 2

8 × O[2]

X [7] = E [3] + W 78 × O[3] = E [3]−W 3

8 × O[3]

87







88





DefinitionMatrix formulationPropertiesExamples of discrete unitary transforms

Discrete unitary transform

1 Reminder



4 Time sampling




8 Discrete unitary transformDefinitionMatrix formulationPropertiesExamples of discrete unitarytransforms

90






Discrete unitary transform

Discrete unitary transform ⇐⇒ Discrete Linear orthonormal transform.

Discrete unitary transforms are linear and reversible.

im T→ IM, resolution=N × N

Inverse Unitary Transform:

im[k, l ] =

N−1∑u=0

N−1∑v=0

IM[u, v ]ak,l (u, v)

Unitary Transform:

IM[u, v ] =

N−1∑k=0

N−1∑l=0

im[k, l ]a∗k,l (u, v)

The basis images on which the decomposition is performed:{ak,l (u, v)

}0≤u,v<N .

91






Discrete unitary transform for 2D signal N × N

~IM = A ~im

A is a matrix N2 × N2:

A =

[ak,0(u, 0)

]0≤k,u<N

[ak,1(u, 0)

]0≤k,u<N · · ·[

ak,0(u, 1)]0≤k,u<N

[ak,1(u, 1)

]0≤k,u<N · · ·

......

...[ak,0(u,N − 1)

]0≤k,u<N

[ak,1(u,N − 1)

]0≤k,u<N · · ·

~im is a vector of size N2:

im(0, 0)im(1, 0)

...im(N − 1, 0)

im(0, 1)

...im(N − 1, 1)

...im(0,N − 1)

...im(N − 1,N − 1)

~im = A−1 ~IM

with, A−1 = A∗T .

92






Properties

A transformation is unitary when A−1 = A∗T

A unitary transformation is separable:

ak,l [u, v ] = aC [u, k]× aR [v , l ]

where aC [u, k] is applied on the colomns and aR [v , l ] is applied on the rows.

IM(u, v) =

N−1∑k=0

N−1∑l=0

im(k, l)ak,l [u, v ]

=

N−1∑k=0

aC [u, k]

N−1∑l=0

aR [v , l ]im[k, l ]

︸︷︷︸Transformation 1D on the kth row.

93






Properties

A unitary transformation is separable: O(N4)→ O(N3).

~IM = A ~im

= AC ~imATR

where:→ The columns of AC is composed of {aC (k)}0≤k<N ;→ The columns of AR is composed of {aR (l)}0≤l<N ;

{aC (k)}0≤k<N and {aR(l)}0≤l<N are two orthonormal basis of CN .Inverse transform:

~im = A−1C

~IMA−1R

T

= A∗TC~IMA∗R

94






Discrete 2D Fourier Transform

Discrete Fourier transform (image N × N)

IM[u, v ] =

N−1∑k=0

N−1∑l=0

im[k, l ]ak,l [u, v ]

ak,l [u, v ] =1N

exp(−j2π(kN

u +lN

v))

Separable transform:

ak,l [u, v ] =1√

Nexp(−j2π

kN

u)︸︷︷︸aC [u,k]

1√

Nexp(−j2π

lN

v)︸︷︷︸aR [v,l ]

Note that A = AC = AR = 1√N

[Wαβ

]0≤α,β<N .

A is then complex and symmetric: aC (k, u) = aC (u, k).

95







A is unitary?

〈ak , al 〉 =

{1 k = l0 k 6= l

where ak and al represent the columns and lines of A.

1N

N−1∑p=0

W pkW−pl =1N

N−1∑p=0

W p(k−l)

if k = l , W p(k−l ) = W 0 = 1, 1N∑N−1

p=0 W pkW−pl = 1;

if k 6= l , 1N∑N−1

p=0 W p(k−l ) = 1N

1−WN(k−l )

1−W (k−l )= 0.

A is unitary:

A−1 = A∗T

and is A symmetric

A−1 = A∗

96







Inverse Discrete Fourier transform (image N × N)

im = A∗IMA∗

Forward Discrete Fourier transform (image N × N)

IM = AimA

where A = 1√N

[Wαβ

], 0 ≤ α, β < N − 1.

97






Discrete Cosine Transform

The DFT transforms a complex signal into its complex spectrum. However, if thesignal is real as in most of the applications, half of the data is redundant:

In time domain: the imaginary part of the signal is all zero;

In frequency domain: the real part of the spectrum is even symmetric andimaginary part odd.

How to avoid this high redundancy? We would need a real unitarytransform that transforms a sequence of real data points into its real

spectrum.

98







Let x[m] a real signal defined over N sample (m = {0, . . . ,N − 1}).Construction of a new sequence of 2N samples:

xp [m] =

{x[m] 0 ≤ m < N

x[−m − 1] −N ≤ m ≤ −1

xp [m] is now even symmetric with respect to the point m = − 12 .

we define m′ = m + 12 , to get an even symmetry with respect to m′ = 0. The

DFT of this 2N-point even symmetric sequence:

X [n] =1√2N

N− 12∑

m′=−N+ 12

xp [m′ −12

]exp(−j2π2N

m′n)

=1√2N

N− 12∑

m′=−N+ 12

xp [m′ −12

]cos(2πm′n2N

)−j√2N

N− 12∑

m′=−N+ 12

xp [m′ −12

]sin(2πm′n2N

)

xp [m] is even and sin(π(2m+1)n2N ) is odd. The second term is then null.

99







X [n] =1√2N

N− 12∑

m′=−N+ 12

xp [m′ −12

]cos(2πm′n2N

)

X [n] is then real and even X [n] = X [−n]. We replace m′ with m + 12 .

X [n] =

√2N

N−1∑m=0

x[m]cos(π(2m + 1)n

2N)

with n = {0, . . . , 2N − 1}.

100







Direct DCT:

X [n] =

√2N

N−1∑m=0

x[m]cos(π(2m + 1)n

2N)

X [n] =

N−1∑m=0

x[m]a[n,m]

with n,m = {0, . . . ,N − 1} and a[n,m] =√

2N cos(π(2m+1)n

2N ).To make this transformation unitary (orthonormal),the previous formula is multiplied by λ[n]:

λ[n] =

√

1N n = 0√2N n 6= 0

Inverse DCT:

x[n] =

N−1∑m=0

x[m]a[n,m]

101







Direct 2D DCT:

X [u, v ] = λ(u)λ(v)

N−1∑k=0

N−1∑l=0

x[k, l ]cos(π(2k + 1)u

2N)cos(

π(2l + 1)v2N

)

λ(u) = λ(v) =

√

1N n = 0√2N n 6= 0

Inverse 2D DCT:

x[k, l ] =

N−1∑u=0

N−1∑v=0

λ(u)λ(v)X [u, v ]cos(π(2k + 1)u

2N)cos(

π(2l + 1)v2N

)

λ(u) = λ(v) =

√

1N n = 0√2N n 6= 0

102






Two-dimensional Discrete Hadamard Transform

Direct 2D Hadamard tansform:

IM[u, v ] =1N

N−1∑k=0

N−1∑l=0

im[k, l ](−1)∑n−1

i=0 (bi (k)bi (u)+bi (l)bi (v))

with, N = 2n, bi (z) is the i th bit in the binary representation of z.

Inverse 2D Hadamard transform:

im[k, l ] =1N

N−1∑u=0

N−1∑v=0

IM[u, v ](−1)∑n−1

i=0 (bi (k)bi (u)+bi (l)bi (v))

The kernel is a(k, l , u, v) = 1N (−1)

∑n−1i=0 (bi (k)bi (u)+bi (l)bi (v)). The kernel is orthogonal

and separable (aC (k, u) = 1√N

(−1)∑n−1

i=0 (bi (k)bi (u))).

103







Example of computation of kernel’s coefficients (1D): N = 8 (for the sake of clarity,we won’t consider 1√

N):

a[k, u] = 1√N

(−1)∑n−1

i=0 (bi (k)bi (u))

a[0, 0] = (−1)∑2

i=0(bi (0)bi (0)) = (−1)0+0+0 = 1;

a[0, 1] = (−1)∑2

i=0(bi (0)bi (1)) = (−1)0+0+0 = 1;

...

a[7, 1] = (−1)∑2

i=0(bi (7)bi (1)) = (−1)1+0+0 = −1;...

a[7, 7] = (−1)∑2

i=0(bi (7)bi (7)) = (−1)1+1+1 = −1.

104







Matrix formulation:Direct 2D Hadamard tansform:

IM = HnimHn

with, N = 2n, Hn represents an N × N Hadamard with element values either +1 or−1.

Inverse 2D Hadamard transform:

im = HnIMHn

The Hadamard matrix of the order n is generated in terms of Hadamard matrix oforder n − 1 using Kronecker product ⊗ given by:

H1 =1√2

[1 11 −1

], Hn = Hn−1 ⊗H1

Then for N = 2n equal to a power of two: Hn = 1√N

[Hn−1 Hn−1Hn−1 −Hn−1

]105







An advantage of the Hadamard transform is that most of the computation requiresonly addition and substraction.

The matrices generated by the Kronecker products H2 are in natural order, whichmeans that they are not ordered with respect to the number of sign changes in eachrow.

H3 = 1√8

1 1 1 1 1 1 1 11 −1 1 −1 1 −1 1 −11 1 −1 −1 1 1 −1 −11 −1 −1 1 1 −1 −1 11 1 1 1 −1 −1 −1 −11 −1 1 −1 −1 1 −1 11 1 −1 −1 −1 −1 1 11 −1 −1 1 −1 1 1 −1

The number of sign changes along a column is called the sequency of that column.For H3: 0,7,3,4,1,6,2,5.

106







The rows of the Hadamard matrices are the Walsh functions. The same as Walshkernel, but some of the rows are intermixed. It could be important to order tesequence such that it increases as u increases (Walsh Transform).

H3 = 1√8

1 1 1 1 1 1 1 11 1 1 1 −1 −1 −1 −11 1 −1 −1 −1 −1 1 11 1 −1 −1 1 1 −1 11 −1 −1 1 1 −1 −1 11 −1 −1 1 −1 1 1 −11 −1 1 −1 −1 1 −1 11 −1 1 −1 1 −1 1 −1

For H3: 0,1,2,3,4,5,6,7.

107

Conclusion

1 Reminder



4 Time sampling





Image Processing Image Transform and Fourier/Wavelet Transform

Documents

Transcript of Image Processing Image Transform and Fourier/Wavelet Transform