ESS 265 Spring Quarter 2005

Time Series Analysis: Some Fundamentals of Spectral

Analysis

Lecture 12May 12, 2005

Fourier Series

• Any periodic function (a(t+t)=A(t)) where ω=2π/T is the period can be expressed as a Fourier series

• a(t) must satisfy the condition

• Any “reasonable” function satisfying the above condition can be expanded as a function of sin and cos – complete

• To find the coefficients use the following relationships which result because sin and cosare orthogonal.

)sincos()(1

021 tnstncata n

nn ωω ++= ∑

∞

=

∞<∫ dttaT

0)(

∫∫∫

==

==

=

T

Tn

T

Tn

T

T

ndttntas

ndttntac

dttaa

02

02

02

0

...2,1,0;sin)(

,...2,1,0;cos)(

)(

ω

ω

Some Useful Properties of Fourier Series

• Periodicity – Fourier series are periodic with defined period. The Fourier series converges on the required function only in the given interval.

• Even and odd functions- The sine is odd (a(-t)=-a(t)) while cosine is even (a(-t)=a(t)). Fit even functions with cosine and odd function with sine.

• Half-period series – A function a(t) defined only on the interval (0,T) can be fit with either sines or cosines.

• Least squares approximation- When a Fourier series expansion of a continuous function a(t) is truncated after N terms it is a least squares fit to the original. The area of the squared difference between the two functions is minimised and goes to zero as N increases.

• Gibbs phenomenon- The Fourier series converges in the least-squares sense when a(t) is discontinuous but the truncated series oscillates near the discontinuity.

The Fourier Integral • The Fourier integral transform (FIT) F(ω) of a function f(t) is defined as

• The inverse transform is

• The right hand side is finite if

∫∞

∞−= dtetfF tiω

πω )(

21)(

∫∞

∞−= ωω ω deFtf ti)()(

∞<∫∞

∞−dttf )(

f(t) ↔ F(ω)Shift theorem

f(t-t0) ↔ e-iwt0F(ω)

Derivative df/dt ↔ -iωF(ω)

Convolution theorem

↔ F(ω)G(ω)

Symmetry F(t) Real ↔ F(-ω)=F*(ω)

Parseval =

∫∞

∞−− ''' )()( dtttgtf

ωωπ dF2

)(2 ∫∞

∞−dttf2

)(∫∞

∞−

The Fourier Integral Transform Continued

• The condition is very restrictive. It means that simple functions like f(t)=const. or sin ωt won’t work.

• This can be fixed by using the Dirac delta function δ(t) (δ(t)=0, t≠0; )

• The substitution property makes integration trivial

• Apply the definition of the FIT to the delta function

• .This gives

• For monchromatic waves

∞<∫∞

∞−dttf )(

∫∞

∞−= )0()()( fdttftδ

)()()( 00 tfdttftt =−∫∞

∞−δ

πδ

πω

21)(

21

=∫∞

∞−

− dtet ti

)(2)(2);(2);(2 ωπδωπδωπδπδω ωωω =−=== ∫ ∫∫∞

∞−

∞

∞−

−∞

∞−

tititi edtetde

( ) ( )[ ]

( ) ( )[ ]000

000

21sin

21

21cos

21

ωωδωωδωπ

ωωδωωδωπ

+−−=

++−=

∫

∫∞

∞−

∞

∞−

itdt

tdt

The z-Transform• Assume we have series of measurements in evenly time (or space)

{a}= a0,a1,a2,…,aN-1

• A z-transform is made by creating polynomial in the complex variable z using

• Operations on the z –transform all have counterparts in the time domain. Imagine multiplying the z-transform by z

• This new transform is that you would get if you shifted the original time series by one unit in time. In this case z is called the unit delay operator.

• Multiplication of two z-transforms is called discrete convolution. The discrete convolution theorem is what give the z-transform its power.

• Consider the product of A(z) and B(z) each of different length.

11

2210 ....)( −

−+++= NN zazazaazA

NN zazazazazzA 1

32

210 ....)( −++++=

lM

ll

kN

kk zbzazBzAzC ∑∑

−

=

−

=

==1

0

1

0

)()()(

The z-Transform Continued

• Set p=k+l and change the order of summation

• This is the z-transform of a time series of length N+M-1. • c=a*b is the discrete convolution of a and b• The discrete convolution theorem for z-transformations is given by the

following notation:

• Division by a z-transform is deconvolution. As long as a0≠0 then b0=c0/a0 and

• This is a recursive procedure because the result of each equation is used in all subsequent equations.

pkp

NM

p

p

kk

lkl

N

k

M

lk zbazba −

−+

= =

+−

=

−

=∑ ∑∑∑ =

2

0 0

1

0

1

0

∑=

−=p

kkpkp bac

0

)()()()(

zBzAbazBbzAa

↔∗↔↔

0

1

a

bacb

p

kkpkp

p

⎟⎟⎠

⎞⎜⎜⎝

⎛−

=∑

=−

The Discrete Fourier Transform• Substitute z=e-iω∆t into the z-transform equations and normalize by N

• This equation is a complex Fourier series that is a continuous function of frequency that has been discretised so that

where ∆ν is the sampling frequency.• The discrete Fourier transform is given by

• This formula transforms the N values of the time sequences {ak} } into another sequence of numbers with N Fourier coefficients {Ak}.

• The inverse equation to recover the original time series from the Fourier coefficients is:

• k measures time in units of ∆t up to a maximum of T=N∆t. n measures frequency intervals of ∆ν=1/T up to a maximum of vs=N∆ν=1/ ∆t .

tkiN

kkea

NA ∆−

−

=∑= ωω

1

0

1)(

νπππω ∆=∆

== ntNn

Tn

n 222

1,...,2,1,0;1)( /21

0−=== −

−

=∑ Nnea

NAA Nink

N

kknn

πω

NiknN

nnk eAa /2

1

0

π∑−

=

=

Amplitude, Power and Phase Spectra• The Fourier coefficients An describe the contribution of the particular

frequency ω=2πn∆ν to the original time sequence.

• A signal with just one frequency is a sine wave.– Rn is the maximum – Φn defines the initial point in the cycle.

• Rn plotted against n is called the amplitude spectrum• Rn

2 plotted against n is the power spectrum.• Φn is an angle that describes thephase of this frequency with the time series

and the corresponding plot is a phase spectrum.• The Shift Theorem – multiplication of the DFT by e-iwDt will delay the sequece

by one sampling interval. In other words shifting the time sequence one space will multiple the DFT coefficient An by e-2πin/N. The power spectrum in not changed by the phase is retarded by 2πn/N.

• In deriving the convolution theorem we omitted terms in the sum involving elements of a or b with subscripts which outsde of the specified ranges, 0 to N-1 for a and 0 to M-1 for b. This is no longer correct of periodic functions. Practically this made to work by padding with zeros to extend both series to length N+M+1.

ninn eRA Φ=

Differentiation and Integration

• Differentiation and integration only apply to continuous functions of time so we set t=k∆t and wn=2πn/N∆t so the DFT becomes

• Differentiating with respect to time gives

• Make this discrete by setting t=k∆t and call so that

• This is the inverse DFT so and iωnAnmust be transforms of each other. • Differentiation with respect to time is equivalent to multiplication by frequency in the

frequency domain. • Integration with respect to time is equivalent to division in the frequency domain.

tiN

nn

neAta ω∑−

=

=1

0)(

tin

N

nn

neAidtda ωω∑

−

=

=1

0

tktk dt

daa∆=

=& NiknN

nnnk eAia /2

1

0

πω∑−

=

=&

ka&

Parseval’s Theorem

• From the definition of the DFT we find

• Using Nyquist ‘s theorem where δkl is the Kroneckerdelta.

• Digression on the Kronecker delta - δkl =0 for k≠l and =1 for k=l such that

• Parseval’s theorem guarantees the equality of energy between the time and frequency domains.

∑∑−

=

−−

=

=1

0

/21

0

/22

2 1 N

l

Ninll

N

k

Ninkkn eaea

NA ππ

∑−

=

−− =1

0

/)(21 N

nkl

NlkineN

δπ

lk

kkl aa =∑δ

kllk

lk

N

n

Nklin

lklk

N

nn aa

Neaa

NA δπ ∑∑∑∑ ==

−

=

−−−

= ,

1

0

/)(2

,2

21

0

11

21

0

21

0

1 ∑∑−

=

−

=

=N

kk

N

nn a

NA

The Fast Fourier Transform• The DFT requires a sum over N terms for each of N frequencies. Thus the

total number of calculations required goes as N2. This was a major impediment to doing spectral analysis.

• The fast Fourier transform (FFT) allowed this to be done much faster. • Suppose that N is divisible by 2. Split the DFT into two parts.

• This sum requires us to form the quantity in () (N/2 calculations) and then doing this N/2 times. For all frequencies this means calculations or a reduction of a factor of 2 for large calculations.

• This procedure can be repeated. As long as N is a power of 2 the sum can be divided log2N times, with a total of 4Nlog2N operations.

( ) NinkN

k

inNkk

N

Nk

NinkNinkN

kk eeaaeea /2

12/

02/

1

2/

/2/212/

0

ππππ −−

=

−+

−

=

−−−

=∑∑∑ +=+

1,...,2,1,0;1)( /21

0

−=== −−

=∑ Nnea

NAA Nink

N

kknn

πω

( ) 2/2/ NNN +×

Aliasing and Shannon’s Sampling Theorem• The transformation pair and

are exact. Before digitizing we had continuous functions of time.• The periodicity of the DFT gives AN+n=An.• If the data are real take the complex conjugate and show that AN-n=An

*.

• These equations define aliasing which is the mapping of higher frequencies into the range 0 to N/2 – after digitisation higher frequencies are mapped to lower frequencies.

• Fourier coefficients for frequencies above N/2 are determined exactly from the first N/2+1.

• Above N they are periodic and between N/2 and N the reflect with the same amplitude and phase change of π.

• The DFT allows us to transform N real values in the time domain into any number of complex values An.

• The highest meaningful coefficient in the DFT is AN/2 and the corresponding frequency is the Nyquist frequency νN=1/2∆t.

• We can recover the original signal from digitized samples provided the original sample contained no energy above the Nyquist frequency.

• To reproduce the original time series a(t) from its samples ak=a(k∆t) we can use Shannon's theorem

1,...,2,1,0;1)( /21

0

−=== −−

=∑ Nnea

NAA Nink

N

kknn

πω NiknN

nnk eAa /2

1

0

π∑−

=

=

( )( )tkt

tktataN

NK

kk ∆−

∆−= ∑

−

= πυυπsin)(

1

0

Tapering• In an ideal universe data would be a continuous function of time going

forever, then the Fourier integral transform would give the spectrum.– In reality data are limited and the finite length T limits the frequency spacing to – The sampling interval ∆t limits the maximum meaningful frequency to νN.

• It is frequently useful to assume a finite time series is a section of an infinite series. This is called windowing and is achieved by multiplying the time series by a box car (called a taper) that is zero outside of the window and one inside.

• Since multiplication in the time domain is the same as convolution in the frequency domain this is equivalent to convolution with the DFT of a box car which is a spike and side lobes.

– A single peak is spread across a range of frequencies.– This is called spectral leakage.

• We can improve this by using a window with different side lobes. For resolving peaks we want a narrow function in the frequency domain but that means a broad function in the time domain – the uncertainty principle.

• Time windowing can reduce noise by smoothing the spectrum- noise reduction comes at the expense of resolution.

T1=∆υ

Filtering: The Running Average• Filtering is convolution with second- usually shorter- time series.

– Bandpass filters eliminate ranges of frequencies from the time series.– Low-pass filters eliminate all frequencies above a certain frequency.– High-pass filters eliminate all frequencies above a certain frequency.– The range of frequencies allowed is called the pass band.– The critical frequencies are called cut-off frequencies.

• A running average in which each member of a time series is replaced by an average of M neighboring members is a filter.

– It is a convolution of the original time sequence with a boxcar function.– Intuitively we would expect the running average to remove high frequencies and

therefore be a low-pass filter. – Since convolution the time domain is multiplication in the frequency domain we

have multiplied the DFT of the time series with the Fourier transform of the boxcar.

– This is not an ideal low-pass filter because of the side lobes and the central peak let through a lot of energy above the desired frequency.

– The amplitude spectrum is exactly zero at the frequencies (nN/MT) where the box car transform is zero so this filter is great if you want to eliminated a given frequency.

The Fourier Transform of a Box Car

• The Discrete Fourier Transform of a box car has a central peak and oscillations in frequency.

Filtering: Some Examples• An ideal low-pass filter should have an amplitude spectrum that is zero

outside of the cut-off frequency and one inside it. Gibb's phenomenon prevents such a filter from doing a good job as a low-pass filter.

• We need to taper the filter like we tapered the window.

• Gibbons uses the Butterworth filter ( ) where ωC is the cut-off frequency at which the energy is halved. n controls the sharpness of

the cut-off.

• The corresponding high-pass filter is .

• A bandpass filter can be made by shifting the frequency along the frequency

axis to center it around ωb .

• A notch filter can be made from .

• To construct the time sequence specify the phase (usually zero) and take

the inverse Fourier transform.

( )( ) n

C

F 22

1 11ωω

ω+

=

( ) )(1 ωω lh FF −=

( )( )[ ] n

CbbF 2

2

11

ωωωω

−+=

( ) ( )ωω bn FF −=1

Correlation• The cross correlation of two time series a and b is defined by

where k is the lag, N and M are the lengths of the time series. • The sum is over all N+M+1 possible products. • An autocorrelation is a cross correlation of a time sequence with itself.

• The correlation coefficient is equal to the cross correlation normalized to give one when the two time series are identical (perfectly correlated).

• ψk is 1 for perfect correlation and -1 for anticorrelation.

∑ +−+=

ppkpk ba

MNc

11

pkp

pk aaN +∑−

=12

1φ

∑ ∑∑ +

=p p pppp

p pkpk

bbaa

baψ