Orthogonal Functions∗

May 9, 2003

1 Introduction

A particularly important class of applications of linear algebra within math-ematics arises from treating functions as vectors, with the rules

• (f + g)(x) = f(x) + g(x)

• (αf)(x) = αf(x), where α is a scalar.

• f · g =∫ b

af(x)g(x) dx.

Here, the interval [a, b] over which we integrate is regarded as fixed withinany particular discussion. Two important examples are

1. [a, b] = [0, 1] and the functions in question are taken to be polynomials.

2. [a, b] = [0, 2π] and the functions are periodic, e.g. trigonometric poly-nomials.

A set of functions on [a,b] is linearly independent if no linear combinationof those functions is identically 0 on [a,b]. In particular, the powers of x arelinearly independent on any interval, as are the functions sin(nx), cos(nx)and 1 on [0, 2π], for n and m positive integers. However, the latter set hasan additional important property: they are orthogonal. That is∫ 2π

0

sin(nx) cos(mx) dx =

∫ 2π

0

sin(nx) sin(mx) dx =

∫ 2π

0

cos(nx) cos(mx) dx = 0

∗Preliminary version: subject to later expansion

1

unless m = n, in which case∫ 2π

0

sin(nx) cos(nx) dx = 0;

∫ 2π

0

sin2(nx) dx =

∫ 2π

0

cos2(nx) dx = π.

The powers of x are not orthogonal on any interval. However if we confineour attention to any particular interval, such as [0, 1], we can use the Gram-Schmidt orthogonalization algorithm to produce orthogonal polynomials.

Suppose we have orthogonal functions {fi}0≤i≤n, and a function g =∑ni=0 aifi, which is a linear combination of the functions fi. Then∫ b

a

g(x)fj(x) dx =n∑

i=0

ai

∫ b

a

fi(x)fj(x) dx = aj

∫ b

a

fj(x)2 dx,

so that

aj =

∫ b

ag(x)fj(x) dx∫ b

afj(x)2 dx

,

and the coefficients aj can be recovered by integration.

2 Complex Exponentials and the Discrete Fourier

Transform

In dealing with periodic functions on [0, 2π], it is particularly useful to con-sider the complex exponential functions einx as n varies over the integers.From the power series

ex =∞∑0

xn

n= 1 + x +

x2

2+

x3

3+ · · · ,

cos(x) = 1− x2

2+

x4

4− x6

6+ · · · ,

and

sin(x) = x− x3

3+

x5

5− x7

7+ · · · ,

we recover the identity

eix = cos(x) + i sin(x),

2

which is also consistent with the trigonometric addition formulae.Because we are considering functions that take complex values (although

only as functions of the real variable x), we are, for the first time, consideringa complex vector space. This means that scalars are allowed to be complexnumbers, and also requires an adjustment to the dot product. We recall thatif z = x + iy, then z̄ = x− iy, and set

f · g =

∫ 2π

0

f(x)g(x) dx.

With this definition, einx · eimx = 2π if n = m, and 0 otherwise, sinceeinx = e−inx. In particular, if f(x) =

∑ane

inx, then

f · eimx =

∫ 2π

0

f(x)e−imx = 2πam,

and we can recover the coefficients am by integration, as in the strictly realcase.

Let us now consider how to best approximate the coefficients am for 0 ≤|m| ≤ N − 1 when we know the values f(2πk

N) for a real valued function f .

From the fact that f is real values, it follows that a−m = am for all m, so thatwe need only determine the coefficients am for m ≥ 0. The next observationis that we can approximate the integral that gives us 2πam by the Riemannsum

2π

N

N−1∑k=0

f(2πk

N)e−2πimk

N ,

so that

am ≈ 1

N

N−1∑k=0

f(2πk

N)e−2πimk

N =1

N

N−1∑k=0

f(2πk

N)Wmk

N ,

where we define WN to be e−2πi

N . If we set v to be the vector whose kth com-ponent is f(2π(k−1)

N) and MN to be the complex N × N symmetric matrix

whose (m, k)th entry is W(m−1)(k−1)N , the components of MN v are approxima-

tions to 2πNam 0 ≤ m ≤ N − 1. This approximation is called the discreteFourier transform. It tends to be rather crude for values of m close to N .

3

3 The Fast Fourier Transform

In order to appreciate what follows, one must think of N as very large (per-haps as large as 106), so that the N2 multiplications that are necessary tocompute the discrete Fourier transform will take significant time. We willsee now that there is a trick that allows us to do the computation with manyfewer multiplications if N is a power of 2. The assumption is that multipli-cation is much more time consuming than addition, so that it is much moreefficient to compute a(b + c + d) then ab + ac + ad.

It will be more appropriate for what follows to shift the components andindices so that they go from 0 to N − 1, vk = f(2π(k

N) and the (m, k)th

component of MN is W(m)(k)N . We make the observation that, for N even,

W 2N = WN

2, and W

N2

N = −1. We write veven for the vector formed from the

even components of v taken in order, and vodd for the vector formed from theodd components of v, also taken in order. From this it follows that the firstN2

components of MNv are the components of the vector of length N2

whosejth component, (ending with j = N

2− 1) is

(MN2veven)j + W j

N(MN2vodd)j,

while the (j + N2)th component of MNv (starting with j = 0) is

(MN2veven)j −W j

N(MN2vodd)j.

If we write µ(N) for the minimum number of multiplications required tocompute MNv, it follows from the foregoing that, for N even,

µ(N) ≤ 2µ(N

2) +

N

2.

We are not counting the multiplications required to compute the powers ofWN , because these can be stored and used repeatedly, and we are reallyinterested in the speed for which the computation can be carried out forfixed N and constantly changing v. Since

M2 =1 11 −1

,

it follows that µ(2) ≤ 1 and, from the foregoing by induction, that µ(2p) ≤p 2p−1.

4

4 Problems

1. Let f(x) be the function on [0, 1] defined by

f(x) = x, 0 ≤ x ≤ 1

2; f(x) = 1− x,

1

2≤ x ≤ 1.

Find the orthogonal projection of f on the space of quartic polynomials.Begin by finding an orthogonal basis for the space of quartic polynomi-als by applying Gram-Schmidt to the functions {1, x, x2, x3, x4}. Youare encouraged to use MATLAB for this purpose.

2. Let f(x) = π − |π − x| for 0 ≤ x ≤ 2π.

(a) Find the Fourier coefficients am = 12π

∫ 2π

0f(x)e−imx dx of f by

direct integration for |m| ≤ 4.

(b) Use the values of f at multiples of π2

and the discrete Fouriertransform to approximate the same coefficients.

5