EECS 16B Designing Information Devices and Systems IISpring 2018 J. Roychowdhury and M. Maharbiz Homework 9

This homework is due on Thursday, April 19, 2018, at 11:59AM (NOON).Self-grades are due on Monday, April 23, 2018, at 11:59AM (NOON).

1. Interpolation

Samples from the sinusoid f (x) = sin(0.2πx) are shown in Figure 1. Draw the results of interpolation usingeach of the following three methods:

Figure 1: Samples of f (x)..

(a) Zero order hold interpolation.

Solution:

Figure 2: Zero order hold..

EECS 16B, Spring 2018, Homework 9 1

(b) Linear interpolation.

Solution:

Figure 3: Linear interpolation..

(c) Sinc interpolation.

Solution:

Figure 4: Sinc interpolation..

2. Lagrange Interpolation by PolynomialsGiven n distinct points and the corresponding evaluations/sampling of a function f (x), (xi, f (xi)) for 0 ≤i ≤ n− 1, the Lagrange interpolating polynomial is the polynomial of the least degree that passes throughall the given points.

Given n distinct points and the corresponding evaluations, (xi, f (xi)) for 0 ≤ i ≤ n−1, the Lagrange poly-nomial is

Pn(x) =i=n−1

∑i=0

f (xi)Li(x),

where

Li(x) =j=n−1

∏j=0, j 6=i

(x− x j)

(xi− x j)=

(x− x0)

(xi− x0)· · · (x− xi−1)

(xi− xi−1)

(x− xi+1)

(xi− xi+1)· · · (x− xn−1)

(xi− xn−1)

EECS 16B, Spring 2018, Homework 9 2

.

Here is an example: for two data points, (x0, f (x0)) = (0, 4), (x1, f (x1)) = (-1, -3), we have

L0(x) =x− x1

x0− x1=

x− (−1)0− (−1)

= x+1

and

L1(x) =x− x0

x1− x0=

x− (0)(−1)− (0)

=−x

. ThenP2(x) = f (x0)L0(x)+ f (x1)L1(x) = 4(x+1)+(−3)(−x) = 7x+4

We can sketch those equations on the 2D plane as follows:

(a) Given three data points, (2,3), (0,−1), and (−1,−6), find a polynomial f (x) = ax2 + bx+ c fittingthe three points. Do this by solving a system of linear equations for the unknowns a,b,c. Is thispolynomial unique?

Solution:Plug in the three data points into f (x)= ax2+bx+c, we get 4a+2b+c= 3, c=−1 and a−b+c=−6.Solving these equations for the coefficients a, b and c, we get f (x) =−x2 +4x−1.This polynomial is the unique degree 2 polynomial defined by these three distinct points.

(b) Like the monomial basis {1, x, x2, x3, . . .}, the set {Li(x)} is a new basis for the subspace of degree nor lower polynomials. Pn(x) is the sum of the scaled basis polynomials. Find the Li(x) correspondingto the three sample points in (a). Show your work.

Solution:Assume x0 = 2, x1 = 0 and x2 =−1, each corresponding Li(x) is:

EECS 16B, Spring 2018, Homework 9 3

L0(x) =(x− x1)

(x0− x1)

(x− x2)

(x0− x2)=

(x−0)(2−0)

(x− (−1))(2− (−1))

=x2 + x

6

L1(x) =(x− x0)

(x1− x0)

(x− x2)

(x1− x2)=

(x−2)(0−2)

(x− (−1))(0− (−1))

=x2− x−2−2

L2(x) =(x− x0)

(x2− x0)

(x− x1)

(x2− x1)=

(x−2)(−1− (2))

(x−0)(−1−0)

=x2−2x

3

(c) Find the Lagrange polynomial Pn(x) for the three points in (a). Compare the result to the answer in (a).Are they different from each other? Why or why not?

Solution:

Pn(x) = f (x0)L0(x)+ f (x1)L1(x)+ f (x2)L2(x) = 3× x2 + x6

+(−1)× x2− x−2−2

+(−6)× x2−2x3

Therefore, Pn(x) =−x2 +4x−1.The answer is the same as the one we computed in (a). For these three points, we cannot have apolynomial of less than 2 degree fitting all of them. Lagrange interpolation must construct the leastdegree (n = 2) polynomial passing the three points, which must be unique.

(d) Sketch Pn(x) and each f (xi)Li(x) on the 2D plane.

Solution:

EECS 16B, Spring 2018, Homework 9 4

(e) Show that the Lagrange interpolating polynomial must pass through all given points. In other words,show that Pn(xi) = f (xi) for all xi. Do this in general, not just for the example above.

Solution:

For each xi, the corresponding Li(x) is Li(x) =j=n−1

∏j=0, j 6=i

(x− x j)

(xi− x j). Plugging in xi, we get

Li(xi) =j=n−1

∏j=0; j 6=i

(xi− x j)

(xi− x j)= 1.

For any other Lm(x), where m 6= i, plug in xi, Lm(xi) =j=n−1

∏j=0, j 6=m

(xi− x j)

(xm− x j). Because m 6= i, Lm(xi) must

have this term (xi−xi)(xm−xi)

= 0. Hence all Lm(xi) must be zero.

Therefore, Pn(xi) = ∑k=n−1k=0 f (xk)Lk(xi) = f (xi) ·1 = f (xi).

3. The Vector Space of Polynomials

A polynomial of degree at most n on a single variable can be written as

p(x) = p0 + p1x+ p2x2 + · · · pnxn,

where we assume that the coefficients p0, p1, . . . , pn are real. Let Pn be the vector space of all polynomialsof degree at most n.

(a) Consider the representation of p ∈ Pn as the vector of its coefficients in Rn+1.

~p =[

p0 p1 · · · pn

]T

Show that the set Bn = {1,x,x2, . . . ,xn} forms a basis of Pn, by showing the following.

• Every element of Pn can be expressed as a linear combination of elements in Bn.• No element in Bn can be expressed as a linear combination of the other elements of Bn.

(Hint: Use the aspect of the fundamental theorem of algebra which says that a nonzero polynomialof degree n has at most n roots and use a proof by contradiction.)

Solution:We can write every polynomial of degree n on x as p(x) = p0 + p1x + · · · pnxn, which is a linearcombination of the elements in Bn.Let xi ∈Bn, for 0 ≤ i ≤ n. We show that xi cannot be written as a linear combination of the otherelements of Bn. Suppose that there exist real constants c j for j 6= i, such that xi = ∑ j 6=i c jx j. This givesus an equation

∑0≤ j≤n, j 6=i

c jx j− xi = 0 .

The left hand side is a nonzero polynomial of degree at most n, which we know has at most n rootsby the relevant aspect of the fundamental theorem of algebra. Thus the equality cannot hold for all x,which is a contradiction.

EECS 16B, Spring 2018, Homework 9 5

(b) Suppose that the coefficients p0, . . . , pn of p are unknown. To determine the coefficients, we evaluatep on n+1 points, x0, . . . ,xn. Suppose that p(xi) = yi for 0≤ i≤ n. Write the matrix V in terms of thexi, such that

V

p0p1...

pn

=

y0y1...

yn

.

Solution:We can write each equation p(xi) = yi as

p0 + p1xi + p2x2i + · · ·+ pnxn

i =[1 xi x2

i · · · xni

]

p0p1...

pn

= yi .

If we arrange these n+1 equations into a matrix, we get1 x0 x2

0 · · · xn0

1 x1 x21 · · · xn

1...

......

......

1 xn x2n · · · xn

n

p0p1...

pn

=

y0y1...

yn

.

The matrix V above with rows[1 xi x2

i · · · xni

]is also known as a Vandermonde matrix.

(c) For the case where n = 2, compute the determinant of V and show that it is equal to

det(V ) = ∏0≤i< j≤n

(x j− xi) .

Under what conditions is this matrix invertible?

Solution:If n = 2, we can write V as

V =

1 x0 x20

1 x1 x21

1 x2 x22

We can compute the determinant of V by using row reduction. This gives us the matrix1 x0 x2

00 x1− x0 x2

1− x20

0 0 x22− x1x2− x0x2 + x0x1

Since this is upper-triangular, we can compute the determinant by taking the product of the diagonalentries.

det(V ) = x1x22− x2

1x2− x0x22 + x2

0x2 + x0x21− x2

0x1 .

We can check that this is equal to

(x2− x1)(x2− x0)(x1− x0) = x1x22− x2

1x2− x0x22 + x2

0x2 + x0x21− x2

0x1

Note that det(V ) is nonzero, or V is invertible, if and only if all x0,x1,x2 are distinct. If this is the case,then there is a unique solution for p0, p1, p2.

EECS 16B, Spring 2018, Homework 9 6

(d) We can define an inner product on Pn by setting

〈p,q〉=∫ 1

−1p(x)q(x)dx .

Show that this satisfies the following properties of a real inner product. (We would need a complexconjugate if we wanted a complex inner product).

• 〈p, p〉 ≥ 0, with equality if and only if p = 0.• For all a ∈ R, 〈ap,q〉= a〈p,q〉.• 〈p,q〉= 〈q, p〉.

Solution:• 〈p, p〉=

∫ 1−1 p2(x)dx≥ 0 since p2(x)≥ 0 for all x. The integral is 0 if and only if p = 0 for all x,

or p is the zero polynomial.• 〈ap,q〉=

∫ 1−1(ap(x))(q(x))dx = a

∫ 1−1 p(x)q(x)dx = a〈p,q〉.

• 〈p,q〉=∫ 1−1 p(x)q(x)dx =

∫ 1−1 q(x)p(x)dx = 〈q, p〉.

(e) OPTIONAL: An alternative inner-product could be placed upon real polynomials if we simply repre-sented them by a sequence of their evaluations at 0,1, . . . ,n and adopted the standard Euclidean innerproduct on sequences of real numbers. Can you give an example of an orthonormal basis with thisalternative inner product?

Solution:For p,q ∈ Pn, this alternative inner product is given by

〈p,q〉=n

∑i=0

p(i)q(i) .

An example of an orthonormal basis with this inner product would be the Lagrange polynomials Li(x)constructed on the points 0,1, . . . ,n. Since Li(i) = 1 and Li( j) = 0 for j 6= i, 0≤ i, j ≤ n, thus for i 6= jwe clearly have 〈Li,L j〉= 0 and we can also check that 〈Li,Li〉= 1.

4. Sinc FunctionsThe sinc function is defined as

sinc(x) =

{sin(πx)

πx , x 6= 01, x = 0

.

(a) Verify that

limx→0

sin(πx)πx

= 1.

Hint. Look into L’Hospital’s Rule.

Solution:

limx→0

sin(πx)πx

= limx→0

ddx sin(πx)

ddx πx

= limx→0

π cos(πx)π

= 1 = sinc(0)

EECS 16B, Spring 2018, Homework 9 7

(b) Verify that1π

∫π

0cos(ωx)dω = sinc(x)

Solution:

1π

∫π

0cos(ωx)dω =

1π

[sin(ωx)

x

]π

ω=0

=sin(πx)

πx

(c) If we think of the above integral as a sum of oscillatory functions of different frequencies, the sincfunction has continuous frequency components in the range [0,π]. What is the range of frequencies ofthe function

f (x) = sinc(

xT

)?

Hint: Try to use substitution and the above integral.

Solution:We have,

1π

∫π

0cos(

ωxT

)dω

Let λ = ω

T . Then,1π

T

∫ π

T

0cos(λx)dλ

Since we’re summing over a continuum of frequencies from 0 to π

T , this tells us that the range offrequencies is in

[0, π

T

].

(d) Continuing the idea of integration as the continuous version of sums, we define the continuous innerproduct between two real functions as

〈p,q〉=∫

∞

−∞

p(x)q(x)dx.

Given that the length squared of a vector~v of length n is

‖~v‖2 = 〈~v,~v〉=n

∑i=1

vivi,

what is the analogous length squared of a function, f , using the inner product definition above? Thisis a one-line answer.

Solution:

‖ f‖2 = 〈 f , f 〉=∫

∞

−∞

| f (x)|2dx

We only consider interpolating functions where ‖ f‖ < ∞, so the inner product is well defined. It turns outthat the set of shifted sinc functions are orthonormal with the above inner product! Let

φk(x) = sinc(x− k) , where k is an integer.

EECS 16B, Spring 2018, Homework 9 8

Then,

‖φk‖= 1 and 〈φm,φn〉=

{1, if m = n0, otherwise

5. LTI Inputs

We have an LTI system whose exact characteristics we do not know. However, we know that it has a finiteimpulse response that is not longer than 5 samples. We also observed two sequences, x1 and x2, pass throughthe system and observed the system’s responses y1 and y2.

x1 0 2 2 0 −1 0 0 0 0 0y1 0 4 10 8 −2 −3 1 1 −1 0

x2 −2 0 1 0 0 0 0 0 0 0y2 −4 −6 0 5 −1 −1 1 0 0 0

(a) Given the above sequences, what would be the output for the input?

x3 0 0 −2 0 1 0 0 0 0 0

Solution:This is a shifted version of x2. Since the system is time-invariant, the output is:

y3 0 0 −4 −6 0 5 −1 −1 1 0

(b) What is the output of the system for the input x1− x2?

Solution:Since the system is linear, the output is y1− y2:

EECS 16B, Spring 2018, Homework 9 9

y1− y2 4 10 10 3 −1 −2 0 1 −1 0

(c) Given the above information, how could you find the impulse response of this system? What is theimpulse response?

Solution:Since 1

2(x1 + x3) is an impulse at the second sample, 12(y1 + y3) is the system’s impulse response

starting from the second sample.

y1 + y3 0 4 6 2 −2 2 0 0 0 0

so

n n < 0 0 1 2 3 4 n > 4h[n] 0 2 3 1 −1 1 0

(d) What is the output of this system for the following input:

x4 1 1 1 0 0 0 0 0 0 0

Solution:

y4 2 5 6 3 1 0 1 0 0 0

6. LTI Low Pass Filters

Given a sequence of discrete samples with high frequency noise, we can de-noise our signal with a discretelow-pass filter. Two examples are given below:

y[n] = 0.5y[n−1]+ x[n] (1)

y[n] = 0.25x[n]+0.25x[n−1]+0.25x[n−2]+0.25x[n−3] (2)

(a) Write the impulse responses h[n] for (1) and (2). Are they IIR or FIR?

Solution:For (1),

h[n] =

{0.5n, n≥ 00, n < 0

This has an infinite impulse response (IIR).For (2),

h[n] =

{0.25, n = 0,1,2,30, otherwise

This has a finite impulse response (FIR).

(b) Are either of these filters causal? Are either of these filters stable?

Solution:Yes, y[n] depends only on inputs from x[n] and before.Yes, both filters are stable because the impulse responses are absolutely summable.

EECS 16B, Spring 2018, Homework 9 10

(c) Find the output y for each filter given the input sequence x[n] = 2cos(πn)+ n from n = 0 to n = 7.Assume that y[n] = 0 for n < 0.

Solution:For (1),

y = [2,0,4,3,7.5,6.75,11.375,10.6875]

For (2),y = [0.5,0.25,1.25,1.5,2.5,3.5,4.5,5.5]

7. (OPTIONAL) Polynomial Interpolation and RegressionVariometers indicate the rate of climb or descent (vertical velocity) in an aerial vehicle. I have a smallvariometer sensor that I would like to use on a quadrotor for altitude control. In order to evaluate thesensor’s performance, I launch it on a small model rocket and record its measurements. The recorded valuesare:

Time (s) Velocity ( ms )

0 3.750.5 −1.51.5 −8.5

(a) Perform polynomial interpolation on these points. Show your work (you can use a computer to calcu-late the matrix inverse).

Solution:Since there are three points, we can find a unique second-order polynomial that will pass through allof the points: 1 x1 x2

11 x2 x2

21 x3 x2

3

a0

a1a2

=

y1y2y3

Plugging in the values from the table, we have:1 0 0

1 0.5 0.251 1.5 2.25

a0

a1a2

=

3.75−1.5−8.5

We can solve for the coefficients ai by inverting the Vandermonde matrix:a0

a1a2

=

1 0 0−8

3 3 −13

43 −2 2

3

15

4−3

2−17

2

=

154−35

373

The final polynomial is

v(t) =73

t2− 353

t +154

EECS 16B, Spring 2018, Homework 9 11

(b) The rocket is in freefall, so I expect that it will approximately follow a parabolic trajectory (its flightwill not be exactly parabolic due to wind resistance). The velocity profile of a parabolic trajectory islinear. Perform polynomial regression using a first-order polynomial.

Solution:The final polynomial is v(t) =−8t + 15

4 .

Contributors:

• John Maidens.

• Siddharth Iyer.

• Justin Yim.

EECS 16B, Spring 2018, Homework 9 12