EE263 homework 4 solutions - Stanford...

28
EE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra and least-squares. Most of the linear algebra you have seen is unchanged when the scalars, matrices, and vectors are complex, i.e., have complex entries. For example, we say a set of complex vectors {v 1 ,...,v n } is dependent if there exist complex scalars α 1 ,...,α n , not all zero, such that α 1 v 1 +··· +α n v n = 0. There are some slight differences when it comes to the inner product and other expressions that, in the real case, involve the transpose operator. For complex matrices (or vectors) we define the Hermitian conjugate as the complex conjugate of the transpose. We denote this as A , which is equal to ( A) T . Thus, the ij entry of the matrix A is given by (A ji ). The Hermitian conjugate of a matrix is sometimes called its conjugate transpose (which is a nice, explanatory name). Note that for a real matrix or vector, the Hermitian conjugate is the same as the transpose. We define the inner product of two complex vectors u, v C n as u, v= u v, which, in general, is a complex number. The norm of a complex vector is defined as u= u, u= |u 1 | 2 + ··· + |u n | 2 1/2 . Note that these two expressions agree with the definitions you already know when the vectors are real. The complex least-squares problem is to find the x C n that minimizes Ax y2 , where A C m×n and y C m are given. Assuming A is full rank and skinny, the solution is x ls = A y, where A is the (complex) pseudo-inverse of A, given by A =(A A) 1 A . (Which also reduces to the pseudo-inverse you’ve already seen when A is real). There are two general approaches to dealing with complex linear algebra problems. In the first, you simply generalize all the results to work for complex matrices, vectors, and scalars. Another approach is to represent complex matrices and vectors using real matrices and vectors of twice the dimensions, and then you apply what you already know about real linear algebra. We’ll explore that idea in this problem. We associate with a complex vector u C n a real vector ˜ u R 2n , given by ˜ u = u u . We associate with a complex matrix A C m×n the real matrix ˜ A R 2m×2n given by ˜ A = A −ℑA A A . 1

Transcript of EE263 homework 4 solutions - Stanford...

Page 1: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

EE263 Prof. S. Boyd

EE263 homework 4 solutions

5.2 Complex linear algebra and least-squares. Most of the linear algebra you have seenis unchanged when the scalars, matrices, and vectors are complex, i.e., have complexentries. For example, we say a set of complex vectors {v1, . . . , vn} is dependent if thereexist complex scalars α1, . . . , αn, not all zero, such that α1v1+· · ·+αnvn = 0. There aresome slight differences when it comes to the inner product and other expressions that,in the real case, involve the transpose operator. For complex matrices (or vectors)we define the Hermitian conjugate as the complex conjugate of the transpose. Wedenote this as A∗, which is equal to (A)T . Thus, the ij entry of the matrix A∗ isgiven by (Aji). The Hermitian conjugate of a matrix is sometimes called its conjugatetranspose (which is a nice, explanatory name). Note that for a real matrix or vector,the Hermitian conjugate is the same as the transpose. We define the inner product oftwo complex vectors u, v ∈ C

n as

〈u, v〉 = u∗v,

which, in general, is a complex number. The norm of a complex vector is defined as

‖u‖ =√

〈u, u〉 =(

|u1|2 + · · · + |un|2)1/2

.

Note that these two expressions agree with the definitions you already know whenthe vectors are real. The complex least-squares problem is to find the x ∈ C

n thatminimizes ‖Ax−y‖2, where A ∈ C

m×n and y ∈ Cm are given. Assuming A is full rank

and skinny, the solution is xls = A†y, where A† is the (complex) pseudo-inverse of A,given by

A† = (A∗A)−1 A∗.

(Which also reduces to the pseudo-inverse you’ve already seen when A is real). Thereare two general approaches to dealing with complex linear algebra problems. In thefirst, you simply generalize all the results to work for complex matrices, vectors, andscalars. Another approach is to represent complex matrices and vectors using realmatrices and vectors of twice the dimensions, and then you apply what you alreadyknow about real linear algebra. We’ll explore that idea in this problem. We associatewith a complex vector u ∈ C

n a real vector u ∈ R2n, given by

u =

[

ℜuℑu

]

.

We associate with a complex matrix A ∈ Cm×n the real matrix A ∈ R

2m×2n given by

A =

[

ℜA −ℑAℑA ℜA

]

.

1

Page 2: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

(a) What is the relation between 〈u, v〉 and 〈u, v〉? Note that the first inner productinvolves complex vectors and the second involves real vectors.

(b) What is the relation between ‖u‖ and ‖u‖?(c) What is the relation between Au (complex matrix-vector multiplication) and Au

(real matrix-vector multiplication)?

(d) What is the relation between AT and A∗?

(e) Using the results above, verify that A†y solves the complex least-squares problemof minimizing ‖Ax− y‖ (where A, x, y are complex). Express A†y in terms of thereal and imaginary parts of A and y. (You don’t need to simplify your expression;you can leave block matrices in it.)

Solution:

(a) We have

〈u, v〉 = u∗v

= (ℜu − jℑu)T (ℜv + jℑv)

= ℜuTℜv + ℑuTℑv + j(

−ℑuTℜv + ℜuTℑv)

〈u, v〉 =[

ℜuT ℑuT][

ℜvℑv

]

= ℜuTℜv + ℑuTℑv

We thus have ℜ〈u, v〉 = 〈u, v〉.(b) We have

‖u‖2 = u∗u = (ℜu − jℑu)T (ℜu − jℑu) = ℜuTℜu + ℑuTℑu = ‖u‖2.

(c) We have

Au = (ℜA + kℑA)(ℜu + jℑu)

= (ℜAℜu −ℑAℑu) + j(ℜAℑu + ℑuℜu).

Now let’s see what Au works out to be:

Au =

[

ℜA −ℑAℑA ℜA

] [

ℜuℑu

]

=

[

ℜAℜu −ℑAℑuℜAℑu + ℑuℜu

]

.

Thus, the real 2n vector Au corresponds to the complex n vector Au.

(d) The real 2n× 2n matrix AT corresponds exactly to the complex n×n matrix A∗.

2

Page 3: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

(e) We can express the complex least squares problem

minimize ‖Ax − y‖in terms of the double size, real problem

minimize ‖Ax − y‖,since norms and matrix multiplication are preserved in this correspondence. There-fore the solution is

x = (AT A)−1AT y.

This gives the solution in terms of the real and imaginary parts; we just have towork out what it means using complex vectors. First of all, AT y corresponds toA∗y. The 2n×2n real matrix AT A corresponds to the n×n complex matrix A∗A.Although we didn’t verify it above (but we should have . . . I’ll add it to the problemfor future years) inverses correspond as well. To see this, we first show that matrixproducts correspond. In other words, if A and B are complex matrices, then thecomplex matrix AB corresponds to the real matrix AB. To show that inversescorrespond, we apply this result to the matrix equation AB = I, where A andB are complex square matrices (so, of course, B is the inverse of A). By theabove argument, we have AB = I, so B is the inverse of A. Now we can putit all together. The correspondence between complex matrices, and the (double-sized) real matrices preserves matrix product, sums, norm, Hermitian conjugatebecomes transpose, inverses are preserved, etc. To minimize ‖Ax − y‖ we canminimize ‖Ax − y‖. That’s a real least-squares problem, with solution

x = (AT A)−1AT y.

This correponds, according to our discussion above, to the complex equation

x = (A∗A)−1A∗y,

which is exactly x = A†y, as claimed.

Note: In order to solve a complex linear algebra problem, it is usually not necessaryto convert it into a real one in the way shown in this problem. Many numericallinear algebra codes and software directly handle complex matrices. For example,Matlab handles complex matrices and vectors automatically; you don’t need to do theconversion described in the problem. Indeed, in Matlab pinv(A) gives the pseudo-inverse (A†) of a (skinny, full rank) complex matrix. In Matlab you can solve thecomplex least-squares problem of minimizing ‖Ax − y‖ over complex x using the fourASCII characters x=A\y.

6.2 Optimal control of unit mass. In this problem you will use Matlab to solve an optimalcontrol problem for a force acting on a unit mass. Consider a unit mass at positionp(t) with velocity p(t), subjected to force f(t), where f(t) = xi for i − 1 < t ≤ i, fori = 1, . . . , 10.

3

Page 4: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

(a) Assume the mass has zero initial position and velocity: p(0) = p(0) = 0. Find xthat minimizes ∫ 10

t=0f(t)2 dt

subject to the following specifications: p(10) = 1, p(10) = 0, and p(5) = 0. Plotthe optimal force f , and the resulting p and p. Make sure the specifications aresatisfied. Give a short intuitive explanation for what you see.

(b) Assume the mass has initial position p(0) = 0 and velocity p(0) = 1. Our goal isto bring the mass near or to the origin at t = 10, at or near rest, i.e., we want

J1 = p(10)2 + p(10)2,

small, while keeping

J2 =∫ 10

t=0f(t)2 dt

small, or at least not too large. Plot the optimal trade-off curve between J2 andJ1. Check that the end points make sense to you. Hint: the parameter µ hasto cover a very large range, so it usually works better in practice to give it alogarithmic spacing, using, e.g., logspace in Matlab. You don’t need more than50 or so points on the trade-off curve.

Your solution to this problem should consist of a clear written narrative that explainswhat you are doing, and gives formulas symbolically; the Matlab source code youdevise to find the numerical answers, along with comments explaining it all; and thefinal plots produced by Matlab. Solution:

(a) First note that∫ 100 f(t)2dt is nothing but ‖x‖2 because

∫ 10

0f(t)2dt =

∫ 1

0x2

1 dt +∫ 2

1x2

2 dt + · · · +∫ 10

9x2

10 dt

= x21 + x2

2 + · · · + x210

= ‖x‖2.

The constraints p(10) = 1, p(10) = 0, and p(5) = 0 can be expressed as linearconstraints in the force program x. From homework 1 we know that

p(10)p(10)p(5)

=

100

=

9.5 8.5 7.5 6.5 5.5 4.5 3.5 2.5 1.5 0.51 1 1 1 1 1 1 1 1 1

4.5 3.5 2.5 1.5 0.5 0 0 0 0 0

x. (1)

Therefore the optimal x is given by the minimizer of ‖x‖ subject to (1). In otherwords, the optimal x is the minimum norm solution x = xln of (1). x can becalculated using Matlab as follows:

4

Page 5: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

>> a1=linspace(9.5,0.5,10);

>> a2=ones(1,10);

>> a3=[linspace(4.5,0.5,5), 0 0 0 0 0];

>> A=[a1;a2;a3]

A =

Columns 1 through 7

9.5000 8.5000 7.5000 6.5000 5.5000 4.5000 3.5000

1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000

4.5000 3.5000 2.5000 1.5000 0.5000 0 0

Columns 8 through 10

2.5000 1.5000 0.5000

1.0000 1.0000 1.0000

0 0 0

>> y=[1;0;0]

y =

1

0

0

>> x=pinv(A)*y

x =

-0.0455

-0.0076

0.0303

0.0682

0.1061

0.0939

0.0318

-0.0303

-0.0924

-0.1545

>> A*x

ans =

1.0000

0.0000

-0.0000

>> norm(x)

ans =

0.2492

>>

Note that between times t = i and t = i + 1 for i = 0, . . . , 9 the force f ispiecewise constant, the velocity p is piecewise linear, and the position p is piecewisequadratic. The following Matlab commands plot f , p and p.

5

Page 6: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

>> figure

>> subplot(3,1,1)

>> stairs([x;0])

>> grid on

>> xlabel(’t’)

>> ylabel(’f(t)’)

>> T1=toeplitz([ones(10,1)],[1,zeros(1,9)])

T1 =

1 0 0 0 0 0 0 0 0 0

1 1 0 0 0 0 0 0 0 0

1 1 1 0 0 0 0 0 0 0

1 1 1 1 0 0 0 0 0 0

1 1 1 1 1 0 0 0 0 0

1 1 1 1 1 1 0 0 0 0

1 1 1 1 1 1 1 0 0 0

1 1 1 1 1 1 1 1 0 0

1 1 1 1 1 1 1 1 1 0

1 1 1 1 1 1 1 1 1 1

>> p_dot=T1*x

p_dot =

-0.0455

-0.0530

-0.0227

0.0455

0.1515

0.2455

0.2773

0.2470

0.1545

0.0000

>> subplot(3,1,2)

>> plot(linspace(0,10,11),[0;p_dot])

>> grid on

>> xlabel(’t’)

>> ylabel(’p_dot(t)’)

>> T2=toeplitz(linspace(0.5,9.5,10)’,[0.5,zeros(1,9)])

T2 =

Columns 1 through 7

0.5000 0 0 0 0 0 0

1.5000 0.5000 0 0 0 0 0

2.5000 1.5000 0.5000 0 0 0 0

3.5000 2.5000 1.5000 0.5000 0 0 0

6

Page 7: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

4.5000 3.5000 2.5000 1.5000 0.5000 0 0

5.5000 4.5000 3.5000 2.5000 1.5000 0.5000 0

6.5000 5.5000 4.5000 3.5000 2.5000 1.5000 0.5000

7.5000 6.5000 5.5000 4.5000 3.5000 2.5000 1.5000

8.5000 7.5000 6.5000 5.5000 4.5000 3.5000 2.5000

9.5000 8.5000 7.5000 6.5000 5.5000 4.5000 3.5000

Columns 8 through 10

0 0 0

0 0 0

0 0 0

0 0 0

0 0 0

0 0 0

0 0 0

0.5000 0 0

1.5000 0.5000 0

2.5000 1.5000 0.5000

>> p=T2*x

p =

-0.0227

-0.0720

-0.1098

-0.0985

-0.0000

0.1985

0.4598

0.7220

0.9227

1.0000

>> subplot(3,1,3)

>> plot(linspace(0,10,11),[0;p])

>> grid on

>> xlabel(’t’)

>> ylabel(’p(t)’)

7

Page 8: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

0 1 2 3 4 5 6 7 8 9 10−0.2

0

0.2

0 1 2 3 4 5 6 7 8 9 10−0.5

0

0.5

1

0 1 2 3 4 5 6 7 8 9 10

0

t

t

t

p(t)

p(t)

f(t

)

Note that the plot for p(t) is not quite right because the plot command in Matlabplots p(t) as being piecewise linear and not piecewise quadratic. It is interestingthat the optimal force is such that p(t) < 0 for 0 < t < 5 which means that themass doesn’t stay at position zero for 0 < t < 5. It backups a little bit so thesecond time it reaches position zero it has positive velocity.

(b) In this case there are two competing objectives that we want to keep small. J2 =‖x‖2 as it was shown above. To express J1 we rewrite the motion equations, thistime for p(0) = 0 and p(0) = 1. It is easy to verify that:

[

p(10)p(10)

]

=

[

00

]

=

[

9.5 8.5 7.5 6.5 5.5 4.5 3.5 2.5 1.5 0.51 1 1 1 1 1 1 1 1 1

]

x+

[

101

]

.

(2)

Therefore, to minimize J1 we have to minimize ‖Ax − y‖2, where y =

[

−10−1

]

.

So, all we need is to calculate the minimizer xµ of the expression J1 + µJ2 =‖Ax− y‖2 +µ‖x‖2 for different values of µ. As it was proven in the lecture notes,

xµ = (AT A + µI)−1AT y.

We use Matlab to calculate xµ and the resulting pairs (J1, J2) and plot the tradeoffcurve. A sample Matlab script is given below:

clear; clf;

N = 50;

a1 = linspace(9.5,0.5,10); a2 = ones(1,10); A = [a1; a2];

y = [-10; -1];

F = eye(10);

8

Page 9: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

mu = logspace(-2, 4, N);

for i=1:N;

f = inv(A’*A + mu(i)*F’*F)*A’*y;

J_1(i) = (norm(A*f - y))^2;

J_2(i) = (norm(f))^2;

end;

plot(J_2,J_1); title(’Optimal Tradeoff Curve’); xlabel(’J_2 =

||x||^2’) ylabel(’J_1 = p(10)^2 + p_dot(10)^2’);

The resulting tradeoff curve is shown below:

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.450

10

20

30

40

50

60

70

80

90

100

J2

J1

Optimal Tradeoff Curve

We can now use the curve to decide which xµ we are going to use. The choicewill depend on the importance we decide to attach to each of the two objectives.A final comment: Note that there is a value of xµ after which J1 becomes equalto 0. This value is the minimum-norm solution, because it corresponds to theminimum J2 = ‖x‖2 for which Ax = y. On the other hand, we cannot drive thesystem to or near the desired state without spending some energy, so J1 is large

for small values of J2. For J2 = 0, J1 is equal to

∥∥∥∥∥

[

101

]∥∥∥∥∥

2

= 101.

6.5 Interpolation with rational functions. In this problem we consider a function f : R → R

of the form

f(x) =a0 + a1x + · · · + amxm

1 + b1x + · · · + bmxm,

where a0, . . . , am, and b1, . . . , bm are parameters, with either am 6= 0 or bm 6= 0. Such afunction is called a rational function of degree m. We are given data points x1, . . . , xN ∈R and y1, . . . , yN ∈ R, where yi = f(xi). The problem is to find a rational function

9

Page 10: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

of smallest degree that is consistent with this data. In other words, you are to findm, which should be as small as possible, and a0, . . . , am, b1, . . . , bm, which satisfyf(xi) = yi. Explain how you will solve this problem, and then carry out your methodon the problem data given in ri_data.m. (This contains two vectors, x and y, that givethe values x1, . . . , xN , and y1, . . . , yN , respectively.) Give the value of m you find, andthe coefficients a0, . . . , am, b1, . . . , bm. Please show us your verification that yi = f(xi)holds (possibly with some small numerical errors). Solution. The interpolationcondition f(xi) = yi is

f(xi) =a0 + a1xi + · · · + amxm

i

1 + b1xi + · · · + bmxmi

= yi, i = 1, . . . , N.

This is a set of complicated nonlinear functions of the coefficient vectors a and b. Ifwe multiply out by the denominator, we get

yi(1 + b1xi + · · · + bmxmi ) − (a0 + a1xi + · · · + amxm

i ) = 0, i = 1, . . . , N.

These equations are linear in a and b. We can write these equations in matrix form as

G

[

ab

]

= y, (3)

where

a =

a0

a1...

am

, b =

b1

b2...

bm

,

and

G =

1 x1 · · · xm1 −y1x1 −y1x

21 · · · −y1x

m1

1 x2 · · · xm2 −y2x2 −y2x

22 · · · −y2x

m2

1 x3 · · · xm3 −y3x3 −y3x

23 · · · −y3x

m3

......

......

......

1 xN · · · xmN −yNxN −yNx2

N · · · −yNxmN

.

Thus, we can interpolate the data if and only if the equation (3) has a solution. Ourproblem is to find the smallest m for which these linear equations can be solved, or,equivalently, y ∈ R(G). We can do this by finding the smallest value of m for which

Rank(G) = Rank([G y]).

Then we can find a set of coefficients by solving the equation (3) for a and b. Thefollowing Matlab code carries out this method.

clear all

close all

10

Page 11: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

rat_int_data

for m=1:20 %we sweep over different values of m

G=ones(N,1);

for i=1:m;

G=[G x.^i];

end

for i=1:m

G=[G -x.^i.*y];

end

if rank(G)== rank([G y])

break;

end

end

ab=G\y;

a=ab(1:m+1);

b=ab(m+2:2*m+1);

m

a

b

Matlab produces the following output:

m =

5

a =

0.2742

1.0291

1.2906

-5.8763

-2.6738

6.6845

b =

-1.2513

-6.5107

3.2754

17.3797

6.6845

Thus, we find that m = 5 is the lowest order rational function that interpolates thedata, and a rational function that interpolates the data is given by

f(x) =0.2742 + 1.0291x + 1.2906x2 − 5.8763x3 − 2.6738x4 + 6.6845x5

1 − 1.2513x − 6.5107x2 + 3.2754x3 + 17.3797x4 + 6.6845x5

11

Page 12: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

(we have truncated the coefficients to shorten the formula). We now verify that thisexpression interpolates the given points.

num=zeros(N,1);

for i=1:m+1

num=a(i)*x.^(i-1)+num;

end

den=ones(N,1);

for i=1:m

den=b(i)*x.^i+den;

end

f=num./den;

err=norm(f-y)

Matlab produces the following output

err =

7.7649e-14.

This shows that the output is interpolated up to numerical precision.

6.12 Estimation with sensor offset and drift. We consider the usual estimation setup:

yi = aTi x + vi, i = 1, . . . ,m,

where

• yi is the ith (scalar) measurement

• x ∈ Rn is the vector of parameters we wish to estimate from the measurements

• vi is the sensor or measurement error of the ith measurement

In this problem we assume the measurements yi are taken at times evenly spaced, Tseconds apart, starting at time t = T . Thus, yi, the ith measurement, is taken at timet = iT . (This isn’t really material; it just makes the interpretation simpler.) You canassume that m ≥ n and the measurement matrix

A =

aT1

aT2...

aTm

is full rank (i.e., has rank n). Usually we assume (often implicitly) that the measure-ment errors vi are random, unpredictable, small, and centered around zero. (You don’t

12

Page 13: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

need to worry about how to make this idea precise.) In such cases, least-squares esti-mation of x works well. In some cases, however, the measurement error includes somepredictable terms. For example, each sensor measurement might include a (common)offset or bias, as well as a term that grows linearly with time (called a drift). Wemodel this situation as

vi = α + βiT + wi

where α is the sensor bias (which is unknown but the same for all sensor measurements),β is the drift term (again the same for all measurements), and wi is part of the sensorerror that is unpredictable, small, and centered around 0. If we knew the offset α andthe drift term β we could just subtract the predictable part of the sensor signal, i.e.,α + βiT from the sensor signal. But we’re interested in the case where we don’t knowthe offset α or the drift coefficient β. Show how to use least-squares to simultaneouslyestimate the parameter vector x ∈ R

n, the offset α ∈ R, and the drift coefficientβ ∈ R. Clearly explain your method. If your method always works, say so. Otherwisedescribe the conditions (on the matrix A) that must hold for your method to work,and give a simple example where the conditions don’t hold. Solution:Substituting the expression for the noise into the measurement equation gives

yi = aTi x + α + βiT + wi i = 1, . . . ,m.

In matrix form we can write this as

y1

y2...

ym

︸ ︷︷ ︸

y

=

aT1 1 T

aT2 1 2T...

......

aTm 1 mT

︸ ︷︷ ︸

A

xαβ

︸ ︷︷ ︸

x

.

If A is skinny (m ≥ n + 2) and full-rank, least-squares can be used to estimate x, α,and β. In that case,

xls =

xls

αls

βls

= (AT A)−1AT y.

The requirement that A be skinny (or at least, square) makes perfect sense: you can’textract n + 2 parameters (i.e., x, α, β) from fewer than n + 2 measurements. Even isA is skinny and full-rank, A may not be. For example, with

A =

2 02 12 02 1

,

13

Page 14: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

we have

A =

2 0 1 T2 1 1 2T2 0 1 3T2 1 1 4T

,

which is not full-rank. In this example we can understand exactly what happened.The first column of A, which tells us how the sensors respond to the first componentof x, has exactly the same form as an offset, so it is not possible to separate the offsetfrom the signal induced by x1. In the general case, A is not full rank only if somelinear combinations of the sensor signals looks exactly like an offset or drift, or somelinear combination of offset and drift. Some people asserted that A is full rank if thevector of ones and the vector (T, 2T, . . . ,mT ) are not in the span of the columns ofA. This is false. Several people went into the case when A is not full rank in greatdetail, suggesting regularization and other things. We weren’t really expecting you togo into such detail about this case. Most people who went down this route, however,failed to mention the most important thing about what happens. When A is not fullrank, you cannot separate the offset and drift parameters from the parameter x by anymeans at all. Regularization means that the numerics will work, but the result will bequite meaningless. Some people pointed out that the normal eqautions always have asolution, even when A is not full rank. Again, this is true, but the most important thinghere is that even if you solve the normal equations, the results are meaningless, sinceyou cannot separate out the offset and drift terms from the parameter x. Accountingfor known noise characteristics (like offset and drift) can greatly improve estimationaccuracy. The following matlab code shows an example.

T = 60; % Time between samples

alpha = 3; % sensor offset

beta = .05; % sensor drift constant

num_x = 3;

num_y = 8;

A =[ 1 4 0

2 0 1

-2 -2 3

-1 1 -4

-3 1 1

0 -2 2

3 2 3

0 -4 -6 ]; % matrix whose rows are a_i^T

x = [-8; 20; 5];

, with v a (Gaussian, random) noise

y = A*x;

for i = 1:num_y;

y(i) = y(i)+alpha+beta*T*i+randn;

14

Page 15: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

end;

x_ls = A\y;

for i = 1:num_y

last_col(i) = T*i;

end

A = [A ones(num_y,1) last_col’];

x_ls_with_noise_model = A\y;

norm(x-x_ls)

norm(x-x_ls_with_noise_model(1:3))

Additional comments. Many people correctly stated that A needed to be full rank andthen presented a condition they claimed was equivalent. Unfortunately, many of thesestatements were incorrect. The most common error was to claim that if neither of thetwo column vectors that were appended to A in creating A was in R(A), then A wasfull rank. As a counter-example, take

A =

23...

m + 1

,

and

A =

2 1 13 1 2...

......

m + 1 1 m

.

Since the first column of A is the sum of the last two, A has rank 2, not 3.

6.14 Identifying a system from input/output data. We consider the standard setup:

y = Ax + v,

where A ∈ Rm×n, x ∈ R

n is the input vector, y ∈ Rm is the output vector, and v ∈ R

m

is the noise or disturance. We consider here the problem of estimating the matrix A,given some input/output data. Specifically, we are given the following:

x(1), . . . , x(N) ∈ Rn, y(1), . . . , y(N) ∈ R

m.

These represent N samples or observations of the input and output, respectively, pos-sibly corrupted by noise. In other words, we have

y(k) = Ax(k) + v(k), k = 1, . . . , N,

where v(k) are assumed to be small. The problem is to estimate the (coefficients of the)matrix A, based on the given input/output data. You will use a least-squares criterion

15

Page 16: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

to form an estimate A of A. Specifically, you will choose as your estimate A the matrixthat minimizes the quantity

J =N∑

k=1

‖Ax(k) − y(k)‖2

over A.

(a) Explain how to do this. If you need to make an assumption about the in-put/output data to make your method work, state it clearly. You may wantto use the matrices X ∈ R

n×N and Y ∈ Rm×N given by

X =[

x(1) · · · x(N)]

, Y =[

y(1) · · · y(N)]

in your solution.

(b) On the course web site you will find some input/output data for an instance ofthis problem in the mfile sysid_data.m. Executing this mfile will assign valuesto m, n, and N , and create two matrices that contain the input and output data,respectively. The n × N matrix variable X contains the input data x(1), . . . , x(N)

(i.e., the first column of X contains x(1), etc.). Similarly, the m × N matrix Y

contains the output data y(1), . . . , y(N). You must give your final estimate A, yoursource code, and also give an explanation of what you did.

Solution.

(a) We start by expressing the objective function J as

J =N∑

k=1

‖Ax(k) − y(k)‖2

=N∑

k=1

m∑

i=1

(Ax(k) − y(k))2i

=N∑

k=1

m∑

i=1

(aTi x(k) − y

(k)i )2

=m∑

i=1

(N∑

k=1

(aTi x(k) − y

(k)i )2

)

,

where ai is the ith row of A. The last expression shows that J is a sum ofexpressions Ji (shown in parentheses), each of which only depends on ai. Thismeans that to minimize J , we can minimize each of these expressions separately.That makes sense: we can estimate the rows of A separately. Now let’s see howto minimize

Ji =N∑

k=1

(aTi x(k) − y

(k)i )2,

16

Page 17: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

which is the contribution to J from the ith row of A. First we write it as

Ji =

∥∥∥∥∥∥∥∥

x(1)T

...x(N)T

ai −

y(1)i...

y(N)i

∥∥∥∥∥∥∥∥

2

.

Now that we have the problem in the standard least-squares format, we’re prettymuch done. Using the matrix X ∈ R

n×N given by

X =[

x(1) · · · x(N)]

,

we can express the estimate as

ai = (XXT )−1X

y(1)i...

y(N)i

.

Using the matrix Y ∈ Rm×N given by

Y =[

y(1) · · · y(N)]

,

we can express the estimate of A as

AT = (XXT )−1XY T .

Transposing this gives the final answer:

A = Y XT (XXT )−1.

(b) Once you have the neat formula found above, it’s easy to get matlab to computethe estimate. It’s a little inefficient, but perfectly correct, to simply use

Ahat = Y*X’*inv(X*X’);

This yields the estimate

A =

2.03 5.02 5.010.01 7 1.017.04 0 6.94

7 3.98 49.01 1.04 74.01 3.96 9.034.99 6.97 8.037.94 6.09 3.020.01 8.97 −0.041.06 8.02 7.03

.

17

Page 18: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

Once you’ve got A, it’s a good idea to check the residuals, just to make sure it’sreasonable, by comparing it to

N∑

k=1

‖y(k)‖2.

Here we get (64.5)2, around 4.08%. There are several other ways to compute Ain matlab. You can calculate the rows of A one at a time, using

a1hat = (X’\(Y(i,:)’))’;

In fact, the backslash operator in matlab solves multiple least-squares problemsat once, so you can use

AhatT = X’ \ (Y’);

Ahat = AhatT’;

or

Ahat = (X’\(Y’))’;

In any case, it’s not exactly a long matlab program . . .

6.26 Estimating direction and amplitude of a light beam. A light beam with (nonnegative)amplitude a comes from a direction d ∈ R

3, where ‖d‖ = 1. (This means the beamtravels in the direction −d.) The beam falls on m ≥ 3 photodetectors, each of whichgenerates a scalar signal that depends on the beam amplitude and direction, and thedirection in which the photodetector is pointed. Specifically, photodetector i generatesan output signal pi, with

pi = aα cos θi + vi,

where θi is the angle between the beam direction d and the outward normal vector qi

of the surface of the ith photodetector, and α is the photodetector sensitivity. You caninterpret qi ∈ R

3, which we assume has norm one, as the direction the ith photodetectoris pointed. We assume that |θi| < 90◦, i.e., the beam illuminates the top of thephotodetectors. The numbers vi are small measurement errors.

You are given the photodetector direction vectors q1, . . . , qm ∈ R3, the photodetector

sensitivity α, and the noisy photodetector outputs, p1, . . . , pm ∈ R. Your job is toestimate the beam direction d ∈ R

3 (which is a unit vector), and a, the beam amplitude.

To describe unit vectors q1, . . . , qm and d in R3 we will use azimuth and elevation,

defined as follows:

q =

cos φ cos θcos φ sin θ

sin φ

.

Here φ is the elevation (which will be between 0◦ and 90◦, since all unit vectors in thisproblem have positive 3rd component, i.e., point upward). The azimuth angle θ, which

18

Page 19: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

varies from 0◦ to 360◦, gives the direction in the plane spanned by the first and secondcoordinates. If q = e3 (i.e., the direction is directly up), the azimuth is undefined.

(a) Explain how to do this, using a method or methods from this class. The simplerthe method the better. If some matrix (or matrices) needs to be full rank for yourmethod to work, say so.

(b) Carry out your method on the data given in beam_estim_data.m. This mfiledefines p, the vector of photodetector outputs, a vector det_az, which gives theazimuth angles of the photodetector directions, and a vector det_el, which givesthe elevation angles of the photodetector directions. Note that both of these aregiven in degrees, not radians. Give your final estimate of the beam amplitude aand beam direction d (in azimuth and elevation, in degrees).

Solution.

(a) Since cos θi = qTi d/(‖qi‖‖d‖) = qT

i d (using ‖qi‖ = ‖d‖ = 1), we have

pi = aαqTi d + vi.

In this equation we are given pi, α, and qi; we are to estimate a ∈ R and d ∈ R3,

using the given information that vi is small. At first glance it looks like a nonlinearproblem, since two of the variables we need to estimate, a and d, are multipliedtogether in this formula.

But a little thought reveals that things are actually much simpler. Let’s definex ∈ R

3 as x = ad. We can just as well work with x since given any nonzerox ∈ R

3, we have a = ‖x‖ and d = x/‖x‖. (Conversely, given any a and d, wehave x = ad by definition.)

We can therefore express the problem in terms of the variable x as

p = α

qT1...

qTm

x + v = αQx + v,

where p = (p1, . . . , pm), v = (v1, . . . , vm), and Q is the matrix with rows qTi .

Now we can get a reasonable guess of x using least-squares. Assuming Q is fullrank, we have the least-squares estimate

x = (1/α)(QT Q)−1QT p.

We then form estimates of a and d using a = ‖x‖, d = x/‖x‖.The matrix Q is full rank (i.e., rank 3), if and only if the vectors {q1, . . . , qm}span R

3. In other words, we cannot have all photodetectors pointing in a commonplane.

19

Page 20: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

(b) The following code solves the problem for the given data.

beam_estim_data

for i=1:m

Q(i,:)=[ cosd(det_el(i))*cosd(det_az(i)),...

cosd(det_el(i))*sind(det_az(i)),...

sind(det_el(i)) ];

end

xhat=(1/alpha)*(Q\p);

ahat=norm(xhat);

dhat=xhat/norm(xhat);

elevation=asind(dhat(3))

azimuth=acosd(dhat(1)/cosd(elevation))

The result is a = 5.0107, φd = 38.7174, and θd = 77.6623.

7.3 Curve-smoothing. We are given a function F : [0, 1] → R (whose graph gives a curve inR

2). Our goal is to find another function G : [0, 1] → R, which is a smoothed versionof F . We’ll judge the smoothed version G of F in two ways:

• Mean-square deviation from F , defined as

D =∫ 1

0(F (t) − G(t))2 dt.

• Mean-square curvature, defined as

C =∫ 1

0G′′(t)2 dt.

We want both D and C to be small, so we have a problem with two objectives. Ingeneral there will be a trade-off between the two objectives. At one extreme, we canchoose G = F , which makes D = 0; at the other extreme, we can choose G to be anaffine function (i.e., to have G′′(t) = 0 for all t ∈ [0, 1]), in which case C = 0. Theproblem is to identify the optimal trade-off curve between C and D, and explain howto find smoothed functions G on the optimal trade-off curve. To reduce the problemto a finite-dimensional one, we will represent the functions F and G (approximately)by vectors f, g ∈ R

n, where

fi = F (i/n), gi = G(i/n).

You can assume that n is chosen large enough to represent the functions well. Usingthis representation we will use the following objectives, which approximate the onesdefined for the functions above:

20

Page 21: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

• Mean-square deviation, defined as

d =1

n

n∑

i=1

(fi − gi)2.

• Mean-square curvature, defined as

c =1

n − 2

n−1∑

i=2

(

gi+1 − 2gi + gi−1

1/n2

)2

.

In our definition of c, note that

gi+1 − 2gi + gi−1

1/n2

gives a simple approximation of G′′(i/n). You will only work with this approximateversion of the problem, i.e., the vectors f and g and the objectives c and d.

(a) Explain how to find g that minimizes d+µc, where µ ≥ 0 is a parameter that givesthe relative weighting of sum-square curvature compared to sum-square deviation.Does your method always work? If there are some assumptions you need to make(say, on rank of some matrix, independence of some vectors, etc.), state themclearly. Explain how to obtain the two extreme cases: µ = 0, which correspondsto minimizing d without regard for c, and also the solution obtained as µ → ∞(i.e., as we put more and more weight on minimizing curvature).

(b) Get the file curve smoothing.m from the course web site. This file defines aspecific vector f that you will use. Find and plot the optimal trade-off curvebetween d and c. Be sure to identify any critical points (such as, for example, anyintersection of the curve with an axis). Plot the optimal g for the two extremecases µ = 0 and µ → ∞, and for three values of µ in between (chosen to show thetrade-off nicely). On your plots of g, be sure to include also a plot of f , say withdotted line type, for reference. Submit your Matlab code.

Solution:

(a) Let’s start with the two extreme cases. When µ = 0, finding g to minimize d+µcreduces to finding g to minimize d. Since d is a sum of squares, d ≥ 0. Choosingg = f trivially achieves d = 0. This makes perfect sense: to minimize thedeviation measure, just take the smoothed version to be the same as the originalfunction. This yields zero deviation, naturally, but also, it yields no smoothing!Next, consider the extreme case where µ → ∞. This means we want to make thecurvature as small as possible. Can we drive it to zero? The answer is yes, wecan: the curvature is zero if and only if g is an affine function, i.e., has the formgi = ai + b for some constants a and b. There are lots of vectors g that have this

21

Page 22: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

form; in fact, we have one for every pair of numbers a, b. All of these vectors gmake c zero. Which one do we choose? Well, even if µ is huge, we still have asmall contribution to d+µc from d, so among all g that make c = 0, we’d like theone that minimizes d. Basically, we want to find the best affine approximation,in the sum of squares sense, to f . We want to find a and b that minimize

∥∥∥∥∥f − A

[

ab

]∥∥∥∥∥

where A =

1 12 13 1...

...n 1

.

For n ≥ 2, A is skinny and full rank, and a and b can be found using least-squares.Specifically, [a b]T = (AT A)−1AT f . In the general case, minimizing d + µc, is thesame as choosing g to minimize

∥∥∥∥∥

1√n

Ig − 1√n

f

∥∥∥∥∥

2

+ µ

∥∥∥∥∥∥∥∥∥∥∥∥∥∥∥

n2

√n − 2

−1 2 −1 0 · · · 00 −1 2 −1 · · · 0

0 0. . . . . . . . .

...0 0 · · · −1 2 −1

︸ ︷︷ ︸

S

g

∥∥∥∥∥∥∥∥∥∥∥∥∥∥∥

2

.

This is a multi-objective least-squares problem. The minimizing g is

g = (AT A)−1AT y where A =

[I√n√µS

]

and y =

[f√n

0

]

.

The inverse of A always always exists because I is full rank. The expression canalso be written as g = ( I

n+ µST S)−1 f

n.

(b) The following plots show the optimal trade-off curve and the optimal g corre-sponding to representative µ values on the curve.

22

Page 23: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

0 0.05 0.1 0.15 0.2 0.25 0.3 0.350

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2x 10

6

d = Sum−square deviation (x intercept = 0.3347)

c =

Sum

−sq

uare

cur

vatu

re (

y in

terc

ept =

1.9

724e

06)

Optimal tradeoff curve

5 10 15 20 25 30 35 40 45 50−3

−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

Curves illustrating the trade−off

f u = 0 u = 10e−7 u = 10e−5 u = 10e−4 u = infinity

The following matlab code finds and plots the optimal trade-off curve between d

23

Page 24: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

and c. It also finds and plots the optimal g for representative values of µ. Asexpected, when µ = 0, g = f and no smoothing occurs. At the other extreme, asµ goes to infinity, we get an affine approximation of f . Intermediate values of µcorrespond to approximations of f with different degrees of smoothness.

close all;

clear all;

curve_smoothing

S = toeplitz([-1; zeros(n-3,1)],[-1 2 -1 zeros(1,n-3)]);

S = S*n^2/(sqrt(n-2));

I = eye(n);

g_no_deviation = f;

error_curvature(1) = norm(S*g_no_deviation)^2;

error_deviation(1) = 0;

u = logspace(-8,-3,30);

for i = 1:length(u)

A_tilde = [1/sqrt(n)*I; sqrt(u(i))*S];

y_tilde = [1/sqrt(n)*f; zeros(n-2,1)];

g = A_tilde\y_tilde;

error_deviation(i+1) = norm(1/sqrt(n)*I*g-f/sqrt(n))^2;

error_curvature(i+1) = norm(S*g)^2;

end

a1 = 1:n;

a1 = a1’;

a2 = ones(n,1);

A = [a1 a2];

affine_param = inv(A’*A)*A’*f;

for i = 1:n

g_no_curvature(i) = affine_param(1)*i+affine_param(2);

end

g_no_curvature = g_no_curvature’;

error_deviation(length(u)+2) = 1/n*norm(g_no_curvature-f)^2;

error_curvature(length(u)+2) = 0;

figure(1);

plot(error_deviation, error_curvature);

xlabel(’Sum-square deviation (y intercept = 0.3347)’);

ylabel(’Sum-square curvature (x intercept = 1.9724e06)’);

title(’Optimal tradeoff curve ’);

print curve_extreme.eps;

u1 = 10e-7;

A_tilde = [1/sqrt(n)*I;sqrt(u1)*S];

y_tilde = [1/sqrt(n)*f;zeros(n-2,1)];

g1 = A_tilde\y_tilde;

24

Page 25: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

u2 = 10e-5;

A_tilde = [1/sqrt(n)*I;sqrt(u2)*S];

y_tilde = [1/sqrt(n)*f;zeros(n-2,1)];

g2 = A_tilde\y_tilde;

u3 = 10e-4;

A_tilde = [1/sqrt(n)*I;sqrt(u3)*S];

y_tilde = [1/sqrt(n)*f;zeros(n-2,1)];

g3 = A_tilde\y_tilde;

figure(3);

plot(f,’*’);

hold;

plot(g_no_deviation);

plot(g1,’--’);

plot(g2,’-.’);

plot(g3,’-’);

plot(g_no_curvature,’:’);

axis tight;

legend(’f’,’u = 0’,’u = 10e-7’, ’u = 10e-5’, ’u = 10e-4’,’u = infinity’,0);

title(’Curves illustrating the trade-off’);

print curve_tradeoff.eps;

Note: Several exams had a typo that defined

c =1

n − 1

n−1∑

i=2

(

gi+1 − 2gi + gi−1

1/n2

)2

instead of

c =1

n − 2

n−1∑

i=2

(

gi+1 − 2gi + gi−1

1/n2

)2

.

The solutions above reflect the second definition. Full credit was given for answersconsistent with either definition. Some common errors

• Several students tried to approximate f using low-degree polynomials. Whilefitting f to a polynomial does smooth f , it does not necessarily minimize d+µc forsome µ ≥ 0, nor does it illustrate the trade-off between curvature and deviation.

• In explaining how to find the g that minimizes d + µc as µ → ∞, many peoplecorrectly observed that if g ∈ N (S), then c = 0. For full credit, however, solutionshad to show how to choose the vector in N (S) that minimizes d.

• Many people chose to zoom in on a small section of the trade-off curve ratherthan plot the whole range from 0 to µ → ∞. Those solutions received full-creditprovided they calculated the intersections with the axes (i.e. provided they foundthe minimum value for d + µc when µ = 0 and when µ → ∞).

25

Page 26: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

8.2 Simultaneous left inverse of two matrices. Consider a system where

y = Gx, y = Gx

where G ∈ Rm×n, G ∈ R

m×n. Here x is some variable we wish to estimate or find, ygives the measurements with some set of (linear) sensors, and y gives the measurementswith some alternate set of (linear) sensors. We want to find a reconstruction matrixH ∈ R

n×m such that HG = HG = I. Such a reconstruction matrix has the niceproperty that it recovers x perfectly from either set of measurements (y or y), i.e.,x = Hy = Hy. Consider the specific case

G =

2 31 00 41 1

−1 2

, G =

−3 −1−1 0

2 −3−1 −3

1 2

.

Either find an explicit reconstruction matrix H, or explain why there is no such H.Solution:The requirements HG = I and HG = I are a set of linear equations in the variablesHij. Since H ∈ R

n×m there are mn unknowns; each equation of the form HG = Igives n2 equations, so all together we have 2n2 equations. Roughly speaking, it’sreasonable to expect a solution to exist when there are more variables than equations,i.e., mn ≥ 2n2, which implies that m ≥ 2n. This condition makes sense: to invert twodifferent sensor measurements we need a redundancy factor of two. Now let’s look atthe specific case given. Suppose that

H =

[

hT1

h22

]

where h1, h2 ∈ R5. HG = I implies that hT

1 G = eT1 and hT

2 G = eT2 where e1 and e2 are

the unit vectors in R2. Similarly we should have hT

1 G = eT1 and hT

2 G = eT2 . In block

matrix form

GT 0

GT 00 GT

0 GT

[

h1

h2

]

=

e1

e1

e2

e2

.

Now by defining

A =

GT 0

GT 00 GT

0 G

, x =

[

h1

h2

]

, b =

e1

e2

e1

e2

we get the standard form Ax = b. If b ∈ R(A) a solution exists and H can be found. Inthis case, A happens to be full rank so (AAT )−1 exists and we can set x = AT (AAT )−1b.Using Matlab:

26

Page 27: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

>> G=[2 3;1 0;0 4;1 1;-1 2]

G =

2 3

1 0

0 4

1 1

-1 2

>> tilde_G=[-3 -1;-1 0;2 -3;-1 -3;1 2]

tilde_G =

-3 -1

-1 0

2 -3

-1 -3

1 2

>> zero=zeros(2,5)

zero =

0 0 0 0 0

0 0 0 0 0

>> A=[G’ zero;tilde_G’ zero;zero G’;zero tilde_G’]

A =

2 1 0 1 -1 0 0 0 0 0

3 0 4 1 2 0 0 0 0 0

-3 -1 2 -1 1 0 0 0 0 0

-1 0 -3 -3 2 0 0 0 0 0

0 0 0 0 0 2 1 0 1 -1

0 0 0 0 0 3 0 4 1 2

0 0 0 0 0 -3 -1 2 -1 1

0 0 0 0 0 -1 0 -3 -3 2

>> b=[1;0;1;0;0;1;0;1]

b =

1

0

1

0

0

1

0

1

>> x=pinv(A)*b

x =

-0.2782

2.0944

27

Page 28: EE263 homework 4 solutions - Stanford Universityweb.stanford.edu/class/archive/ee/ee263/ee263.1082/hw/hw4sol.pdfEE263 Prof. S. Boyd EE263 homework 4 solutions 5.2 Complex linear algebra

0.8609

-1.2284

-0.6904

0.1616

0.2273

0.0808

-0.3030

0.2475

>> H=[x(1:5)’; x(6:10)’]

H =

-0.2782 2.0944 0.8609 -1.2284 -0.6904

0.1616 0.2273 0.0808 -0.3030 0.2475

>>

Finally we can check that HG = HG = I using Matlab:

>> H*G

ans =

1.0000 -0.0000

-0.0000 1.0000

>> H*tilde_G

ans =

1.0000 0.0000

0.0000 1.0000

>>

Of course, there are other solutions as well. Indeed, since there are 10 variables and8 independent equations, there is a 2 dimensional set of H’s that satisfy the requiredcondition.

28