Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra....

62
Abstract Linear Algebra Linear Algebra. Session 11 Dr. Marco A Roque Sol 04 / 21 / 2020 Dr. Marco A Roque Sol Linear Algebra. Session 11

Transcript of Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra....

Page 1: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra

Linear Algebra. Session 11

Dr. Marco A Roque Sol

04 / 21 / 2020

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 2: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Inner Product.

Find the distance from the point (2, 1,−2) to the plane Π, givenby:.

6(x − 1) + 2(y − 3) + 3(z + 4) = 0.

Solution

The Solution of the above equation is : xyz

= α

10−2

+ β

03−2

+

13−4

so the vectors x1 = (1, 0,−2)T and x2 = (0, 3,−2)T span theplane Π.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 3: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

The orthogonal projection,p, of v = (2, 1,−2)T − (1, 3,−4)T =(1,−2, 2)T onto the plane Π is given by:

p =< v, x1 >

< x1, x1 >x1 +

< v, x2 >

< x2, x2 >x2

where< v, x1 >=< 1,−2, 2 > · < 1, 0,−2 >= −3

< v, x2 >=< 1,−2, 2 > · < 0, 3,−2 >= −10

< x1, x1 >=< 1, 0,−2 > · < 1, 0,−2 >= 5

< x2, x2 >=< 0, 3,−2 > · < 0, 3,−2 >= 13

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 4: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

It follows that

p = −35x1 − 10

13x2 = −35(1, 0,−2)T − 10

13(0, 3,−2)T ⇒

p = (−35 ,−

1013 ,

20865 )T

Finally, the distance from (2, 1,−2) to the palne Π is :

||o = v − p|| = ||(1,−2, 2)T − (−3

5,−10

13,

208

65)T || =

||o|| = ||(8/5,−16/13,−78/65)T || = 65||(104,−80,−78)T ||

||o|| = 65√

1042 + 802 + 782 = 65√

23300 �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 5: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Orthogonal Subspaces.

(a) Let S be the subspace of R3 spanned by the vectorsx = (x1, x2, x3)T and y = (y1, y2, y3)T , and let A be the matrixdefined by:

A =

(x1 x2 x3y1 y2 y3

)(a) Show that S⊥ = N(A).

(b) Find the orthogonal complement of the subspace of R3

spanned by x = (1, 2, 1)T and y = (1,−1, 2)T .

Solution

a) A vector v = (v1, v2, v3) belongs to S⊥ if and only if

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 6: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

< v, x >= 0

< v, y >= 0

but that is true if and only if

v1x1 + v2x2 + v3x3 = 0

v1y1 + v2y2 + v3y3 = 0

if and only if, in matrix notation

(x1 x2 x3y1 y2 y3

) v1v2v3

=

(00

)

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 7: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

if and only if

Av = 0

if and only if v ∈ S⊥ �

b) Using a), we are looking for the solutions of

Av =

(x1 x2 x3y1 y2 y3

) v1v2v3

=

(1 2 11 −1 2

) v1v2v3

=

(00

)

using elementary row operations we find the row-echelon form ofthe matrix A

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 8: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

(1 2 10 −3 1

) v1v2v3

⇒ (1 2 10 1 −1/3

) v1v2v3

=

(00

)

and we obtain the system

v1 + 2v2 + v3 = 0v2 − 1

3v3 = 0

whose solution is the one-dimensional subspace:

N(A) =

v ∈ R3/ v = α

−513

, α ∈ R

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 9: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Least Squares Solutions.

Find the least squares solution of each of the following system:

−x1 + x2 = 102x1 + x2 = 5x1 − 2x2 = 20

Solution

In matrix notation, the above system can be written as

Ax = b⇒

−1 12 11 −2

( x1x2

)=

105

20

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 10: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

and we know that a vector y is a lest square approximation for thesystem if, it minimizes ||r(y)|| = ||Ay − b||, and this is happeningif y satisfies the normal equations. namely,

ATAy = ATb

(−1 2 11 1 −2

) −1 12 11 −2

( y1y2

)=

(−1 2 11 1 −2

) 105

20

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 11: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

(6 −1−1 6

)(y1y2

)=

(20−25

)⇒

y =

(2.71−3.71

)�

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 12: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Inner Product.

In C [−π, π] with inner product defined by

< f , g >=1

π

∫ π

−πf (x)g(x)dx ,

show that cos(mx) and sin(nx) are orthogonal and that both areunit vectors. Determine the distance between the two vectors.

Solution

< cos(mx), sin(nx) >=1

π

∫ π

−πcos(mx) sin(nx)dx

and using the Trig. Identitiy

cos a sin b =1

2[sin(a + b)x − sin(a− b)x ]

we obtain:Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 13: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

< cos(mx), sin(nx) >=1

∫ π

−π[sin(m + n)x − sin(m − n)x ]dx =

1

∫ π

−πsin(m + n)x − 1

∫ π

−πsin(m − n)x = 0− 0 = 0

On the other hand

< cos(mx), cos(mx) >=1

π

∫ π

−πcos2(mx)dx =

1

∫ π

−π[1+cos(2mx)]dx

=1

∫ π

−π1dx +

1

∫ π

−πcos(2mx)dx = 1 + 0 = 1.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 14: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

< sin(nx), sin(nx) >1

π

∫ π

−πsin2(mx)dx =

1

∫ π

−π[1− cos(2mx)]dx

=1

∫ π

−π1dx − 1

∫ π

−πcos(2mx)dx = 1− 0 = 1

Now the distance between x = cos(mx) and y = sin(nx) is givenbiy

||x− y||2 =< x− y, x− y >=1

π

∫ π

−π[cos(mx)− sin(nx)]2dx =

1

π

∫ π

−π[cos2(mx)−2 cos(mx) sin(nx) + sin2(mx)]dx = 1−0 + 1 = 2

Therefore ||x− y|| =√

2 �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 15: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Norms

Determine whether the following define norms on C [a, b]:

(a) ||f || = |f (a)|+ |f (b)|

(b) ||f || =∫ ba |f (x)|dx

(c) ||f || = maxa≤x≤b |f (x)|

Solution

Remmember that a function || · || : V→ R is called a norm definedin the vector space V, if satisfies:

||x|| ≥ 0, ||x|| = 0 only for x = 0

||rx|| = |r |||x|| for all r ∈ R.

||x + y|| ≤ ||x||+ ||y||.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 16: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

a) ||f || = |f (a)|+ |f (b)| .

||f || = |f (a)|+ |f (b)| = 0⇒ |f (a)| = 0, |f (b)| = 0, but notnecessarily that the function f (x) = 0 in the whole interval,Therefore, this function, does not define a norm ... !!!!

b) ||f || =∫ ba |f (x)|dx .

||f || =∫ ba |f (x)|dx > 0 provided f (x) 6= 0. Now, if

||f || =∫ ba |f (x)|dx = 0 this implies, from Calculus, that f (x) = 0

on [a, b]

||rf || =∫ ba |rf (x)|dx = |r |

∫ ba |f (x)|dx = |r |||f ||.

||f + g || =∫ ba |f (x) + g(x)|dx ≤

∫ ba |f (x)|dx +

∫ ba |g(x)|dx =

||f ||+ ||g ||.

Therefore, this function, defines a norm.Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 17: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

c) ||f || = maxa≤x≤b |f (x)| .

||f || = maxa≤x≤b |f (x)| > 0 provided f (x) 6= 0. Now, if||f || = maxa≤x≤b |f (x)| = 0 this implies that f (x) = 0 on [a, b]

||rf || = maxa≤x≤b |rf (x)| = maxa≤x≤b |r ||f (x)| =

|r |maxa≤x≤b |f (x)| = |r |||f ||.

||f + g || = maxa≤x≤b |f (x) + g(x)| ≤= maxa≤x≤b |f (x)|+ |g(x)| =

||f ||+ ||g ||.

Therefore, this function, defines a norm.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 18: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Orthogonal Polynomials

Let p0, p1, . . . be a sequence of orthogonal polynomials and let andenote the lead coefficient of pn. Prove that

||pn||2 = an < xn, pn >

Solution

If p0, p1, . . . is a sequence of orthogonal polynomials, then weknow that

i) deg(pn) = n

ii)< pn, xk >= 0, for 0 ≤ k ≤ n − 1

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 19: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Thus, if pn(x) is given by

pn(x) = anxn + an−1x

n−1 + · · ·+ a1x + a0

then

||pn||2 =< pn(x), pn(x) >

||pn||2 =< anxn + an−1x

n−1 + · · ·+ a1x + a0, pn >=

an < xn, pn > +an−1 < xn−1, pn > + · · ·+ a1 < x , pn > +a0 < 1, pn >

but due to ii), we have that

||pn||2 = an < xn, pn > �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 20: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Orthogonal Polynomials

Let Tn(x) denote the Chebyshev polynomial of degree n and define

Un−1(x) =1

nT ′n(x); n = 1, 2, . . .

(a) Compute U0(x),U1(x), and U2(x).

(b) Show that if x = cos(θ) , then

Un−1 =sin(nθ)

sin(θ).

Solution

a) T1 = x ⇒ U0 = T ′1(x) = 1

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 21: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

T2 = 2x2 − 1⇒ U1 = 12T′2(x) = 2x

T3 = 4x3 − 3x ⇒ U2 = 13T′3(3) = 4x2 − 1

T4 = 8x4 − 8x2 + 1⇒ U3 = 14T′4(x) = 8x3 − 4x

b)

Un−1(x) =1

nT ′n(x)

but we know that Tn(cos θ) = cos(nθ), therefore

Un−1(x) =1

n

dTn(x)

dx=

1

n

dTn(x)

dx=

1

n

dTn(cos(θ))

dθ/dx

dθ⇒

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 22: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Un−1(x) =1

n

d cos(nθ)

dθ/dx

Un−1(x) =1

n[−n sin(nθ)] /[− sin(θ)]

Un−1(x) =sin(nθ)

sin(θ)�

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 23: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Markov Chain

Let A be an n × n stochastic matrix and let e be the vector in Rn

whose entries are all equal to 1. Show that e is is an eigenvector ofAT . Explain why a stochastic matrix must have λ = 1 as aneigenvalue.

Solution

Let A be the stochastic matrix with entries given by

A =

a11 a12 · · · a1na21 a22 · · · a2n

... · · ·an1 an2 · · · ann

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 24: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

By the definition of a stochastic matrix ( the entries of eachcolumn of A are nonnegative numbers that add up to 1 ), then Asatifies:

a1j + a2j + · · ·+ anj = 1; for all 1 ≤ j ≤ n

and At = (bij) satisfies

bi1 + bi2 + · · ·+ bin = 1; for all 1 ≤ i ≤ n

Therefore

ATe =

b11 b12 · · · b1nb21 b22 · · · b2n

... · · ·bn1 bn2 · · · bnn

11...1

=

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 25: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

b11 b12 · · · b1nb21 b22 · · · b2n

... · · ·bn1 bn2 · · · bnn

11...1

=

b11 + b12 + · · ·+ bnb21 + b22 + · · ·+ b2n

...bn1 + bn2 + · · ·+ bnn

b11 b12 · · · b1nb21 b22 · · · b2n

... · · ·bn1 bn2 · · · bnn

11...1

=

11...1

Thus, e is an eigenvector of AT with eigenvalue λ = 1. �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 26: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Markov Chain

In the Dark Ages, Harvard, Dartmouth, and Yale admitted onlymale students. Assume that, at that time, 80 percent of the sonsof Harvard men went to Harvard and the rest went to Yale, 40percent of the sons of Yale men went to Yale, and the rest splitevenly between Harvard and Dartmouth; and of the sons ofDartmouth men, 70 percent went to Dartmouth, 20 percent toHarvard, and 10 percent to Yale.

(i) Find the probability that the grandson of a man from Harvardwent to Harvard.

(ii) Modify the above by assuming that the son of a Harvard manalways went to Harvard. Again, find the probability that thegrandson of a man from Harvard went to Harvard.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 27: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Solution

i)

We first form a Markov chain with state space S = {H,D,Y } andthe following transition probability matrix :

P =

.8 0 .2.2 .7 .1.3 .3 .4

Recall: the ij th entry of the matrix Pn gives the probability that theMarkov chain starting in state i will be in state j after n steps.sd

Thus, the probability that the grandson of a man from Harvardwent to Harvard is the upper-left element of the matrix P2.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 28: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

P2 =

.7 .06 .24.33 .52 .15.42 .33 .25

ii)

If all sons of men from Harvard went to Harvard, this would givethe following matrix for the new Markov chain with the same set ofstates:

P =

1 0 0.2 .7 .1.3 .3 .4

and The upper-left element of P2 is 1. Does that make sense ????

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 29: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Markov Chain

Let consider a hypothetical market with Markov properties wherehistorical data has given us the following patterns: After a weekcharacterized of a bull market trend there is a 90% chance thatanother bullish week will follow. Additionally, there is a 7.5%chance that the bull week instead will be followed by a bearish one,or a 2.5% chance that it will be a stagnant one. After a bearishweek theres an 80% chance that the upcoming week also will bebearish, and so on.

Calculate the possibilities of a bull, bear or stagnant week for, 1and 5 weeks, into the future.

Solution

By compiling these probabilities into a table, we get the followingtransition matrix M:Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 30: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

M =

0.9 0.075 0.0250.15 0.8 0.050.25 0.25 0.05

Then create an initial vector x0 which contains information aboutwhich of the three different states any current week is in. In thisexample we will choose to set the current week as bearish,therefore x0 = (0, 1, 0)

Given the state of the current week, we can then calculate thepossibilities of a bull, bear or stagnant week for any number of nweeks into the future. In this way, after one and five weeks we willhave:

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 31: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

x1 = (0, 1, 0)

0.9 0.055 0.0250.15 0.8 0.050.25 0.25 0.05

1

= (0.15, 0.8, 0.05)

and

x5 = (0, 1, 0)

0.9 0.055 0.0250.15 0.8 0.050.25 0.25 0.05

5

= (0.48, 0.45, 0.07) �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 32: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Solve, in the traditional way ( using elementary operations -row reduction method ), each of the following systems

i)

x1 + x2 + x3 + x4 + x5 = 2x1 + x2 + x3 + 2x4 + 2x5 = 3x1 + x2 + x3 + 2x4 + 3x5 = 2

ii)

x1 + 2x2 + x3 = 12x1 + 4x2 + 2x3 = 3

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 33: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Solution

i) Using row-reduction to the augmented matrix

1 1 1 1 1 21 1 1 2 2 31 1 1 2 3 2

⇒ 1 1 1 1 1 2

0 0 0 1 1 10 0 0 1 2 0

⇒ 1 1 1 1 1 2

0 0 0 1 1 10 0 0 0 1 −1

⇒ 1 1 1 0 0 1

0 0 0 1 1 10 0 0 0 1 −1

⇒ 1 1 1 0 0 1

0 0 0 1 0 20 0 0 0 1 −1

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 34: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Thus, we have that the solution is

x5 = −1, x4 = 2, x3 = α, x2 = β, x1 = 1− α− β

x =

x1x2x3x4x5

=

1− α− β

βα2−1

=

1002−1

−10100

−11000

Thus, the solution is 2− dimensional subspace span by vectorsv1 = (1, 0, 1, 0, 0)T and v2 = (−1, 1, 0, 0, 0, )T

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 35: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Determine whether the following are linear transformations fromP2 to P3.

a) L(p(x)) = xp(x)

(b) L(p(x)) = x2 + p(x)

(c) L(p(x)) = p(x) + xp(x) + x2p′(x)

Solution

a) For a and b real numbers, we have:

L(ap(x) + bq(x)) = x [ap(x) + bq(x)] = x [ap(x)] + x [bq(x)] =

a[xp(x)] + b[xq(x)] = aL(p(x)) + bL(q(x))

so, the Transformation is linear.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 36: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

b) For a and b real numbers, we have:

L(ap(x) + bq(x)) = x2 + [ap(x) + bq(x)] 6=

x2 + [ap(x)] + x2 + [bq(x)] = aL(p(x)) + bL(q(x))

so, the Transformation is not linear.

c) For a and b real numbers, we have:

L(ap(x) + bq(x)) = (ap(x) + bq(x)) + x [ap(x) + bq(x)]+

x2[ap(x) + bq(x)]′ = a[p(x) + xp(x) + p′(x)]+

b[q(x) + xq(x) + q′(x)] = aL(p(x)) + bL(q(x))

so, the Transformation is Linear.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 37: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Let

b1 =

110

, b1 =

101

, b3 =

011

,

and let L be the linear transformation from R2 into R3 defined by

L(x) = x1b1 + x2b2 + (x1 + x2)b3

Find the matrix A representing L with respect to the ordered bases{e1, e2} and {b1,b2,b3}.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 38: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Solution

L(e1) = b1 + b3 = 1 · b1 + 0 · b2 + 1 · b3

L(e2) = b2 + b3 = 0 · b1 + 1 · b2 + 1 · b3

Thus, the matrix AL representing the transformation w.r.t. thegiven bases is

AL =

1 00 11 1

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 39: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

A linear transformation L : V→W is said to be one-to-one ifL(v1) = L(v2) implies that v1 = v2 (i.e., no two

distinct vectors v1, v2 in V get mapped into the same vectorw ∈W ). Show that L is one-to-one if and only if

ker(L) = {0V}.

Solution

if L is one to one, then

L(v1) = L(v2)⇒ v1 = v2

Now, if v ∈ ker(L) then L(v) = 0W, but we already know thatL(0V) = 0W ⇒ v = 0 for all v ∈ ker(L)⇒ ker(L) = {0V}

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 40: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

On the other hand, let’s assume that ker(L) = {0V}. Suppose that

L(v1) = L(v2)⇒

L(v1)− L(v2) = 0W ⇒

L(v1 − v2) = 0W ⇒

v1 − v2 ∈ ker(L)⇒

v1 − v2 = 0V ⇒

v1 = v2 ⇒

L is a one-to-one linear transformation. �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 41: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Let V be a subspace of F(R) spanned by functions ex , e−x . Let Lbe a linear operator on V such that

AL =

(2 −1−3 2

)is the matrix of L relative to the basis ex , e−x . Find the matrix ofL relative to the basis

cosh(x) =1

2(ex + e−x), sinh(x) =

1

2(ex − e−x)

Solution

From the above we know that

L(ex) = 2ex − 3e−x ; L(e−x) = −ex + 2e−x

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 42: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

and from the equations

cosh(x) = 12(ex + e−x), sinh(x) = 1

2(ex − e−x)⇒

we have

ex = cosh(x) + sinh(x) e−x = cosh(x)− sinh(x)⇒

L(ex) = 2ex−3e−x = 2(cosh(x)+sinh(x))−3(cosh(x)−sinh(x))⇒

L(ex) = 2ex − 3e−x = − cosh(x) + 5 sinh(x))

and

L(e−x) = −ex + 2e−x =−(cosh(x) + sinh(x)) + 2(cosh(x)− sinh(x))⇒

L(e−x) = −ex + 2e−x = cosh(x)− sinh(x)⇒

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 43: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

L(cosh(x)) = L(12(ex + e−x)) = 12L(ex) + 1

2L(e−x)⇒12 [− cosh(x) + 5 sinh(x)] + 1

2 [cosh(x)− sinh(x)]

L(cosh(x)) = 2 sinh(x)

Similarly

L(sinh(x)) = L(12(ex − e−x)) = 12L(ex)− 1

2L(e−x)⇒12 [− cosh(x) + 5 sinh(x)]− 1

2 [cosh(x)− sinh(x)]

L(sinh(x)) = − cosh(x) + 3 sinh(x)

and the matrix BL, representing L in this new basis is

BL =

(0 −12 3

)�

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 44: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Let L be a linear operator on Rn. Suppose that L(x) = 0 for somex 6= 0. Let A be the matrix representing L with respect to thestandard basis {e1, e2, . . . , en}. Show that A is singular.

Solution

A be the matrix representing L w.r.t. to standard basis, then theequation

L(x) = 0⇒ Ax = 0

is valid for a vector x 6= 0, therefore system of the equations

Av = 0

has not a unique solution, namely, v = 0 and v = x , so thatimplies that A is a singular matrix, otherwise the system wouldhave a unique solution, which is not the case. �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 45: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

let

A =

1 2 10 3 10 5 −1

(i) Find all eigenvalues of the matrix A.

(ii) For each eigenvalue of A, find an associated eigenvector.

(iii) Is the matrix A diagonalizable? Explain.

(iv) Find all eigenvalues of the matrix A2.

Solution

(i) Eigenvalues are given by the characteristic equation:

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 46: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

|A− λI| =

∣∣∣∣∣∣1− λ 2 1

0 3− λ 10 5 −1− λ

∣∣∣∣∣∣ = 0⇒

(1− λ)

∣∣∣∣ 3− λ 15 −1− λ

∣∣∣∣ = (1− λ) [(3− λ)(−1− λ)− 5] =

(λ− 1)(λ− 4)(λ+ 21) = 0

Thus the eigenvalues are: λ1 = 1, λ2 = 4, λ3 = −2 .

(ii) An eigenvector corresponding to the eigenvalue λ1 = 1 is givenby

(A− λ1I)x =

1− λ1 2 10 3− λ1 10 5 −1− λ1

x1x2x3

=

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 47: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

0 2 10 2 10 5 −2

x1x2x3

=

000

Thus, an eigenvector is

v1 =

100

=

An eigenvector corresponding to the eigenvalue λ2 = 4 is given by

(A− λ2I)x =

1− λ2 2 10 3− λ2 10 5 −1− λ2

x1x2x3

=

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 48: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

−3 2 10 −1 10 5 −5

x1x2x3

=

000

Thus, an eigenvector is

v2 =

111

=

An eigenvector corresponding to the eigenvalue λ3 = −2 is givenby

(A− λ2I)x =

1− λ3 2 10 3− λ3 10 5 −1− λ3

x1x2x3

=

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 49: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

3 2 10 5 10 5 1

x1x2x3

=

000

Thus, an eigenvector is

v3 =

11−5

Thus, the matrix A has three linearly independent vectors, namely:v1, v2, v3.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 50: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

(iii) The matrix A is diagonalizable since

A = XDX−1 =

1 1 10 1 10 1 5

1 0 00 4 00 0 −2

1 1 10 1 10 1 5

−1

(iii) The matrix A2 is given by

A2 =(XDX−1

) (XDX−1

)⇒

A2 = XD2X−1

whereD2 =

1 0 00 16 00 0 4

so the eigenvalues of A2 are 1, 16, 4 �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 51: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Find the kernel and range of each of the following linear operatorson P3 :

(a) L(p(x)) = xp′(x)

(b) L(p(x)) = p(x)− p′(x)

(c) L(p(x)) = p(0)x + p(1)

Solution

ker(L) is the set of all polynomials such that L(p(x)) = 0

a)

L(p(x)) = xp′(x) = 0⇒ p′(x) = 0⇒ p(x) = a0 = cte.

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 52: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

b)

L(p(x)) = p(x)− p′(x) = 0⇒ p′(x) = p(x)⇒ p(x) = 0 is theonly polynomial satisfying that eqation.

c)

L(p(x)) = p(0)x + p(1) = 0⇒ p(0) = 0 and p(1) = 0

Thus, the kernel is the set of all quadratic polymomials satisfayingthe above condition, that is p(x) = α(x − x2). �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 53: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Find the singular value decomposition of each of the followingmatrices:

i)

A =

(1 12 2

)ii)

B =

(2 −21 2

)Solution

i)

The matrix ATA is given by :

ATA =

(5 55 5

)Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 54: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

whose eigenvalues and eigenvectors are: λ1 = 10, λ2 = 0

v1 =

(11

); v2 =

(−11

)

The matrix is AAT :

AAT =

(2 44 8

)whose eigenvalues and eigenvectiors are λ1 = 10 and λ2 = 0

u1 =

(12

); u2 =

(−21

)Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 55: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Thus the SVD decomposition of the matrix A

A = UΣVT =

(1 −22 1

)( √10 00 0

)(1 −11 0

)T

ii)

The matrix BTB is given by:

BTB =

(5 −2−2 8

)whose eigenvalues and eigenvectors are: λ1 = 9, λ2 = 4

v1 =

(−12

); v2 =

(21

)

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 56: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

The matrix is BBT :

BBT =

(8 −2−2 5

)whose eigenvalues and eigenvectiors are λ1 = 9 and λ2 = 4

u1 =

(−21

); u2 =

(12

)Thus the SVD decomposition of the matrix B

B = UΣVT =

(−2 11 2

)(3 00 2

)(−1 22 1

)T

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 57: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Given x1 = 12(1, 1, 1,−1) and x2 = 1

6(1, 1, 3, 5)verify that these vectors form an orthonormal set in R4. Extendthis set to an orthonormal basis for R4 by finding an orthonormalbasis for the null space of

A =

(1 1 1 −11 1 3 5

)[Hint: First find a basis for the null space and then use theGram-Schmidt process.]

Solution< x1, x2 >= 1

12 [1 + 1 + 3− 5] = 0

< x1, x1 >= 1 < x2, x2 >= 1

Therefore {x1, x1} form an orthonormal set.Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 58: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Now, let’s find the Null Space of the matrix A, i. e. , solve thesystem

Av = 0⇒(

1 1 1 −11 1 3 5

)v1v2v3v3

=

(00

)Using row-reduction we have

Av = 0⇒(

1 1 1 −1 01 1 3 5 0

)⇒(

1 1 1 −1 00 0 2 6 0

)⇒

(1 1 1 −1 00 0 1 3 0

)⇒(

1 1 0 −4 00 0 1 3 0

)⇒

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 59: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

(1 1 0 −4 00 0 1 3 0

)So, the solution is given by

v1 = 4α− β, v2 = β, v3 = −3α, v4 = α

Thus, a basis for the Null space is v1 = (−1, 1, 0, 0) andv2 = (4, 0,−3,−1)

and using the Gram-schmidt process we come up with

u1 = v1||v1||

u2 = v2 − <v2,u1><u1,u1>

u1/||v2 − <v2,u1><u1,u1>

u1||

u1 = 1√2

(−1, 1, 0, 0) and u2 =√26 (2, 2,−3, 1) �

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 60: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 61: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Dr. Marco A Roque Sol Linear Algebra. Session 11

Page 62: Linear Algebra. Session 11 - math.tamu.eduroquesol/Math_304...Dr. Marco A Roque Sol Linear Algebra. Session 11. Abstract Linear Algebra Orthogonal Polynomials Examples let A = 0 @

Abstract Linear Algebra Orthogonal Polynomials

Examples

Dr. Marco A Roque Sol Linear Algebra. Session 11