Lec 2: Mathematical Economics - Department of Economics...

Lec 2: Mathematical EconomicsIntroduction to Spectral Theory

Sugata Bag

Delhi School of Economics

24th August 2012

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 1 / 17

Introduction Eigenvalues-Eigenvectors

Definition: Eigen values & vectors

Definition

Let T : V → V be a LT. A scalar λ is an eigenvalue of T if there exists anon-zero vector v ∈ V s.t. T (v) = λv . Where v is then an eigenvectorof T , corresponding to the eigenvalue λ.

Note (1) If v is an eigenvector corresponding to λ, then so is cv , for allscalars c , because T (cv) = cT (v) = cλv = λ(cv). Moreover, if v , v

′are

eigenvectors corresponding to the same eigenvalue λ, then for scalarsc1, c2 we have T (c1v + c2v

′) = c1T (v) + c2T (v

′)

= c1λv + c2λv′= λ(c1v + c2v

′). So c1v + c2v

′is also an eigenvector

corresponding to λ. That is, the set of all eigenvectors corresponding to aparticular eigenvalue is a subspace. It (along with the zero vector) is calledthe eigenspace corresponding to λ.



Notions

Note (2) For a typical (n× n) matrix A and a typical (n× 1) vector v ,the vector Av is some point in <n (or C n, if A is a complex matrix). Inthe eigenvalue problem, we are looking for vectors v such that Av lies onthe same line through the origin as v itself.

Example

(1). A matrix for which all non-zero vectors are eigenvectors: In×n. For allnon-zero x ∈ <n, Ix = 1x ; all such x are eigenvectors corresponding to theeigenvalue 1.(2) Using the appropriate rotation matrix, rotate the earth through say90o . Typically, vectors x ∈ <3 will change direction on rotation. Butvectors lying on the axis of rotation will not change direction. For suchvectors v , Av = 1v .

The set of all eigenvalues of a LT T is called the spectrum of T , anddenoted σ(T ).



Characteristic Equation

T (v) = λv ⇒ (T − λI )(v) = 0. That is, eigenvectors v corresponding tothe eigenvalue λ comprise the Nullspace of the LT T − λI (along with thezero vector). In fact the nullspace N(T − λI ) is called the eigenspacew.r.t. λ.For there to be a non-zero solution v to (T − λI )(v) = 0, we needNullity(T − λI ) > 0, or Rank(T − λI ) < dim(V ). Let the matrix An×nrepresent T . So we have Rank(A− λI ) < n, which implies thedeterminant det(A− λI ) = 0. det(A− λI ) is just

det

a11 − λ a12 . . . a1na21 a22 − λ . . . a2n... . . .

. . ....

an1 . . . . . . ann − λ

So det(A− λI ) = 0 is nth-degree polynomial and can be written as -f (λ) = |(A− λI )| = (−λ)n + bn−1(−λ)n−1 + ...+ b1(−λ)1 + b0 ... (1)or in factor form, f (λ) = (λ1 − λ)(λ2 − λ)...(λn − λ) .... (2)[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 4 / 17


Characteristic Equation contd...

So det(A− λI ) = 0 has n solutions (roots), possibly repeated, andpossibly complex. If λ is a complex root of the equation, its conjugate λ̄ isalso a complex root.Comparison of Eq1 and Eq2 gives us -

bn−1 =n

∑i=1

λi = λ1 + λ2 + ...+ λn

bn−2 =n

∑j>i

λiλj = λ1λ2 + ...+ λ1λn + λ2λ3...+ λ2λn

+....+ λn−1λn

bn−r =n

∑k>j>i

λiλjλk [each term is product of r of the λi ]

...b0 = λ1λ2λ3....λn



Examples

The characteristic polynomial for a second-order matrix A = ||aij || is

f (λ) =

∣∣∣∣ a11 − λ a12a21 a22 − λ

∣∣∣∣ = (a11 − λ)(a22 − λ)− a12a21= (−λ)2 + (a11 + a22)(−λ) + a11a22 − a12a21= (−λ)2 + b1(−λ) + b0

hence, b1 = a11 + a22 , b0 = a11a22 − a12 + a21 = |A|Two roots of f (λ) = 0 are -

λ = 12{(a11 + a22)± [(a11 + a22)2 − 4|A|]1/2}

if λ1 and λ2 are the two roots then λ1λ2 = |A| = b0 andλ1 + λ2 = a11 + a22 = b1Recall: A quadratic eqn in λ can be as: λ2 − trace(A)λ+ det(A) = 0.

Note: The tsm of the diagonal elements of a square matrix A (calledtrace(A)) equals the sum of eigen values ∑n

i=1 λi . And det(A) = Πni=1λi ,

the product of eigen values.



Examples

Example 1. Let A2×2 have rows (1, 2) and (8, 1).det(A− λI ) = λ2 − 2λ− 15 = 0 has solutions λ1 = 5,λ2 = −3.Eigenvectors for λ1:(

1− 5 28 1− 5

)(x1x2

)=

(00

)The eigenspace is 1-dimensional; Rank(A− 5I ) = 1. It’s{(x1, x2)| − 4x1 + 2x2 = 0}. If we choose x1 = 1, we solve and getx2 = 2, so x = (1, 2)T is an eigenvector corresponding to λ = 5.Similarly, (1,−2)T is an eigenvector corresponding to λ2 = −3.



Examples

Example

A2×2 has rows (1, 2) and (−2, 1). det(A− λI ) = λ2 − 2λ+ 5 = 0

implies the roots equal 2±√4−202 . So λ1 = 1+ 2i and λ2 = λ1.

Eigenvector corresponding to λ1:

(A− (1+ 2i)I )x =(−2i 2−2 −2i

)(x1x2

)=

(00

)Rank(A− λ1I ) = 1 (the 2nd row is i times the first). Solving−2ix1 + 2x2 = 0 by setting x1 = 1, we get x2 = i . So (1, i)T is aneigenvector corresponding to λ1. The conjugate of this vector, (1,−i)T istherefore an eigenvector corresponding to the conjugate root λ2 (this is ageneral result).



Identical Spectrums

Can we replaced a linear transformation T with a matrix A representingit? Indeed all matrices representing T in different bases are similar to eachother. If A and B are 2 such matrices, then A = SBS−1 for someinvertible (change of basis) matrix S .

Definition

Similarity: If there exists a nonsingular matrix S such that B = SAS−1,the square matrices A and B are said to be similar.

LemmaSimilar matrices have an identical spectrum.

Proof: Let Av = λv , so λ is an eigenvalue of A. If A = SBS−1, then thisimplies SBS−1v = λv . Premultiplying both sides by S−1,BS−1v = S−1λv = λS−1v . So λ is an eigenvalue of B (and S−1v acorresponding eigenvector).[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 9 / 17

Introduction Symmetric Matrix & Characteristic Value

Theorems: Symmetric Matrix

TheoremThe eigenvectors corresponding to different eigenvalues are orthogonal;i.e., if xi , xj are eigen-vectors corresponding to eigenvalues λi and λjrespectively, then x ′i .xj = 0.

Proof.

By assumption, Axi = λixi and Axj = λjxj . Thus xj ′Axi = λixj ′xi andxj ′Axj = λjxj ′xi . Subtracting and noting that xj ′Axi = xj ′Axj , we have(λi − λj )xj ′xi = 0, or xj ′xi = 0 , since λi 6= λj .If x 6= 0 is an eigenvectorof A, then any scalar multiple of x is also an eigenvector of A,corresponding to the same eigenvalue.

Note: For convenience, we shall always assume that the eigenvectors of asymmetric matrix are of unit length; denoted by ui; and they are said tohave been normalized. A set of two or more normalized eigenvectors uicorresponding to different eigenvalues of A satisfies the equation uiuj= δijand are said to be orthonormal.[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 10 / 17


Example

Find the eigen values and eigen vectors of A =

∣∣∣∣ 2√2√

2 1

∣∣∣∣ .The characteristic equationis |A− λI | =

∣∣∣∣ 2− λ√2√

2 1− λ

∣∣∣∣ = λ2 − 3λ = 0

thus the eigenvalues are λ1 = 0 and λ2 = 3.To determine eigen vectors corresponding λi , we must solve the set ofhomogenous equation [A− λI ]x = 0. Let us take λ1 = 0 first, then theset of equations becomes -

2x1 +√2x2 = 0 and

√2x1 + x2 = 0

Implies, x1 = − 1√2x2

If we wish to find an eigen vector of unit length, we must require thatx21 + x

22 = 1. Thus

32x22 = 1; choosing the positive sqare root, we obtain -

x2 =√ 23 and x1 = − 1√

3 .

And the eigenvector of unit length corresponding to λ1 is -u1 = [− 1√

3 ,√ 23 ]



Example contd...

Note that u1 is not completely specified by the requirement of |u1| = 1.The vector −u1 is also an eigen vector of length 1. However u1 and -u1are not linearly independent. Only one linearly independent vectorcorresponds to λ1.For λ2 = 3, the set of equations becomes -

−x1 +√2x2 = 0 and

√2x1 − 2x2 = 0

implies, x1 =√2x2.

If x21 + x22 = 1 then 3x

22 = 1; taking poaitive square root we get -

x2 = 1√3 , x2 =

√ 23

So u2 = [√ 23 ,

1√3 ].

Again there is only one linearly independent eigen vector that correspondsto λ2; it can easily be checked that u1u2 = 0, in agrrement with thetheory.


Introduction Diagonalization

Diagonal Form of Matrix

Theorem

Suppose An×n has n LI eigenvectors. Then S−1AS = D, where thecolumns of S are the n eigenvectors and Λ is the diagonal matrix whosediagonals are the corresponding eigenvalues.

Proof.

Note S = (v1, ..., vn) is invertible (eigenvectors written as columns). ThenAS = A(v1 v2 ... vn) = (Av1 Av2 ... Avn) = (λ1v1 λ2v2 ... λnvn)

=

...

.... . .

...v1 v2 . . . vn...

.... . .

...

λ1 0 . . . 00 λ2 . . . 0...

.... . .

...0 0 . . . λn

= SD

Premultiplying both sides of AS = SD by S−1, S−1AS = D.



When can we Diagonalize?

Lemma

Suppose An×n has n distinct eigenvalues λ1, ...,λn. Then a set ofeigenvectors v1, ..., vn, vi corresponding to λi is LI.

Proof.

(By induction). The statement is trivially true if n = 1. Suppose it’s truefor n = 1, 2, ..., k − 1; i.e. if λ1, ...,λk−1 are distinct and v1, ..., vk−1 arecorresponding eigenvectors, then they are LI. Suppose λk is an eigenvaluedistinct from λ1, ...,λk−1 and vk is a corresponding eigenvector. For somescalars c1, ..., ck suppose that c1v1 + ...+ ckvk = 0 (1) .So, A(c1v1 + ...+ ckvk ) = A0 = 0 . By linearity of A we get -c1Av1 + ...+ ckAvk = 0 ⇒ c1λ1v1 + ...+ ckλkvk = 0 (2).Eq(2) minus λk (Eq(1)) eliminates λk , giving:c1(λ1 − λk )v1 + ...+ ck−1(λk−1 − λk )vk−1 = 0. By LI of {v1, ..., vk−1},ci (λi − λk ) = 0, ∀i = 1, .., k − 1. Since λi 6= λk , ci = 0, ∀i = 1(1)k − 1.Plugging ci’s in Eq(1), we get ck = 0 also, so that {v1, .., vk} are LI.[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 14 / 17


Why Diagonalize?

(1) Interpretation. If A is diagonalizable, then A = SΛS−1. That is A andΛ are similar. Specifically, suppose x = (x1, ..., xn) is a vector written interms of the standard basis. Suppose the LI eigenvectors {v1, ..., vn} arethe new basis. If Sn×n = (v1 v2 ... vn), then w.r.t. the new basis, weknow that x is represented as y = S−1x . Now, the matrix A carries x toAx . In terms of the new basis, Ax is represented as S−1Ax . Since x = Sy ,S−1Ax = S−1ASy = Λy . So wrt the new basis, the action of A on xbecomes the easier to visualize action of Λ on y . Λ just stretches thecoordinates of y by possibly different amounts.(2) Uses - including more general uses of finding the spectrum σ(A): Insolving and interpreting solutions to differential equations; also in somespecification testing in econometrics, interpreting quadratic forms,frequency domain time series analysis etc.



Why Diagonalize

(3) If A is real, symmetric and full-ranked, then we can find a set ofeigenvectors that are LI as well as orthogonal. Then the matrix S ofeigenvectors is called an orthogonal matrix. These are generally rotationmatrices; so A = SΛS−1 means the action of A is a composition of arotation followed by a stretch followed by another rotation. Examples ofsuch matrices: (i) the (XTX ) or (X ∗X ) matrix discussed in the leastsquares application; (ii) the Hessian D2f of a function f : <n → <.



Examples

For the matrix

∣∣∣∣ 2√2√

2 1

∣∣∣∣And the eigenvector of unit length corresponding to λ1 is -

u1 = [− 1√3 ,√ 23 ]

u2 = [√ 23 ,

1√3 ]

S = (u1, u2) = 1√3 [−1 √2√2 1

]

S ′AS = 13

[−1 √2√2 1

] [2√2√

2 1

] [−1 √2√2 1

]= 1

3

[2√2√

2 1

] [0 3√2

0 3

]=

[0 00 3

]=

[λ1 00 λ2

]


Lec 2: Mathematical Economics - Department of Economics...

Documents

Transcript of Lec 2: Mathematical Economics - Department of Economics...