• date post

19-Mar-2018
• Category

## Documents

• view

218

4

Embed Size (px)

### Transcript of Lecture 10: Eigenvectors and eigenvalues - pha.jhu. neufeld/numerical/ 10: Eigenvectors and...

• 220

Lecture 10:Eigenvectors and eigenvalues

(Numerical Recipes, Chapter 11)

The eigenvalue problem,

A x = x,

occurs in many, many contexts:

classical mechanics, quantum mechanics, optics

• 221

Eigenvectors and eigenvalues(Numerical Recipes, Chapter 11)

The textbook solution is obtained from

(A I) x = 0, which can only hold for x 0 if

det (A I) = 0.

This is a polynomial equation for of order N

• 222

Eigenvectors and eigenvalues(Numerical Recipes, Chapter 11)

This method of solution is:

(1) very slow(2) inaccurate, in the presence of roundoff error(3) yields only eigenvalues, not eigenvectors

(4) of no practical value (except, perhaps in the 2 x 2 case where the solution to the resultant quadratic can be written in closed form)

• 223

Review of some basic definitions in linear algebra

A matrix is said to be:

Symmetric if A = AT (its transpose) i.e. if aij = aji

Hermitian if A = A (AT)* i.e. if aij = aji*(its Hermitian conjugate)

Orthonormal if U1 = UT

Unitary if U1 = U

• 224

Review of some basic definitions in linear algebra

All 4 types are normal, meaning that they obey the relations

A A = A AU U = U U

• 225

Eigenvectors and eigenvalues

Hermitian matrices are of particular interest because they have:

real eigenvaluesorthonormal eigenvectors (xi xj = ij )(unless some eigenvalues are degenerate, in which case we can always design an orthonormalset)

• 226

Diagonalization of Hermitian matrices

Hermitian matrices can be diagonalizedaccording to

A = U D U1

Diagonal matrix consisting of the eigenvalues

Unitary matrix whose column are the eigenvectors

• 227

Diagonalization of Hermitian matrices

Clearly, if A = U D U1, then A U = U D..which is simply the eigenproblem(written out N times):

The columns of U are the eigenvectors (mutually orthogonal) and the elements of D (non-zero only on the diagonal) are the corresponding eigenvalues.

• 228

Non-Hermitian matrices

We are often interested in non-symmetric real matrices:

the eigenvalues are real or come in complex-conjugate pairs

the eigenvectors are not orthonormal in general

the eigenvectors may not even span an N-dimensional space

• 229

Diagonalization of non-Hermitian matrices

For a non-Hermitian matrix, we can identify two (different) types of eigenvectors

Right hand eigenvectors are column vectors which obey:A xR = xR

Left hand eigenvectors are row vectors which obey:xL A = xL

The eigenvalues are the same (the roots of det (A I) = 0 in both cases) but, in general, the eigenvectors are not

• 230

Diagonalization of non-Hermitian matrices

Note: for a Hermitian matrix, the two types of eigenvectors are equivalent, since

A xR = xR since A = A since is real

xR A = xR xR A = xR

so if any vector xR is a right-hand eigenvector, then xR is a left-hand eigenvector

• 231

Diagonalization of non-Hermitian matrices

Let D be the diagonal matrix whose elements are the eigenvalues, D = diag (i)

Let XR be the matrix whose columns are the right-hand eigenvectors

Let XL be the matrix whose rows are the left-hand eigenvectors

Then the eigenvalue equations are

A XR = XR D and XL A = D XL

• 232

Diagonalization of non-Hermitian matrices

The eigenvalue equations

A XR = XR D and XL A = D XL

imply XLA XR = XL XR D and XL A XR = D XL XR

Hence, XL A XR = XL XR D = D XL XRXL XR commutes with DXL XR is also diagonalrows of XL are orthogonal to columns of XR

• 233

Diagonalization of non-Hermitian matrices

It follows that any matrix can be diagonalizedaccording to

D = XL A XR = XR1 A XR , or equivalently

D = XL A XL1

The special feature of a Hermitian matrix is that the XL and XR are unitary

• 234

Solving the eigenvalue equation

The problem is entirely equivalent to figuring out how to diagonalize of A according to

A = XR D XR1

All methods proceed by making a series of similarity transformationA = P11 M1 P1 = P11 P21 M2 P2 P1 = = XR D XR1

where M1, M2 are successively more nearly diagonal

• 235

Solving the eigenvalue equation

There are many routines available:EISPACK + Linpack LAPACK (free)NAG, IMSL (expensive)

Before using a totally general technique, consider what you want:eigenvalues alone, or eigenvectors as well?

..and how special your case isreal symmetric & tridiagonal, real symmetric, real non-symmetric, complex Hermitian, complex non-Hermitian

• 236

Solution for a real symmetric matrix: the Householder method (Recipes, 11.2)

The Householder method makes use of a similarity transform based upon

P = I 2 w wT,

where w is a column vector for which wT w = 1 and w wT is an N x N symmetric matrixi.e. (w wT)ij = (w wT)ji = wi wj

• 237

The Householder method

The Householder matrix, P, clearly has the following properties

P is symmetric (being the difference between 2 symmetric matrices, I and 2wwT)

P is orthogonal:PT P = (I 2 w wT)T (I 2 w wT)

= (I 4 w wT + 4 w wT w wT ) = I= 1

• 238

The Householder method

Let a1 be the first column of A

Let w = a1 |a1| e1| a1 |a1| e1|

where e1 = (1,0,0,0,00)T

• 239

The Householder method

Lets consider the action of P = I 2 w wT upon the first column of AP a1 ={ I 2 (a1 |a1| e1 ) (a1 |a1| e1 )T } a1

|a1 |a1| e1 |2

= a1 2 (a1 |a1| e1 ) (|a1|2 |a1| e1Ta1)|a1|2 2 |a1| e1. a1 + |a1|2

= |a1| e1

P zeroes out all but the first element of a1

• 240

The Householder methodSuppose A is symmetric

A = P A =

M1 = P A P1 = P A PT =

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXXX

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

000000X

• 241

The Householder methodSuppose A is symmetric

A = P A =

M1 = P A P1 = P A PT =

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXXX

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXXX

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

000000X

• 242

The Householder methodNext step, choose new Householder matrix of the form

P = to operate on M1 =

M2 = P M1 P1 =

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

XXXXXX0

000000X

XXXXX00

XXXXX00

XXXXX00

XXXXX00

XXXXX00

00000XX

00000XX

0

0

0

0

0

0

0000001

IN1 2 wN1wN1T

• 243

The Householder methodAfter N Householder transformations, the matrix is tridiagonal

MN = = PNP2P1 A P11P21 PN1

XX00000

XXX0000

0XXX000

00XXX00

000XXX0

0000XXX

00000XX

• 244

The Householder method: computational cost

You might think that each Householder transformation would require O(N3) operations (2 matrix multiplications), so the whole series would cost ~ O(N4) operations

However, we can rewrite the product P A P1 asfollows P A P1 = P A P = (I 2 w wT) A (I 2 w wT)

= (I 2 w wT) (A 2 A w wT) = (A2 w wTA 2 A w wT + 4 w wTAw wT)

• 245

The Householder method: computational cost

Now define an N x 1 vector v = A w

P1A P = (A 2wwTA 2A w wT + 4 w wT A w wT) = (A 2 w vT 2 v wT + 4 w wTv wT )

This calculation involves an N2 operation to compute v, and a series of N2 operations to compute, subtract and add various matrices

The reduction of the matrix to triadiagonal form therefore costs only O(N3) operations, not O(N4)

scalar

• 246

Diagonalization of a symmetric tridiagonal matrix (Recipes, 11.3)

The diagonalization of a symmetric tridiagonalmatrix can be accomplished by a series of ~N 2 x 2 rotations

Consider the rotation

R =

1000000

0100000

0010000

0001000

0000100

00000C-S

00000SC

(rotation through angle = cos1 C = sin1 S inthe x1 - x2 plane)

• 247

Diagonalization of a symmetric tridiagonal matrix

Applying this similarity transform to the triadiagonal matrix, we obtain

RT R =

XX00000

XXX0000

0XXX000

00XXX00

000XXX0

0000Xvu

00000ut

XX00000

XXX0000

0XXX000

00XXX00

000XXX0

0000XX0

000000X

• 248

Diagonalization of a symmetric tridiagonal matrix

To make the off-diagonal elements (RT MN R)12 = (RT MN R)21 zero, we require

tSC + u(C2 S2) vSC = 0

tan = (t v) (t v)2 + 4u2

2u