Lecture 10: Eigenvectors and eigenvalues - pha.jhu. neufeld/numerical/ 10: Eigenvectors and...

download Lecture 10: Eigenvectors and eigenvalues - pha.jhu. neufeld/numerical/ 10: Eigenvectors and eigenvalues (Numerical Recipes, Chapter 11) ... of no practical value (except, perhaps in

of 29

  • date post

    19-Mar-2018
  • Category

    Documents

  • view

    218
  • download

    4

Embed Size (px)

Transcript of Lecture 10: Eigenvectors and eigenvalues - pha.jhu. neufeld/numerical/ 10: Eigenvectors and...

  • 220

    Lecture 10:Eigenvectors and eigenvalues

    (Numerical Recipes, Chapter 11)

    The eigenvalue problem,

    A x = x,

    occurs in many, many contexts:

    classical mechanics, quantum mechanics, optics

  • 221

    Eigenvectors and eigenvalues(Numerical Recipes, Chapter 11)

    The textbook solution is obtained from

    (A I) x = 0, which can only hold for x 0 if

    det (A I) = 0.

    This is a polynomial equation for of order N

  • 222

    Eigenvectors and eigenvalues(Numerical Recipes, Chapter 11)

    This method of solution is:

    (1) very slow(2) inaccurate, in the presence of roundoff error(3) yields only eigenvalues, not eigenvectors

    (4) of no practical value (except, perhaps in the 2 x 2 case where the solution to the resultant quadratic can be written in closed form)

  • 223

    Review of some basic definitions in linear algebra

    A matrix is said to be:

    Symmetric if A = AT (its transpose) i.e. if aij = aji

    Hermitian if A = A (AT)* i.e. if aij = aji*(its Hermitian conjugate)

    Orthonormal if U1 = UT

    Unitary if U1 = U

  • 224

    Review of some basic definitions in linear algebra

    All 4 types are normal, meaning that they obey the relations

    A A = A AU U = U U

  • 225

    Eigenvectors and eigenvalues

    Hermitian matrices are of particular interest because they have:

    real eigenvaluesorthonormal eigenvectors (xi xj = ij )(unless some eigenvalues are degenerate, in which case we can always design an orthonormalset)

  • 226

    Diagonalization of Hermitian matrices

    Hermitian matrices can be diagonalizedaccording to

    A = U D U1

    Diagonal matrix consisting of the eigenvalues

    Unitary matrix whose column are the eigenvectors

  • 227

    Diagonalization of Hermitian matrices

    Clearly, if A = U D U1, then A U = U D..which is simply the eigenproblem(written out N times):

    The columns of U are the eigenvectors (mutually orthogonal) and the elements of D (non-zero only on the diagonal) are the corresponding eigenvalues.

  • 228

    Non-Hermitian matrices

    We are often interested in non-symmetric real matrices:

    the eigenvalues are real or come in complex-conjugate pairs

    the eigenvectors are not orthonormal in general

    the eigenvectors may not even span an N-dimensional space

  • 229

    Diagonalization of non-Hermitian matrices

    For a non-Hermitian matrix, we can identify two (different) types of eigenvectors

    Right hand eigenvectors are column vectors which obey:A xR = xR

    Left hand eigenvectors are row vectors which obey:xL A = xL

    The eigenvalues are the same (the roots of det (A I) = 0 in both cases) but, in general, the eigenvectors are not

  • 230

    Diagonalization of non-Hermitian matrices

    Note: for a Hermitian matrix, the two types of eigenvectors are equivalent, since

    A xR = xR since A = A since is real

    xR A = xR xR A = xR

    so if any vector xR is a right-hand eigenvector, then xR is a left-hand eigenvector

  • 231

    Diagonalization of non-Hermitian matrices

    Let D be the diagonal matrix whose elements are the eigenvalues, D = diag (i)

    Let XR be the matrix whose columns are the right-hand eigenvectors

    Let XL be the matrix whose rows are the left-hand eigenvectors

    Then the eigenvalue equations are

    A XR = XR D and XL A = D XL

  • 232

    Diagonalization of non-Hermitian matrices

    The eigenvalue equations

    A XR = XR D and XL A = D XL

    imply XLA XR = XL XR D and XL A XR = D XL XR

    Hence, XL A XR = XL XR D = D XL XRXL XR commutes with DXL XR is also diagonalrows of XL are orthogonal to columns of XR

  • 233

    Diagonalization of non-Hermitian matrices

    It follows that any matrix can be diagonalizedaccording to

    D = XL A XR = XR1 A XR , or equivalently

    D = XL A XL1

    The special feature of a Hermitian matrix is that the XL and XR are unitary

  • 234

    Solving the eigenvalue equation

    The problem is entirely equivalent to figuring out how to diagonalize of A according to

    A = XR D XR1

    All methods proceed by making a series of similarity transformationA = P11 M1 P1 = P11 P21 M2 P2 P1 = = XR D XR1

    where M1, M2 are successively more nearly diagonal

  • 235

    Solving the eigenvalue equation

    There are many routines available:EISPACK + Linpack LAPACK (free)NAG, IMSL (expensive)

    Before using a totally general technique, consider what you want:eigenvalues alone, or eigenvectors as well?

    ..and how special your case isreal symmetric & tridiagonal, real symmetric, real non-symmetric, complex Hermitian, complex non-Hermitian

  • 236

    Solution for a real symmetric matrix: the Householder method (Recipes, 11.2)

    The Householder method makes use of a similarity transform based upon

    P = I 2 w wT,

    where w is a column vector for which wT w = 1 and w wT is an N x N symmetric matrixi.e. (w wT)ij = (w wT)ji = wi wj

  • 237

    The Householder method

    The Householder matrix, P, clearly has the following properties

    P is symmetric (being the difference between 2 symmetric matrices, I and 2wwT)

    P is orthogonal:PT P = (I 2 w wT)T (I 2 w wT)

    = (I 4 w wT + 4 w wT w wT ) = I= 1

  • 238

    The Householder method

    Let a1 be the first column of A

    Let w = a1 |a1| e1| a1 |a1| e1|

    where e1 = (1,0,0,0,00)T

  • 239

    The Householder method

    Lets consider the action of P = I 2 w wT upon the first column of AP a1 ={ I 2 (a1 |a1| e1 ) (a1 |a1| e1 )T } a1

    |a1 |a1| e1 |2

    = a1 2 (a1 |a1| e1 ) (|a1|2 |a1| e1Ta1)|a1|2 2 |a1| e1. a1 + |a1|2

    = |a1| e1

    P zeroes out all but the first element of a1

  • 240

    The Householder methodSuppose A is symmetric

    A = P A =

    M1 = P A P1 = P A PT =

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXXX

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    000000X

  • 241

    The Householder methodSuppose A is symmetric

    A = P A =

    M1 = P A P1 = P A PT =

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXXX

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXXX

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    000000X

  • 242

    The Householder methodNext step, choose new Householder matrix of the form

    P = to operate on M1 =

    M2 = P M1 P1 =

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    XXXXXX0

    000000X

    XXXXX00

    XXXXX00

    XXXXX00

    XXXXX00

    XXXXX00

    00000XX

    00000XX

    0

    0

    0

    0

    0

    0

    0000001

    IN1 2 wN1wN1T

  • 243

    The Householder methodAfter N Householder transformations, the matrix is tridiagonal

    MN = = PNP2P1 A P11P21 PN1

    XX00000

    XXX0000

    0XXX000

    00XXX00

    000XXX0

    0000XXX

    00000XX

  • 244

    The Householder method: computational cost

    You might think that each Householder transformation would require O(N3) operations (2 matrix multiplications), so the whole series would cost ~ O(N4) operations

    However, we can rewrite the product P A P1 asfollows P A P1 = P A P = (I 2 w wT) A (I 2 w wT)

    = (I 2 w wT) (A 2 A w wT) = (A2 w wTA 2 A w wT + 4 w wTAw wT)

  • 245

    The Householder method: computational cost

    Now define an N x 1 vector v = A w

    P1A P = (A 2wwTA 2A w wT + 4 w wT A w wT) = (A 2 w vT 2 v wT + 4 w wTv wT )

    This calculation involves an N2 operation to compute v, and a series of N2 operations to compute, subtract and add various matrices

    The reduction of the matrix to triadiagonal form therefore costs only O(N3) operations, not O(N4)

    scalar

  • 246

    Diagonalization of a symmetric tridiagonal matrix (Recipes, 11.3)

    The diagonalization of a symmetric tridiagonalmatrix can be accomplished by a series of ~N 2 x 2 rotations

    Consider the rotation

    R =

    1000000

    0100000

    0010000

    0001000

    0000100

    00000C-S

    00000SC

    (rotation through angle = cos1 C = sin1 S inthe x1 - x2 plane)

  • 247

    Diagonalization of a symmetric tridiagonal matrix

    Applying this similarity transform to the triadiagonal matrix, we obtain

    RT R =

    XX00000

    XXX0000

    0XXX000

    00XXX00

    000XXX0

    0000Xvu

    00000ut

    XX00000

    XXX0000

    0XXX000

    00XXX00

    000XXX0

    0000XX0

    000000X

  • 248

    Diagonalization of a symmetric tridiagonal matrix

    To make the off-diagonal elements (RT MN R)12 = (RT MN R)21 zero, we require

    tSC + u(C2 S2) vSC = 0

    tan = (t v) (t v)2 + 4u2

    2u