Eigenproblems II: Computation Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration...

download Eigenproblems II: Computation Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration Eigenproblems

If you can't read please download the document

  • date post

    17-Jun-2020
  • Category

    Documents

  • view

    0
  • download

    0

Embed Size (px)

Transcript of Eigenproblems II: Computation Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration...

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Eigenproblems II: Computation

    CS 205A: Mathematical Methods for Robotics, Vision, and Graphics

    Justin Solomon

    CS 205A: Mathematical Methods Eigenproblems II: Computation 1 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Setup

    A ∈ Rn×n symmetric ~x1, . . . , ~xn ∈ Rn eigenvectors

    |λ1| ≥ |λ2| ≥ · · · ≥ |λn| eigenvalues

    Review (Spectral Theorem): What do we know about the eigenvectors?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 2 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Setup

    A ∈ Rn×n symmetric ~x1, . . . , ~xn ∈ Rn eigenvectors

    |λ1| ≥ |λ2| ≥ · · · ≥ |λn| eigenvalues

    Review (Spectral Theorem): What do we know about the eigenvectors?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 2 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Usual Trick

    ~v ∈ Rn ⇓

    ~v = c1~x1 + · · · + cn~xn

    CS 205A: Mathematical Methods Eigenproblems II: Computation 3 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Observation

    Ak~v =

    λk1

    ( c1~x1 +

    ( λ2 λ1

    )k c1~x2 + · · · +

    ( λn λ1

    )k cn~xn

    )

    CS 205A: Mathematical Methods Eigenproblems II: Computation 4 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    For Large k

    Ak~v ≈ λk1~x1 (assuming |λ2| < |λ1|)

    CS 205A: Mathematical Methods Eigenproblems II: Computation 5 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Power Iteration

    ~vk = A~vk−1

    Question: What if |λ1| > 1?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 6 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Power Iteration

    ~vk = A~vk−1

    Question: What if |λ1| > 1?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 6 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Normalized Power Iteration

    ~wk = A~vk−1

    ~vk = ~wk ‖~wk‖

    Question: Which norm?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 7 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Normalized Power Iteration

    ~wk = A~vk−1

    ~vk = ~wk ‖~wk‖

    Question: Which norm?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 7 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Eigenvalues of Inverse Matrix

    A~v = λ~v =⇒ A−1~v = 1 λ ~v

    Question: What is the largest-magnitude eigenvalue?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 8 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Eigenvalues of Inverse Matrix

    A~v = λ~v =⇒ A−1~v = 1 λ ~v

    Question: What is the largest-magnitude eigenvalue?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 8 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Inverse Iteration

    ~wk = A −1~vk−1

    ~vk = ~wk ‖~wk‖

    Question: How to make faster?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 9 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Inverse Iteration

    ~wk = A −1~vk−1

    ~vk = ~wk ‖~wk‖

    Question: How to make faster?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 9 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Inverse Iteration with LU

    Solve L~yk = ~vk−1

    Solve U ~wk = ~yk

    Normalize ~vk = ~wk ‖~wk‖

    CS 205A: Mathematical Methods Eigenproblems II: Computation 10 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Eigenvalues of Inverse Matrix

    A~v = λ~v =⇒ (A− σI)~v = (λ− σ)~v

    CS 205A: Mathematical Methods Eigenproblems II: Computation 11 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Shifted Inverse Iteration

    To find eigenvalue closest to σ:

    ~vk+1 = (A− σI)−1~vk ‖(A− σI)−1~vk‖

    CS 205A: Mathematical Methods Eigenproblems II: Computation 12 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Heuristic: Convergence Rate

    Ak~v =

    λk1

    ( c1~x1 +

    ( λ2 λ1

    )k c1~x2 + · · · +

    ( λn λ1

    )k cn~xn

    )

    CS 205A: Mathematical Methods Eigenproblems II: Computation 13 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Strategy for Better Convergence

    Find σ with∣∣∣∣λ2 − σλ1 − σ ∣∣∣∣ < ∣∣∣∣λ2λ1

    ∣∣∣∣ CS 205A: Mathematical Methods Eigenproblems II: Computation 14 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Least-Squares Approximation

    If ~v0 is approximately an eigenvector:

    argmin λ ‖A~v0 − λ~v0‖2 =

    ~v>0 A~v0 ‖~v0‖22

    CS 205A: Mathematical Methods Eigenproblems II: Computation 15 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Rayleigh Quotient Iteration

    σk = ~v>k−1A~vk−1

    ‖~vk−1‖22 ~wk = (A− σkI)−1~vk−1

    ~vk = ~wk ‖~wk‖

    Efficiency per iteration vs. number of iterations?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 16 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Rayleigh Quotient Iteration

    σk = ~v>k−1A~vk−1

    ‖~vk−1‖22 ~wk = (A− σkI)−1~vk−1

    ~vk = ~wk ‖~wk‖

    Efficiency per iteration vs. number of iterations?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 16 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Unlikely Failure Mode for Iteration

    What is ~v0?

    What happens when ~v0 · ~x1 = 0?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 17 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Unlikely Failure Mode for Iteration

    What is ~v0?

    What happens when ~v0 · ~x1 = 0?

    CS 205A: Mathematical Methods Eigenproblems II: Computation 17 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Bug or Feature?

    1. Compute ~x0 via power iteration.

    2. Project ~x0 out of ~v0.

    3. Compute ~x1 via power iteration.

    4. Project span {~x0, ~x1} out of ~v0. 5. · · ·

    Assumption: A is symmetric.

    CS 205A: Mathematical Methods Eigenproblems II: Computation 18 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Bug or Feature?

    1. Compute ~x0 via power iteration.

    2. Project ~x0 out of ~v0.

    3. Compute ~x1 via power iteration.

    4. Project span {~x0, ~x1} out of ~v0. 5. · · ·

    Assumption: A is symmetric.

    CS 205A: Mathematical Methods Eigenproblems II: Computation 18 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Avoiding Numerical Drift

    Do power iteration on P>AP where P projects

    out known eigenvectors.

    Deflation Modify A so that power iteration reveals an

    eigenvector you have not yet computed.

    CS 205A: Mathematical Methods Eigenproblems II: Computation 19 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Similarity Transformations

    Similar matrices Two matrices A and B are similar if there exists

    T with B = T−1AT.

    Proposition Similar matrices have the same eigenvalues.

    CS 205A: Mathematical Methods Eigenproblems II: Computation 20 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Similarity Transformations

    Similar matrices Two matrices A and B are similar if there exists

    T with B = T−1AT.

    Proposition Similar matrices have the same eigenvalues.

    CS 205A: Mathematical Methods Eigenproblems II: Computation 20 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Householder Asymmetric Deflation

    H~x1 = ~e1

    =⇒ HAH>~e1 = HAH~e1 by symmetry = HA~x1 since H

    2 = I

    = λ1H~x1

    = λ1~e1

    CS 205A: Mathematical Methods Eigenproblems II: Computation 21 / 27

  • Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

    Householder Asymmetric Deflation

    HAH> =

    ( λ1 ~b

    >

    ~0 B

    ) Similarity transform of A =⇒ same eigenvalues.

    Do power iteration on B.

    CS 205A: Mathematical Methods Eigenproblems II: Computation 22 / 27