• date post

17-Jun-2020
• Category

## Documents

• view

1

0

Embed Size (px)

### Transcript of Eigenproblems II: Computation · PDF file Power Iteration Other Eigenvalues Multiple...

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Eigenproblems II: Computation

CS 205A: Mathematical Methods for Robotics, Vision, and Graphics

Justin Solomon

CS 205A: Mathematical Methods Eigenproblems II: Computation 1 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Setup

A ∈ Rn×n symmetric ~x1, . . . , ~xn ∈ Rn eigenvectors

|λ1| ≥ |λ2| ≥ · · · ≥ |λn| eigenvalues

Review (Spectral Theorem): What do we know about the eigenvectors?

CS 205A: Mathematical Methods Eigenproblems II: Computation 2 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Setup

A ∈ Rn×n symmetric ~x1, . . . , ~xn ∈ Rn eigenvectors

|λ1| ≥ |λ2| ≥ · · · ≥ |λn| eigenvalues

Review (Spectral Theorem): What do we know about the eigenvectors?

CS 205A: Mathematical Methods Eigenproblems II: Computation 2 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Usual Trick

~v ∈ Rn ⇓

~v = c1~x1 + · · · + cn~xn

CS 205A: Mathematical Methods Eigenproblems II: Computation 3 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Observation

Ak~v =

λk1

( c1~x1 +

( λ2 λ1

)k c1~x2 + · · · +

( λn λ1

)k cn~xn

)

CS 205A: Mathematical Methods Eigenproblems II: Computation 4 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

For Large k

Ak~v ≈ λk1~x1 (assuming |λ2| < |λ1|)

CS 205A: Mathematical Methods Eigenproblems II: Computation 5 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Power Iteration

~vk = A~vk−1

Question: What if |λ1| > 1?

CS 205A: Mathematical Methods Eigenproblems II: Computation 6 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Power Iteration

~vk = A~vk−1

Question: What if |λ1| > 1?

CS 205A: Mathematical Methods Eigenproblems II: Computation 6 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Normalized Power Iteration

~wk = A~vk−1

~vk = ~wk ‖~wk‖

Question: Which norm?

CS 205A: Mathematical Methods Eigenproblems II: Computation 7 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Normalized Power Iteration

~wk = A~vk−1

~vk = ~wk ‖~wk‖

Question: Which norm?

CS 205A: Mathematical Methods Eigenproblems II: Computation 7 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Eigenvalues of Inverse Matrix

A~v = λ~v =⇒ A−1~v = 1 λ ~v

Question: What is the largest-magnitude eigenvalue?

CS 205A: Mathematical Methods Eigenproblems II: Computation 8 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Eigenvalues of Inverse Matrix

A~v = λ~v =⇒ A−1~v = 1 λ ~v

Question: What is the largest-magnitude eigenvalue?

CS 205A: Mathematical Methods Eigenproblems II: Computation 8 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Inverse Iteration

~wk = A −1~vk−1

~vk = ~wk ‖~wk‖

Question: How to make faster?

CS 205A: Mathematical Methods Eigenproblems II: Computation 9 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Inverse Iteration

~wk = A −1~vk−1

~vk = ~wk ‖~wk‖

Question: How to make faster?

CS 205A: Mathematical Methods Eigenproblems II: Computation 9 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Inverse Iteration with LU

Solve L~yk = ~vk−1

Solve U ~wk = ~yk

Normalize ~vk = ~wk ‖~wk‖

CS 205A: Mathematical Methods Eigenproblems II: Computation 10 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Eigenvalues of Inverse Matrix

A~v = λ~v =⇒ (A− σI)~v = (λ− σ)~v

CS 205A: Mathematical Methods Eigenproblems II: Computation 11 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Shifted Inverse Iteration

To find eigenvalue closest to σ:

~vk+1 = (A− σI)−1~vk ‖(A− σI)−1~vk‖

CS 205A: Mathematical Methods Eigenproblems II: Computation 12 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Heuristic: Convergence Rate

Ak~v =

λk1

( c1~x1 +

( λ2 λ1

)k c1~x2 + · · · +

( λn λ1

)k cn~xn

)

CS 205A: Mathematical Methods Eigenproblems II: Computation 13 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Strategy for Better Convergence

Find σ with∣∣∣∣λ2 − σλ1 − σ ∣∣∣∣ < ∣∣∣∣λ2λ1

∣∣∣∣ CS 205A: Mathematical Methods Eigenproblems II: Computation 14 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Least-Squares Approximation

If ~v0 is approximately an eigenvector:

argmin λ ‖A~v0 − λ~v0‖2 =

~v>0 A~v0 ‖~v0‖22

CS 205A: Mathematical Methods Eigenproblems II: Computation 15 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Rayleigh Quotient Iteration

σk = ~v>k−1A~vk−1

‖~vk−1‖22 ~wk = (A− σkI)−1~vk−1

~vk = ~wk ‖~wk‖

Efficiency per iteration vs. number of iterations?

CS 205A: Mathematical Methods Eigenproblems II: Computation 16 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Rayleigh Quotient Iteration

σk = ~v>k−1A~vk−1

‖~vk−1‖22 ~wk = (A− σkI)−1~vk−1

~vk = ~wk ‖~wk‖

Efficiency per iteration vs. number of iterations?

CS 205A: Mathematical Methods Eigenproblems II: Computation 16 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Unlikely Failure Mode for Iteration

What is ~v0?

What happens when ~v0 · ~x1 = 0?

CS 205A: Mathematical Methods Eigenproblems II: Computation 17 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Unlikely Failure Mode for Iteration

What is ~v0?

What happens when ~v0 · ~x1 = 0?

CS 205A: Mathematical Methods Eigenproblems II: Computation 17 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Bug or Feature?

1. Compute ~x0 via power iteration.

2. Project ~x0 out of ~v0.

3. Compute ~x1 via power iteration.

4. Project span {~x0, ~x1} out of ~v0. 5. · · ·

Assumption: A is symmetric.

CS 205A: Mathematical Methods Eigenproblems II: Computation 18 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Bug or Feature?

1. Compute ~x0 via power iteration.

2. Project ~x0 out of ~v0.

3. Compute ~x1 via power iteration.

4. Project span {~x0, ~x1} out of ~v0. 5. · · ·

Assumption: A is symmetric.

CS 205A: Mathematical Methods Eigenproblems II: Computation 18 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Avoiding Numerical Drift

Do power iteration on P>AP where P projects

out known eigenvectors.

Deflation Modify A so that power iteration reveals an

eigenvector you have not yet computed.

CS 205A: Mathematical Methods Eigenproblems II: Computation 19 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Similarity Transformations

Similar matrices Two matrices A and B are similar if there exists

T with B = T−1AT.

Proposition Similar matrices have the same eigenvalues.

CS 205A: Mathematical Methods Eigenproblems II: Computation 20 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Similarity Transformations

Similar matrices Two matrices A and B are similar if there exists

T with B = T−1AT.

Proposition Similar matrices have the same eigenvalues.

CS 205A: Mathematical Methods Eigenproblems II: Computation 20 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Householder Asymmetric Deflation

H~x1 = ~e1

=⇒ HAH>~e1 = HAH~e1 by symmetry = HA~x1 since H

2 = I

= λ1H~x1

= λ1~e1

CS 205A: Mathematical Methods Eigenproblems II: Computation 21 / 27

• Power Iteration Other Eigenvalues Multiple Eigenvalues QR Iteration

Householder Asymmetric Deflation

HAH> =

( λ1 ~b

>

~0 B

) Similarity transform of A =⇒ same eigenvalues.

Do power iteration on B.

CS 205A: Mathematical Methods Eigenproblems II: Computation 22 / 27