Francis’s AlgorithmFrancis’s Algorithm David S. Watkins [email protected] Department of...

172
Francis’s Algorithm David S. Watkins [email protected] Department of Mathematics Washington State University Francis’s Algorithm – p.

Transcript of Francis’s AlgorithmFrancis’s Algorithm David S. Watkins [email protected] Department of...

Page 1: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmDavid S. Watkins

[email protected]

Department of Mathematics

Washington State University

Francis’s Algorithm – p. 1

Page 2: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

Francis’s Algorithm – p. 2

Page 3: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

How to solve?

Francis’s Algorithm – p. 2

Page 4: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

Francis’s Algorithm – p. 2

Page 5: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s Algorithm – p. 2

Page 6: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm,

Francis’s Algorithm – p. 2

Page 7: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm, aka

Francis’s Algorithm – p. 2

Page 8: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm, aka

the implicitly shiftedQR algorithm

Francis’s Algorithm – p. 2

Page 9: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm, aka

the implicitly shiftedQR algorithm

50 years!

Francis’s Algorithm – p. 2

Page 10: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Eigenvalue Problem: Av = λv

How to solve?

lambda = eig(A)

How doeseig do it?

Francis’s algorithm, aka

the implicitly shiftedQR algorithm

50 years!

Top Ten of the century (Dongarra and Sullivan)

Francis’s Algorithm – p. 2

Page 11: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

John Francis

Francis’s Algorithm – p. 3

Page 12: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?

Francis’s Algorithm – p. 4

Page 13: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?born near London in 1934

Francis’s Algorithm – p. 4

Page 14: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

Francis’s Algorithm – p. 4

Page 15: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

Francis’s Algorithm – p. 4

Page 16: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

Francis’s Algorithm – p. 4

Page 17: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

no software

Francis’s Algorithm – p. 4

Page 18: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

no software

experimented with a variety of methods

Francis’s Algorithm – p. 4

Page 19: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

no software

experimented with a variety of methods

invented His algorithm and programmed it

Francis’s Algorithm – p. 4

Page 20: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Who is John Francis?born near London in 1934

employed in late 50’s, Pegasus computer

linear algebra, eigenvalue routines

primitive computer

no software

experimented with a variety of methods

invented His algorithm and programmed it

moved on to other things

Francis’s Algorithm – p. 4

Page 21: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Some History

Francis’s Algorithm – p. 5

Page 22: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s Algorithm – p. 5

Page 23: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

Francis’s Algorithm – p. 5

Page 24: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

A − ρI = QR, RQ + ρI = A

Francis’s Algorithm – p. 5

Page 25: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

A − ρI = QR, RQ + ρI = A repeat!

Francis’s Algorithm – p. 5

Page 26: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

A − ρI = QR, RQ + ρI = A repeat!

Kublanovskaya

Francis’s Algorithm – p. 5

Page 27: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Some HistoryRutishauser (q-d 1954, LR 1958)

Francis’s first paper (QR)

A − ρI = QR, RQ + ρI = A repeat!

Kublanovskaya

. . . but this is not “Francis’s Algorithm”

Francis’s Algorithm – p. 5

Page 28: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithm

Francis’s Algorithm – p. 6

Page 29: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

Francis’s Algorithm – p. 6

Page 30: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

real matrices

Francis’s Algorithm – p. 6

Page 31: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

Francis’s Algorithm – p. 6

Page 32: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

Francis’s Algorithm – p. 6

Page 33: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

Francis’s Algorithm – p. 6

Page 34: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

two steps at once

Francis’s Algorithm – p. 6

Page 35: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

two steps at once

double-shiftQR algorithm

Francis’s Algorithm – p. 6

Page 36: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

two steps at once

double-shiftQR algorithm

radically different from basic QR

Francis’s Algorithm – p. 6

Page 37: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s AlgorithmSecond paper of Francis

real matrices with complex pairs of eigenvalues

complex shifts

want to stay in real arithmetic

two steps at once

double-shiftQR algorithm

radically different from basic QR

Usual justification: Francis’s implicit-Q theorem

Francis’s Algorithm – p. 6

Page 38: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithm

Francis’s Algorithm – p. 7

Page 39: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

Francis’s Algorithm – p. 7

Page 40: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

Francis’s Algorithm – p. 7

Page 41: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI)

Francis’s Algorithm – p. 7

Page 42: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

Francis’s Algorithm – p. 7

Page 43: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1

Francis’s Algorithm – p. 7

Page 44: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1 cheap!

Francis’s Algorithm – p. 7

Page 45: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1 cheap!

Build unitaryQ0 with q1 = αp(A)e1.

Francis’s Algorithm – p. 7

Page 46: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1 cheap!

Build unitaryQ0 with q1 = αp(A)e1.

Perform similarity transformA → Q−1

0AQ0.

Francis’s Algorithm – p. 7

Page 47: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithmupper Hessenberg form

pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)

p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

computep(A)e1 cheap!

Build unitaryQ0 with q1 = αp(A)e1.

Perform similarity transformA → Q−1

0AQ0.

Hessenberg form is disturbed.

Francis’s Algorithm – p. 7

Page 48: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

An Upper Hessenberg Matrix@

@@

@@

@@

@@

@@

@@

Francis’s Algorithm – p. 8

Page 49: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

After the Transformation ( Q−10 AQ0)

@@

@@

@@

@@

@@

Francis’s Algorithm – p. 9

Page 50: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

After the Transformation ( Q−10 AQ0)

@@

@@

@@

@@

@@

Now return the matrix to Hessenberg form.

Francis’s Algorithm – p. 9

Page 51: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Chasing the Bulge@

@@@

@@

@@

@@@

Francis’s Algorithm – p. 10

Page 52: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Chasing the Bulge@

@@

@@

@@

@@

@

Francis’s Algorithm – p. 11

Page 53: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Done@

@@

@@

@@

@@

@@

@@

Francis’s Algorithm – p. 12

Page 54: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Done@

@@

@@

@@

@@

@@

@@

The Francis iteration is complete!

Francis’s Algorithm – p. 12

Page 55: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Summary of Francis Iteration

Francis’s Algorithm – p. 13

Page 56: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Summary of Francis IterationPick some shifts.

Francis’s Algorithm – p. 13

Page 57: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Francis’s Algorithm – p. 13

Page 58: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Build Q0 with first columnq1 = αp(A)e1.

Francis’s Algorithm – p. 13

Page 59: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Build Q0 with first columnq1 = αp(A)e1.

Make a bulge. (A → Q−1

0AQ0)

Francis’s Algorithm – p. 13

Page 60: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Build Q0 with first columnq1 = αp(A)e1.

Make a bulge. (A → Q−1

0AQ0)

Chase the bulge. (return to Hessenberg form)

Francis’s Algorithm – p. 13

Page 61: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Summary of Francis IterationPick some shifts.

Computep(A)e1. (p determined by shifts)

Build Q0 with first columnq1 = αp(A)e1.

Make a bulge. (A → Q−1

0AQ0)

Chase the bulge. (return to Hessenberg form)

A = Q−1AQ

Francis’s Algorithm – p. 13

Page 62: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Quicker Summary

Francis’s Algorithm – p. 14

Page 63: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Quicker SummaryMake a bulge.

Francis’s Algorithm – p. 14

Page 64: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Quicker SummaryMake a bulge.

Chase it.

Francis’s Algorithm – p. 14

Page 65: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Remarks

Francis’s Algorithm – p. 15

Page 66: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

RemarksThis is pretty simple.

Francis’s Algorithm – p. 15

Page 67: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

RemarksThis is pretty simple.

noQR decomposition in sight!

Francis’s Algorithm – p. 15

Page 68: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Francis’s Algorithm – p. 15

Page 69: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Confusion!

Francis’s Algorithm – p. 15

Page 70: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Confusion!

Can we think of another name?

Francis’s Algorithm – p. 15

Page 71: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Confusion!

Can we think of another name?

I’m calling it Francis’s Algorithm.

Francis’s Algorithm – p. 15

Page 72: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

RemarksThis is pretty simple.

noQR decomposition in sight!

Why call it theQR algorithm?

Confusion!

Can we think of another name?

I’m calling it Francis’s Algorithm.

This is not a radical move.

Francis’s Algorithm – p. 15

Page 73: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Question

Francis’s Algorithm – p. 16

Page 74: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

QuestionHow should we view Francis’s algorithm?

Francis’s Algorithm – p. 16

Page 75: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Francis’s Algorithm – p. 16

Page 76: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly?

Francis’s Algorithm – p. 16

Page 77: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

Francis’s Algorithm – p. 16

Page 78: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

. . . and the answer is:

Francis’s Algorithm – p. 16

Page 79: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

. . . and the answer is:Why not?

Francis’s Algorithm – p. 16

Page 80: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

. . . and the answer is:Why not?

This simplifies the presentation.

Francis’s Algorithm – p. 16

Page 81: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

QuestionHow should we view Francis’s algorithm?

Do we have to start with the basicQR algorithm?

Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?

. . . and the answer is:Why not?

This simplifies the presentation.

I’m putting my money where my mouth is.

Francis’s Algorithm – p. 16

Page 82: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Francis’s Algorithm – p. 17

Page 83: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

I’m putting my money where my mouth is . . .

Francis’s Algorithm – p. 17

Page 84: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

I’m putting my money where my mouth is . . .

. . . and saving one entire section!

Francis’s Algorithm – p. 17

Page 85: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Pedagogical Pathway

Francis’s Algorithm – p. 18

Page 86: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Pedagogical Pathwayreduction to Hessenberg form

Francis’s Algorithm – p. 18

Page 87: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Pedagogical Pathwayreduction to Hessenberg form

Francis’s algorithm

Francis’s Algorithm – p. 18

Page 88: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Pedagogical Pathwayreduction to Hessenberg form

Francis’s algorithm

Try it out!

Francis’s Algorithm – p. 18

Page 89: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Pedagogical Pathwayreduction to Hessenberg form

Francis’s algorithm

Try it out!

It works great!

Francis’s Algorithm – p. 18

Page 90: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Pedagogical Pathwayreduction to Hessenberg form

Francis’s algorithm

Try it out!

It works great!

Why does it work?

Francis’s Algorithm – p. 18

Page 91: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Ingredients of Francis’s Algorithm

Francis’s Algorithm – p. 19

Page 92: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Ingredients of Francis’s Algorithmsubspace iteration (power method)

Francis’s Algorithm – p. 19

Page 93: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Francis’s Algorithm – p. 19

Page 94: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Krylov subspaces

Francis’s Algorithm – p. 19

Page 95: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Krylov subspaces(instead of the implicit-Q theorem)

Francis’s Algorithm – p. 19

Page 96: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Krylov subspaces(instead of the implicit-Q theorem)

Krylov subspaces and subspace iteration

Francis’s Algorithm – p. 19

Page 97: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Ingredients of Francis’s Algorithmsubspace iteration (power method)

subspace iterationwith changes of coordinate system

Krylov subspaces(instead of the implicit-Q theorem)

Krylov subspaces and subspace iteration

Krylov subspaces and Hessenberg form

Francis’s Algorithm – p. 19

Page 98: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iteration

Francis’s Algorithm – p. 20

Page 99: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

Francis’s Algorithm – p. 20

Page 100: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

Francis’s Algorithm – p. 20

Page 101: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

Francis’s Algorithm – p. 20

Page 102: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj

Francis’s Algorithm – p. 20

Page 103: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Francis’s Algorithm – p. 20

Page 104: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Substitutep(A) for A

Francis’s Algorithm – p. 20

Page 105: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Substitutep(A) for A (shifts, multiple steps)

Francis’s Algorithm – p. 20

Page 106: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Substitutep(A) for A (shifts, multiple steps)

S, p(A)S, p(A)2S, p(A)3S, . . .

Francis’s Algorithm – p. 20

Page 107: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Power Method, Subspace Iterationv, Av, A2v, A3v, . . .

convergence rate|λ2/λ1 |

S, AS, A2S, A3S, . . .

subspaces of dimensionj (|λj+1/λj |)

Substitutep(A) for A (shifts, multiple steps)

S, p(A)S, p(A)2S, p(A)3S, . . .

convergence rate|p(λj+1)/p(λj) |

Francis’s Algorithm – p. 20

Page 108: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Subspace Iterationwith changes of coordinate system

Francis’s Algorithm – p. 21

Page 109: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

Francis’s Algorithm – p. 21

Page 110: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

Francis’s Algorithm – p. 21

Page 111: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

Francis’s Algorithm – p. 21

Page 112: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

change coordinate system:A = Q−1AQ

Francis’s Algorithm – p. 21

Page 113: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

change coordinate system:A = Q−1AQ

qk → Q−1qk = Q∗qk = ek

Francis’s Algorithm – p. 21

Page 114: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

change coordinate system:A = Q−1AQ

qk → Q−1qk = Q∗qk = ek

span{q1, . . . , qj} → span{e1, . . . , ej}

Francis’s Algorithm – p. 21

Page 115: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Subspace Iterationwith changes of coordinate system

takeS = span{e1, . . . , ej}

p(A)S = span{p(A)e1, . . . , p(A)ej}

= span{q1, . . . , qj} (orthonormal)

build unitaryQ = [q1 · · · qj · · ·]

change coordinate system:A = Q−1AQ

qk → Q−1qk = Q∗qk = ek

span{q1, . . . , qj} → span{e1, . . . , ej}

ready for next iterationFrancis’s Algorithm – p. 21

Page 116: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

This version of subspace iteration . . .

Francis’s Algorithm – p. 22

Page 117: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

This version of subspace iteration . . .

. . . holds the subspace fixed

Francis’s Algorithm – p. 22

Page 118: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

This version of subspace iteration . . .

. . . holds the subspace fixed

while the matrix changes.

Francis’s Algorithm – p. 22

Page 119: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

This version of subspace iteration . . .

. . . holds the subspace fixed

while the matrix changes.

. . . moving toward a matrix under which

span{e1, . . . , ej}

is invariant.

Francis’s Algorithm – p. 22

Page 120: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

This version of subspace iteration . . .

. . . holds the subspace fixed

while the matrix changes.

. . . moving toward a matrix under which

span{e1, . . . , ej}

is invariant.

A →

[

A11 A12

0 A22

]

(A11 is j × j.)

Francis’s Algorithm – p. 22

Page 121: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration(first pass)

Francis’s Algorithm – p. 23

Page 122: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

Francis’s Algorithm – p. 23

Page 123: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method

Francis’s Algorithm – p. 23

Page 124: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method+ change of coordinates

Francis’s Algorithm – p. 23

Page 125: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method+ change of coordinates

q1 → Q−1q1 = e1

Francis’s Algorithm – p. 23

Page 126: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method+ change of coordinates

q1 → Q−1q1 = e1

casej = 1 of subspace iteration with a change ofcoordinate system

Francis’s Algorithm – p. 23

Page 127: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration(first pass)

A = Q−1AQ where q1 = αp(A)e1.

power method+ change of coordinates

q1 → Q−1q1 = e1

casej = 1 of subspace iteration with a change ofcoordinate system

. . . but this is just a small part of the story.

Francis’s Algorithm – p. 23

Page 128: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .

Francis’s Algorithm – p. 24

Page 129: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Subspace Iteration

Francis’s Algorithm – p. 24

Page 130: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

Francis’s Algorithm – p. 24

Page 131: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Francis’s Algorithm – p. 24

Page 132: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Kj(A, q) are “determined byq”.

Francis’s Algorithm – p. 24

Page 133: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Kj(A, q) are “determined byq”.

p(A)Kj(A, q) = Kj(A, p(A)q)

Francis’s Algorithm – p. 24

Page 134: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Kj(A, q) are “determined byq”.

p(A)Kj(A, q) = Kj(A, p(A)q)

. . . becausep(A)A = Ap(A)

Francis’s Algorithm – p. 24

Page 135: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span

{

q, Aq,A2q, . . . , Aj−1q}

j = 1, 2, 3, . . . (nested subspaces)

Kj(A, q) are “determined byq”.

p(A)Kj(A, q) = Kj(A, p(A)q)

. . . becausep(A)A = Ap(A)

Conclusion: Power method induces nested subspaceiterations on Krylov subspaces.

Francis’s Algorithm – p. 24

Page 136: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

power method: q → p(A)kq

Francis’s Algorithm – p. 25

Page 137: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

power method: q → p(A)kq

nested subspace iterations:

p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .

Francis’s Algorithm – p. 25

Page 138: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

power method: q → p(A)kq

nested subspace iterations:

p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .

convergence rates:

|p(λj+1)/p(λj) |, j = 1, 2, 3, . . . , n − 1

Francis’s Algorithm – p. 25

Page 139: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .

Francis’s Algorithm – p. 26

Page 140: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Hessenberg matrices . . .

Francis’s Algorithm – p. 26

Page 141: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Hessenberg matrices . . .

. . . go hand in hand.

Francis’s Algorithm – p. 26

Page 142: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Hessenberg matrices . . .

. . . go hand in hand.

A properly upper Hessenberg=⇒

Kj(A, e1) = span{e1, . . . , ej}.

Francis’s Algorithm – p. 26

Page 143: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov Subspaces . . .. . . and Hessenberg matrices . . .

. . . go hand in hand.

A properly upper Hessenberg=⇒

Kj(A, e1) = span{e1, . . . , ej}.

More generally . . .

Francis’s Algorithm – p. 26

Page 144: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov-Hessenberg Relationship

Francis’s Algorithm – p. 27

Page 145: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov-Hessenberg Relationship

If A = Q−1AQ,

Francis’s Algorithm – p. 27

Page 146: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov-Hessenberg Relationship

If A = Q−1AQ,

andA is properly upper Hessenberg,

Francis’s Algorithm – p. 27

Page 147: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov-Hessenberg Relationship

If A = Q−1AQ,

andA is properly upper Hessenberg,

then forj = 1, 2, 3, . . . ,

Francis’s Algorithm – p. 27

Page 148: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Krylov-Hessenberg Relationship

If A = Q−1AQ,

andA is properly upper Hessenberg,

then forj = 1, 2, 3, . . . ,

span{q1, . . . , qj} = Kj(A, q1).

Francis’s Algorithm – p. 27

Page 149: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

Francis’s Algorithm – p. 28

Page 150: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

Francis’s Algorithm – p. 28

Page 151: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.

Francis’s Algorithm – p. 28

Page 152: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

Francis’s Algorithm – p. 28

Page 153: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

Francis’s Algorithm – p. 28

Page 154: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

Francis’s Algorithm – p. 28

Page 155: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

subspace iteration with a change of coordinatesystem

Francis’s Algorithm – p. 28

Page 156: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

subspace iteration with a change of coordinatesystem forj = 1, 2, 3, . . . ,n − 1

Francis’s Algorithm – p. 28

Page 157: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Application to Francis’s Iteration

A = Q−1AQ where q1 = αp(A)e1.

power method with a change of coordinate system.Moreover . . .

p(A)Kj(A, e1) = Kj(A, p(A)e1)

i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

subspace iteration with a change of coordinatesystem forj = 1, 2, 3, . . . ,n − 1

|p(λj+1)/p(λj) | j = 1, 2, 3, . . . ,n − 1

Francis’s Algorithm – p. 28

Page 158: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Details

Francis’s Algorithm – p. 29

Page 159: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Detailschoice of shifts

Francis’s Algorithm – p. 29

Page 160: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Detailschoice of shifts

We change the shifts at each step.

Francis’s Algorithm – p. 29

Page 161: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Detailschoice of shifts

We change the shifts at each step.

⇒ quadratic or cubic convergence

Francis’s Algorithm – p. 29

Page 162: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Detailschoice of shifts

We change the shifts at each step.

⇒ quadratic or cubic convergence

Watkins (2007, 2010)

Francis’s Algorithm – p. 29

Page 163: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Detailschoice of shifts

We change the shifts at each step.

⇒ quadratic or cubic convergence

Watkins (2007, 2010)

Francis’s Algorithm – p. 29

Page 164: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Where is John Francis?

Francis’s Algorithm – p. 30

Page 165: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Where is John Francis?question asked frequently by Gene Golub

Francis’s Algorithm – p. 30

Page 166: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Where is John Francis?question asked frequently by Gene Golub

inquiries by Golub and Uhlig

Francis’s Algorithm – p. 30

Page 167: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Where is John Francis?question asked frequently by Gene Golub

inquiries by Golub and Uhlig

Francis is alive and well,retired in the South of England.

Francis’s Algorithm – p. 30

Page 168: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Where is John Francis?question asked frequently by Gene Golub

inquiries by Golub and Uhlig

Francis is alive and well,retired in the South of England.

was unaware of the impact of his algorithm

Francis’s Algorithm – p. 30

Page 169: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Where is John Francis?question asked frequently by Gene Golub

inquiries by Golub and Uhlig

Francis is alive and well,retired in the South of England.

was unaware of the impact of his algorithm

appearance at the Biennial Numerical AnalysisConference in Glasgow in June of 2009

Francis’s Algorithm – p. 30

Page 170: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

John Francis speaking in Glasgow

Francis’s Algorithm – p. 31

Page 171: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

A Portion of the Audience

Francis’s Algorithm – p. 32

Page 172: Francis’s AlgorithmFrancis’s Algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Francis’s Algorithm – p. 1Published in: American

Afterwards

Photos courtesy of Frank UhligFrancis’s Algorithm – p. 33