Francis’s AlgorithmFrancis’s Algorithm David S. Watkins [email protected] Department of...
Transcript of Francis’s AlgorithmFrancis’s Algorithm David S. Watkins [email protected] Department of...
Francis’s AlgorithmDavid S. Watkins
Department of Mathematics
Washington State University
Francis’s Algorithm – p. 1
Eigenvalue Problem: Av = λv
Francis’s Algorithm – p. 2
Eigenvalue Problem: Av = λv
How to solve?
Francis’s Algorithm – p. 2
Eigenvalue Problem: Av = λv
How to solve?
lambda = eig(A)
Francis’s Algorithm – p. 2
Eigenvalue Problem: Av = λv
How to solve?
lambda = eig(A)
How doeseig do it?
Francis’s Algorithm – p. 2
Eigenvalue Problem: Av = λv
How to solve?
lambda = eig(A)
How doeseig do it?
Francis’s algorithm,
Francis’s Algorithm – p. 2
Eigenvalue Problem: Av = λv
How to solve?
lambda = eig(A)
How doeseig do it?
Francis’s algorithm, aka
Francis’s Algorithm – p. 2
Eigenvalue Problem: Av = λv
How to solve?
lambda = eig(A)
How doeseig do it?
Francis’s algorithm, aka
the implicitly shiftedQR algorithm
Francis’s Algorithm – p. 2
Eigenvalue Problem: Av = λv
How to solve?
lambda = eig(A)
How doeseig do it?
Francis’s algorithm, aka
the implicitly shiftedQR algorithm
50 years!
Francis’s Algorithm – p. 2
Eigenvalue Problem: Av = λv
How to solve?
lambda = eig(A)
How doeseig do it?
Francis’s algorithm, aka
the implicitly shiftedQR algorithm
50 years!
Top Ten of the century (Dongarra and Sullivan)
Francis’s Algorithm – p. 2
John Francis
Francis’s Algorithm – p. 3
Who is John Francis?
Francis’s Algorithm – p. 4
Who is John Francis?born near London in 1934
Francis’s Algorithm – p. 4
Who is John Francis?born near London in 1934
employed in late 50’s, Pegasus computer
Francis’s Algorithm – p. 4
Who is John Francis?born near London in 1934
employed in late 50’s, Pegasus computer
linear algebra, eigenvalue routines
Francis’s Algorithm – p. 4
Who is John Francis?born near London in 1934
employed in late 50’s, Pegasus computer
linear algebra, eigenvalue routines
primitive computer
Francis’s Algorithm – p. 4
Who is John Francis?born near London in 1934
employed in late 50’s, Pegasus computer
linear algebra, eigenvalue routines
primitive computer
no software
Francis’s Algorithm – p. 4
Who is John Francis?born near London in 1934
employed in late 50’s, Pegasus computer
linear algebra, eigenvalue routines
primitive computer
no software
experimented with a variety of methods
Francis’s Algorithm – p. 4
Who is John Francis?born near London in 1934
employed in late 50’s, Pegasus computer
linear algebra, eigenvalue routines
primitive computer
no software
experimented with a variety of methods
invented His algorithm and programmed it
Francis’s Algorithm – p. 4
Who is John Francis?born near London in 1934
employed in late 50’s, Pegasus computer
linear algebra, eigenvalue routines
primitive computer
no software
experimented with a variety of methods
invented His algorithm and programmed it
moved on to other things
Francis’s Algorithm – p. 4
Some History
Francis’s Algorithm – p. 5
Some HistoryRutishauser (q-d 1954, LR 1958)
Francis’s Algorithm – p. 5
Some HistoryRutishauser (q-d 1954, LR 1958)
Francis’s first paper (QR)
Francis’s Algorithm – p. 5
Some HistoryRutishauser (q-d 1954, LR 1958)
Francis’s first paper (QR)
A − ρI = QR, RQ + ρI = A
Francis’s Algorithm – p. 5
Some HistoryRutishauser (q-d 1954, LR 1958)
Francis’s first paper (QR)
A − ρI = QR, RQ + ρI = A repeat!
Francis’s Algorithm – p. 5
Some HistoryRutishauser (q-d 1954, LR 1958)
Francis’s first paper (QR)
A − ρI = QR, RQ + ρI = A repeat!
Kublanovskaya
Francis’s Algorithm – p. 5
Some HistoryRutishauser (q-d 1954, LR 1958)
Francis’s first paper (QR)
A − ρI = QR, RQ + ρI = A repeat!
Kublanovskaya
. . . but this is not “Francis’s Algorithm”
Francis’s Algorithm – p. 5
Francis’s Algorithm
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
real matrices
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
real matrices with complex pairs of eigenvalues
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
real matrices with complex pairs of eigenvalues
complex shifts
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
real matrices with complex pairs of eigenvalues
complex shifts
want to stay in real arithmetic
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
real matrices with complex pairs of eigenvalues
complex shifts
want to stay in real arithmetic
two steps at once
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
real matrices with complex pairs of eigenvalues
complex shifts
want to stay in real arithmetic
two steps at once
double-shiftQR algorithm
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
real matrices with complex pairs of eigenvalues
complex shifts
want to stay in real arithmetic
two steps at once
double-shiftQR algorithm
radically different from basic QR
Francis’s Algorithm – p. 6
Francis’s AlgorithmSecond paper of Francis
real matrices with complex pairs of eigenvalues
complex shifts
want to stay in real arithmetic
two steps at once
double-shiftQR algorithm
radically different from basic QR
Usual justification: Francis’s implicit-Q theorem
Francis’s Algorithm – p. 6
Francis’s Algorithm
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI)
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1 cheap!
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1 cheap!
Build unitaryQ0 with q1 = αp(A)e1.
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1 cheap!
Build unitaryQ0 with q1 = αp(A)e1.
Perform similarity transformA → Q−1
0AQ0.
Francis’s Algorithm – p. 7
Francis’s Algorithmupper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1 cheap!
Build unitaryQ0 with q1 = αp(A)e1.
Perform similarity transformA → Q−1
0AQ0.
Hessenberg form is disturbed.
Francis’s Algorithm – p. 7
An Upper Hessenberg Matrix@
@@
@@
@@
@@
@@
@@
Francis’s Algorithm – p. 8
After the Transformation ( Q−10 AQ0)
@@
@@
@@
@@
@@
Francis’s Algorithm – p. 9
After the Transformation ( Q−10 AQ0)
@@
@@
@@
@@
@@
Now return the matrix to Hessenberg form.
Francis’s Algorithm – p. 9
Chasing the Bulge@
@@@
@@
@@
@@@
Francis’s Algorithm – p. 10
Chasing the Bulge@
@@
@@
@@
@@
@
Francis’s Algorithm – p. 11
Done@
@@
@@
@@
@@
@@
@@
Francis’s Algorithm – p. 12
Done@
@@
@@
@@
@@
@@
@@
The Francis iteration is complete!
Francis’s Algorithm – p. 12
Summary of Francis Iteration
Francis’s Algorithm – p. 13
Summary of Francis IterationPick some shifts.
Francis’s Algorithm – p. 13
Summary of Francis IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Francis’s Algorithm – p. 13
Summary of Francis IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Build Q0 with first columnq1 = αp(A)e1.
Francis’s Algorithm – p. 13
Summary of Francis IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Build Q0 with first columnq1 = αp(A)e1.
Make a bulge. (A → Q−1
0AQ0)
Francis’s Algorithm – p. 13
Summary of Francis IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Build Q0 with first columnq1 = αp(A)e1.
Make a bulge. (A → Q−1
0AQ0)
Chase the bulge. (return to Hessenberg form)
Francis’s Algorithm – p. 13
Summary of Francis IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Build Q0 with first columnq1 = αp(A)e1.
Make a bulge. (A → Q−1
0AQ0)
Chase the bulge. (return to Hessenberg form)
A = Q−1AQ
Francis’s Algorithm – p. 13
Quicker Summary
Francis’s Algorithm – p. 14
Quicker SummaryMake a bulge.
Francis’s Algorithm – p. 14
Quicker SummaryMake a bulge.
Chase it.
Francis’s Algorithm – p. 14
Remarks
Francis’s Algorithm – p. 15
RemarksThis is pretty simple.
Francis’s Algorithm – p. 15
RemarksThis is pretty simple.
noQR decomposition in sight!
Francis’s Algorithm – p. 15
RemarksThis is pretty simple.
noQR decomposition in sight!
Why call it theQR algorithm?
Francis’s Algorithm – p. 15
RemarksThis is pretty simple.
noQR decomposition in sight!
Why call it theQR algorithm?
Confusion!
Francis’s Algorithm – p. 15
RemarksThis is pretty simple.
noQR decomposition in sight!
Why call it theQR algorithm?
Confusion!
Can we think of another name?
Francis’s Algorithm – p. 15
RemarksThis is pretty simple.
noQR decomposition in sight!
Why call it theQR algorithm?
Confusion!
Can we think of another name?
I’m calling it Francis’s Algorithm.
Francis’s Algorithm – p. 15
RemarksThis is pretty simple.
noQR decomposition in sight!
Why call it theQR algorithm?
Confusion!
Can we think of another name?
I’m calling it Francis’s Algorithm.
This is not a radical move.
Francis’s Algorithm – p. 15
Question
Francis’s Algorithm – p. 16
QuestionHow should we view Francis’s algorithm?
Francis’s Algorithm – p. 16
QuestionHow should we view Francis’s algorithm?
Do we have to start with the basicQR algorithm?
Francis’s Algorithm – p. 16
QuestionHow should we view Francis’s algorithm?
Do we have to start with the basicQR algorithm?
Couldn’t we just as well introduce Francis’salgorithm directly?
Francis’s Algorithm – p. 16
QuestionHow should we view Francis’s algorithm?
Do we have to start with the basicQR algorithm?
Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?
Francis’s Algorithm – p. 16
QuestionHow should we view Francis’s algorithm?
Do we have to start with the basicQR algorithm?
Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?
. . . and the answer is:
Francis’s Algorithm – p. 16
QuestionHow should we view Francis’s algorithm?
Do we have to start with the basicQR algorithm?
Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?
. . . and the answer is:Why not?
Francis’s Algorithm – p. 16
QuestionHow should we view Francis’s algorithm?
Do we have to start with the basicQR algorithm?
Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?
. . . and the answer is:Why not?
This simplifies the presentation.
Francis’s Algorithm – p. 16
QuestionHow should we view Francis’s algorithm?
Do we have to start with the basicQR algorithm?
Couldn’t we just as well introduce Francis’salgorithm directly? . . . bypassing the basicQRalgorithm entirely?
. . . and the answer is:Why not?
This simplifies the presentation.
I’m putting my money where my mouth is.
Francis’s Algorithm – p. 16
Francis’s Algorithm – p. 17
I’m putting my money where my mouth is . . .
Francis’s Algorithm – p. 17
I’m putting my money where my mouth is . . .
. . . and saving one entire section!
Francis’s Algorithm – p. 17
Pedagogical Pathway
Francis’s Algorithm – p. 18
Pedagogical Pathwayreduction to Hessenberg form
Francis’s Algorithm – p. 18
Pedagogical Pathwayreduction to Hessenberg form
Francis’s algorithm
Francis’s Algorithm – p. 18
Pedagogical Pathwayreduction to Hessenberg form
Francis’s algorithm
Try it out!
Francis’s Algorithm – p. 18
Pedagogical Pathwayreduction to Hessenberg form
Francis’s algorithm
Try it out!
It works great!
Francis’s Algorithm – p. 18
Pedagogical Pathwayreduction to Hessenberg form
Francis’s algorithm
Try it out!
It works great!
Why does it work?
Francis’s Algorithm – p. 18
Ingredients of Francis’s Algorithm
Francis’s Algorithm – p. 19
Ingredients of Francis’s Algorithmsubspace iteration (power method)
Francis’s Algorithm – p. 19
Ingredients of Francis’s Algorithmsubspace iteration (power method)
subspace iterationwith changes of coordinate system
Francis’s Algorithm – p. 19
Ingredients of Francis’s Algorithmsubspace iteration (power method)
subspace iterationwith changes of coordinate system
Krylov subspaces
Francis’s Algorithm – p. 19
Ingredients of Francis’s Algorithmsubspace iteration (power method)
subspace iterationwith changes of coordinate system
Krylov subspaces(instead of the implicit-Q theorem)
Francis’s Algorithm – p. 19
Ingredients of Francis’s Algorithmsubspace iteration (power method)
subspace iterationwith changes of coordinate system
Krylov subspaces(instead of the implicit-Q theorem)
Krylov subspaces and subspace iteration
Francis’s Algorithm – p. 19
Ingredients of Francis’s Algorithmsubspace iteration (power method)
subspace iterationwith changes of coordinate system
Krylov subspaces(instead of the implicit-Q theorem)
Krylov subspaces and subspace iteration
Krylov subspaces and Hessenberg form
Francis’s Algorithm – p. 19
Power Method, Subspace Iteration
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Substitutep(A) for A
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Substitutep(A) for A (shifts, multiple steps)
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Substitutep(A) for A (shifts, multiple steps)
S, p(A)S, p(A)2S, p(A)3S, . . .
Francis’s Algorithm – p. 20
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Substitutep(A) for A (shifts, multiple steps)
S, p(A)S, p(A)2S, p(A)3S, . . .
convergence rate|p(λj+1)/p(λj) |
Francis’s Algorithm – p. 20
Subspace Iterationwith changes of coordinate system
Francis’s Algorithm – p. 21
Subspace Iterationwith changes of coordinate system
takeS = span{e1, . . . , ej}
Francis’s Algorithm – p. 21
Subspace Iterationwith changes of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
Francis’s Algorithm – p. 21
Subspace Iterationwith changes of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
Francis’s Algorithm – p. 21
Subspace Iterationwith changes of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
change coordinate system:A = Q−1AQ
Francis’s Algorithm – p. 21
Subspace Iterationwith changes of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
change coordinate system:A = Q−1AQ
qk → Q−1qk = Q∗qk = ek
Francis’s Algorithm – p. 21
Subspace Iterationwith changes of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
change coordinate system:A = Q−1AQ
qk → Q−1qk = Q∗qk = ek
span{q1, . . . , qj} → span{e1, . . . , ej}
Francis’s Algorithm – p. 21
Subspace Iterationwith changes of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
change coordinate system:A = Q−1AQ
qk → Q−1qk = Q∗qk = ek
span{q1, . . . , qj} → span{e1, . . . , ej}
ready for next iterationFrancis’s Algorithm – p. 21
This version of subspace iteration . . .
Francis’s Algorithm – p. 22
This version of subspace iteration . . .
. . . holds the subspace fixed
Francis’s Algorithm – p. 22
This version of subspace iteration . . .
. . . holds the subspace fixed
while the matrix changes.
Francis’s Algorithm – p. 22
This version of subspace iteration . . .
. . . holds the subspace fixed
while the matrix changes.
. . . moving toward a matrix under which
span{e1, . . . , ej}
is invariant.
Francis’s Algorithm – p. 22
This version of subspace iteration . . .
. . . holds the subspace fixed
while the matrix changes.
. . . moving toward a matrix under which
span{e1, . . . , ej}
is invariant.
A →
[
A11 A12
0 A22
]
(A11 is j × j.)
Francis’s Algorithm – p. 22
Application to Francis’s Iteration(first pass)
Francis’s Algorithm – p. 23
Application to Francis’s Iteration(first pass)
A = Q−1AQ where q1 = αp(A)e1.
Francis’s Algorithm – p. 23
Application to Francis’s Iteration(first pass)
A = Q−1AQ where q1 = αp(A)e1.
power method
Francis’s Algorithm – p. 23
Application to Francis’s Iteration(first pass)
A = Q−1AQ where q1 = αp(A)e1.
power method+ change of coordinates
Francis’s Algorithm – p. 23
Application to Francis’s Iteration(first pass)
A = Q−1AQ where q1 = αp(A)e1.
power method+ change of coordinates
q1 → Q−1q1 = e1
Francis’s Algorithm – p. 23
Application to Francis’s Iteration(first pass)
A = Q−1AQ where q1 = αp(A)e1.
power method+ change of coordinates
q1 → Q−1q1 = e1
casej = 1 of subspace iteration with a change ofcoordinate system
Francis’s Algorithm – p. 23
Application to Francis’s Iteration(first pass)
A = Q−1AQ where q1 = αp(A)e1.
power method+ change of coordinates
q1 → Q−1q1 = e1
casej = 1 of subspace iteration with a change ofcoordinate system
. . . but this is just a small part of the story.
Francis’s Algorithm – p. 23
Krylov Subspaces . . .
Francis’s Algorithm – p. 24
Krylov Subspaces . . .. . . and Subspace Iteration
Francis’s Algorithm – p. 24
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
Francis’s Algorithm – p. 24
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Francis’s Algorithm – p. 24
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Kj(A, q) are “determined byq”.
Francis’s Algorithm – p. 24
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Kj(A, q) are “determined byq”.
p(A)Kj(A, q) = Kj(A, p(A)q)
Francis’s Algorithm – p. 24
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Kj(A, q) are “determined byq”.
p(A)Kj(A, q) = Kj(A, p(A)q)
. . . becausep(A)A = Ap(A)
Francis’s Algorithm – p. 24
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Kj(A, q) are “determined byq”.
p(A)Kj(A, q) = Kj(A, p(A)q)
. . . becausep(A)A = Ap(A)
Conclusion: Power method induces nested subspaceiterations on Krylov subspaces.
Francis’s Algorithm – p. 24
power method: q → p(A)kq
Francis’s Algorithm – p. 25
power method: q → p(A)kq
nested subspace iterations:
p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .
Francis’s Algorithm – p. 25
power method: q → p(A)kq
nested subspace iterations:
p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .
convergence rates:
|p(λj+1)/p(λj) |, j = 1, 2, 3, . . . , n − 1
Francis’s Algorithm – p. 25
Krylov Subspaces . . .
Francis’s Algorithm – p. 26
Krylov Subspaces . . .. . . and Hessenberg matrices . . .
Francis’s Algorithm – p. 26
Krylov Subspaces . . .. . . and Hessenberg matrices . . .
. . . go hand in hand.
Francis’s Algorithm – p. 26
Krylov Subspaces . . .. . . and Hessenberg matrices . . .
. . . go hand in hand.
A properly upper Hessenberg=⇒
Kj(A, e1) = span{e1, . . . , ej}.
Francis’s Algorithm – p. 26
Krylov Subspaces . . .. . . and Hessenberg matrices . . .
. . . go hand in hand.
A properly upper Hessenberg=⇒
Kj(A, e1) = span{e1, . . . , ej}.
More generally . . .
Francis’s Algorithm – p. 26
Krylov-Hessenberg Relationship
Francis’s Algorithm – p. 27
Krylov-Hessenberg Relationship
If A = Q−1AQ,
Francis’s Algorithm – p. 27
Krylov-Hessenberg Relationship
If A = Q−1AQ,
andA is properly upper Hessenberg,
Francis’s Algorithm – p. 27
Krylov-Hessenberg Relationship
If A = Q−1AQ,
andA is properly upper Hessenberg,
then forj = 1, 2, 3, . . . ,
Francis’s Algorithm – p. 27
Krylov-Hessenberg Relationship
If A = Q−1AQ,
andA is properly upper Hessenberg,
then forj = 1, 2, 3, . . . ,
span{q1, . . . , qj} = Kj(A, q1).
Francis’s Algorithm – p. 27
Application to Francis’s Iteration
Francis’s Algorithm – p. 28
Application to Francis’s Iteration
A = Q−1AQ where q1 = αp(A)e1.
Francis’s Algorithm – p. 28
Application to Francis’s Iteration
A = Q−1AQ where q1 = αp(A)e1.
power method with a change of coordinate system.
Francis’s Algorithm – p. 28
Application to Francis’s Iteration
A = Q−1AQ where q1 = αp(A)e1.
power method with a change of coordinate system.Moreover . . .
Francis’s Algorithm – p. 28
Application to Francis’s Iteration
A = Q−1AQ where q1 = αp(A)e1.
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
Francis’s Algorithm – p. 28
Application to Francis’s Iteration
A = Q−1AQ where q1 = αp(A)e1.
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}
Francis’s Algorithm – p. 28
Application to Francis’s Iteration
A = Q−1AQ where q1 = αp(A)e1.
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}
subspace iteration with a change of coordinatesystem
Francis’s Algorithm – p. 28
Application to Francis’s Iteration
A = Q−1AQ where q1 = αp(A)e1.
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}
subspace iteration with a change of coordinatesystem forj = 1, 2, 3, . . . ,n − 1
Francis’s Algorithm – p. 28
Application to Francis’s Iteration
A = Q−1AQ where q1 = αp(A)e1.
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}
subspace iteration with a change of coordinatesystem forj = 1, 2, 3, . . . ,n − 1
|p(λj+1)/p(λj) | j = 1, 2, 3, . . . ,n − 1
Francis’s Algorithm – p. 28
Details
Francis’s Algorithm – p. 29
Detailschoice of shifts
Francis’s Algorithm – p. 29
Detailschoice of shifts
We change the shifts at each step.
Francis’s Algorithm – p. 29
Detailschoice of shifts
We change the shifts at each step.
⇒ quadratic or cubic convergence
Francis’s Algorithm – p. 29
Detailschoice of shifts
We change the shifts at each step.
⇒ quadratic or cubic convergence
Watkins (2007, 2010)
Francis’s Algorithm – p. 29
Detailschoice of shifts
We change the shifts at each step.
⇒ quadratic or cubic convergence
Watkins (2007, 2010)
Francis’s Algorithm – p. 29
Where is John Francis?
Francis’s Algorithm – p. 30
Where is John Francis?question asked frequently by Gene Golub
Francis’s Algorithm – p. 30
Where is John Francis?question asked frequently by Gene Golub
inquiries by Golub and Uhlig
Francis’s Algorithm – p. 30
Where is John Francis?question asked frequently by Gene Golub
inquiries by Golub and Uhlig
Francis is alive and well,retired in the South of England.
Francis’s Algorithm – p. 30
Where is John Francis?question asked frequently by Gene Golub
inquiries by Golub and Uhlig
Francis is alive and well,retired in the South of England.
was unaware of the impact of his algorithm
Francis’s Algorithm – p. 30
Where is John Francis?question asked frequently by Gene Golub
inquiries by Golub and Uhlig
Francis is alive and well,retired in the South of England.
was unaware of the impact of his algorithm
appearance at the Biennial Numerical AnalysisConference in Glasgow in June of 2009
Francis’s Algorithm – p. 30
John Francis speaking in Glasgow
Francis’s Algorithm – p. 31
A Portion of the Audience
Francis’s Algorithm – p. 32
Afterwards
Photos courtesy of Frank UhligFrancis’s Algorithm – p. 33