Chapter 2Tridiagonal matrices

Gerard MEURANT

January-February, 2012

1 Similarity

2 Cholesky-like factorizations

3 Eigenvalues

4 Inverse

5 The QD algorithm

6 History

We have seen that orthogonal polynomials can be defined throughthe Jacobi matrices of their three-term recurrence coefficients

The zeros of orthogonal polynomials are given by eigenvalues oftridiagonal matrices

Hence, it is useful to study properties of tridiagonal matrices

Similarity

Let

Tk =

α1 ω1

β1 α2 ω2

. . .. . .

. . .

βk−2 αk−1 ωk−1

βk−1 αk

and βi 6= ωi , i = 1, . . . , k − 1

Proposition

Assume that the coefficients ωj , j = 1, . . . , k − 1 are different fromzero and the products βj ωj are positive. Then, the matrix Tk issimilar to a symmetric tridiagonal matrix. Therefore, itseigenvalues are real

Proof.Consider D−1

k TkDk which is similar to Tk , Dk diagonal matrixwith diagonal elements δj

Take

δ1 = 1, δ2j =

βj−1 · · ·β1

ωj−1 · · ·ω1, j = 2, . . . k

Let

Jk =

α1 β1

β1 α2 β2

. . .. . .

. . .

βk−2 αk−1 βk−1

βk−1 αk

where the values βj , j = 1, . . . , k − 1 are assumed to be nonzero

Proposition

det(Jk+1) = αk+1 det(Jk)− β2k det(Jk−1)

with initial conditions

det(J1) = α1, det(J2) = α1α2 − β21 .

The eigenvalues of Jk are the zeros of det(Jk − λI )

The zeros do not depend on the signs of the coefficientsβj , j = 1, . . . , k − 1

We suppose βj > 0 and we have a Jacobi matrix

Cholesky-like factorizations

Let ∆k be a diagonal matrix with diagonal elements δj ,j = 1, . . . , k and

Lk =

1l1 1

. . .. . .

lk−2 1lk−1 1

Jk = Lk∆kLT

k

The entries of Lk and ∆k are given by

δ1 = α1, l1 = β1/δ1

δj = αj −β2

j−1

δj−1, j = 2, . . . , k, lj = βj/δj , j = 2, . . . , k − 1

The factorization can be completed if no δj is zero forj = 1, . . . , k − 1

This does not happen if Jk is positive definite, all the elements δj

are positive and the genuine Cholesky factorization can beobtained from ∆k

Jk = LCk (LC

k )T

with LCk = Lk∆

1/2k which is

LCk =

√δ1

β1√δ1

√δ2

. . .. . .

βk−2√δk−2

√δk−1

βk−1√δk−1

√δk

The factorization can also be written as

Jk = LDk ∆−1

k (LDk )T

with

LDk =

δ1

β1 δ2

. . .. . .

βk−2 δk−1

βk−1 δk

Clearly, the only elements we have to compute and store are the

diagonal elements δj , j = 1, . . . , k

To solve a linear system Jkx = c , we successively solve

LDk y = c , (LD

k )T x = ∆ky

The previous factorizations proceed from top to bottom (LU)We can also proceed from bottom to top (UL)

Jk = LTk D−1

k Lk

with

Lk =

d

(k)1

β1 d(k)2. . .

. . .

βk−2 d(k)k−1

βk−1 d(k)k

and Dk a diagonal matrix with elements d

(k)j

d(k)k = αk , d

(k)j = αj −

β2j

d(k)j+1

, j = k − 1, . . . , 1

From LU and UL factorizations we can obtain all the so-called“twisted” factorizations of Jk

Jk = MkΩkMTk

Mk is lower bidiagonal at the top for rows with index smaller thanl and upper bidiagonal at the bottom for rows with index largerthan l

ω1 = α1, ωj = αj −β2

j−1

ωj−1, j = 2, . . . , l − 1

ωk = αk , ωj = αj −β2

j

ωj+1, j = k − 1, . . . , l + 1

ωl = αl −β2

l−1

ωl−1−

β2l

ωl+1

Eigenvalues

The eigenvalues of Jk are the zeros of det(Jk − λI )

det(Jk − λI ) = δ1(λ) · · · δk(λ) = d(k)1 (λ) · · · d (k)

k (λ)

This shows that

δk(λ) =det(Jk − λI )

det(Jk−1 − λI ), d

(k)1 (λ) =

det(Jk − λI )

det(J2,k − λI )

TheoremThe eigenvalues θ

(k+1)i of Jk+1 strictly interlace the eigenvalues of

Jk

θ(k+1)1 < θ

(k)1 < θ

(k+1)2 < θ

(k)2 < · · · < θ

(k)k < θ

(k+1)k+1

(Cauchy interlacing theorem)

Proof.Eigenvector x = (y ζ)T of Jk+1 corresponding to θ

Jky + βkζek = θy

βkyk + αk+1ζ = θζ

Eliminating y from these relations, we obtain

(αk+1 − β2k((ek)T (Jk − θI )−1ek))ζ = θζ

αk+1 − β2k

k∑j=1

ξ2j

θ(k)j − θ

− θ = 0

where ξj is the last component of the jth eigenvector of Jk

The zeros of this function interlace the poles θ(k)j

See next slide

−1 0 1 2 3 4 5 6 7−10

−8

−6

−4

−2

0

2

4

6

8

10

Secular function

Results on eigenvector components

Let χj ,k(λ) be the determinant of Jj ,k − λI

The first components of the eigenvectors z i of Jk are

(z i1)

2 =

∣∣∣∣∣∣χ2,k(θ(k)i )

χ′1,k(θ(k)i )

∣∣∣∣∣∣ ,that is,

(z i1)

2 =θ(k)i − θ

(2,k)1

θ(k)i − θ

(k)1

· · ·θ(k)i − θ

(2,k)i−1

θ(k)i − θ

(k)i−1

θ(2,k)i − θ

(k)i

θ(k)i+1 − θ

(k)i

· · ·θ(2,k)k−1 − θ

(k)i

θ(k)k − θ

(k)i

.

The last components of the eigenvectors z i of Jk are

(z ik)2 =

∣∣∣∣∣∣χ1,k−1(θ(k)i )

χ′1,k(θ(k)i )

∣∣∣∣∣∣ ,that is,

(z ik)2 =

θ(k)i − θ

(k−1)1

θ(k)i − θ

(k)1

· · ·θ(k)i − θ

(k−1)i−1

θ(k)i − θ

(k)i−1

θ(k−1)i − θ

(k)i

θ(k)i+1 − θ

(k)i

· · ·θ(k−1)k−1 − θ

(k)i

θ(k)k − θ

(k)i

.

Inverse

What is the inverse of a tridiagonal matrix?

TheoremThere exist two sequences of numbers ui, vi, i = 1, . . . , k suchthat

J−1k =

u1v1 u1v2 u1v3 . . . u1vk

u1v2 u2v2 u2v3 . . . u2vk

u1v3 u2v3 u3v3 . . . u3vk...

......

. . ....

u1vk u2vk u3vk . . . ukvk

Moreover, u1 can be chosen arbitrarily, for instance u1 = 1

see Baranger and Duc-Jacquet; MeurantHence, we just have to compute J−1

k e1 = u1v and J−1k ek = vku

SolveJkv = e1

Use UL factorization of Jk

v1 =1

d(k)1

, vj = (−1)j−1 β1 · · ·βj−1

d(k)1 · · · d (k)

j

, j = 2, . . . , k

ForvkJku = ek

use LU factorization

uk =1

δkvk, uk−j = (−1)j

βk−j · · ·βk−1

δk−j · · · δkvk, j = 1, . . . , k − 1

TheoremThe inverse of the symmetric tridiagonal matrix Jk is characterizedas

(J−1k )i ,j = (−1)j−iβi · · ·βj−1

d(k)j+1 · · · d

(k)k

δi · · · δk, ∀i , ∀j > i

(J−1k )i ,i =

d(k)i+1 · · · d

(k)k

δi · · · δk, ∀i

Proof.

ui = (−1)−(i+1) 1

β1 · · ·βi−1

d(k)1 · · · d (k)

k

δi · · · δk

The diagonal elements of J−1k can also be obtained using twisted

factorizations

TheoremLet l be a fixed index and ωj the diagonal elements of thecorresponding twisted factorizationThen

(J−1k )l ,l =

1

ωl

In the sequel we will be interested in (J−1k )1,1

(J−1k )1,1 =

1

d(k)1

The (1,1) entry of the inverse

Can we compute (J−1k )1,1 incrementally?

Theorem

(J−1k+1)1,1 = (J−1

k )1,1 +(β1 · · ·βk)2

(δ1 · · · δk)2δk+1

Proof.

Jk+1 =

(Jk βkek

βk(ek)T αk+1

)The upper left block of J−1

k+1 is the inverse of the Schurcomplement (

Jk −β2

k

αk+1ek(ek)T

)−1

Inverse of a rank-1 modification of Jk

Use the Sherman–Morrison formula

(A + αxyT )−1 = A−1 − αA−1xyTA−1

1 + αyTA−1x

This gives(Jk −

β2k

αk+1ek(ek)T

)−1

= J−1k +

(J−1k ek)((ek)T J−1

k )αk+1

β2k− (ek)T J−1

k ek

Let lk = J−1k ek

(J−1k+1)1,1 = (J−1

k )1,1 +β2

k(lk1 )2

αk+1 − β2k lkk

lk1 = (−1)k−1 β1 · · ·βk−1

δ1 · · · δk, lkk =

1

δk

To simplify the formulas, we note that

αk+1 − β2k lkk = αk+1 −

β2k

δk= δk+1

We start with (J−11 )1,1 = π1 = 1/α1 and c1 = 1

t = β2k πk , δk+1 = αk+1 − t, πk+1 =

1

δk+1, ck+1 = t ck πk

This gives(J−1

k+1)1,1 = (J−1k )1,1 + ck+1πk+1

The QD algorithm

The QD algorithm is a method introduced by Heinz Rutishauser tocompute the eigenvalues of a tridiagonal matrix

See also Stiefel, Henrici, Fernando and Parlett, Parlett and Laurie

Let us start with the LR algorithm

LR algorithm for eigenvalues

- Start from the Cholesky factorization Jk = LkLTk of the

tridiagonal positive definite matrix Jk

- Compute Jk = LTk Lk ⇒ Jk = L−1

k JkLk

The matrix Jk is similar to the matrix Jk

- Iterate the process obtaining a series of matrices J(i)k with

J(0)k = Jk , J

(1)k = Jk , . . .

This is the basis of the LR algorithm of Rutishauser

The off-diagonal elements tend to zero and in the limit we obtainthe eigenvalues of Jk on the diagonal

Can we compute Lk , the Cholesky factor of Jk = LTk Lk without

explicitly computing Jk?

We have

Jk =

δ1 +β2

1δ1

β1

√δ2δ1

β1

√δ2δ1

δ2 +β2

2δ2

β2

√δ3δ2

. . .. . .

. . .

βk−2

√δk−1

δk−2δk−1 +

β2k−1

δk−1βk−1

√δk

δk−1

βk−1

√δk

δk−1δk

Let√

εj be the subdiagonal entries of LCk

Therefore, εj = β2j /δj

The diagonal entries of the Cholesky-like factorization of Jk aregiven by

δ1 = δ1 +β2

1

δ1= δ1 + ε1

δj = δj +β2

j

δj−

β2j−1δj/δj−1

δj−1

= δj + εj −εj−1δj

δj−1

, j = 2, . . . , k − 1

δk = δk −β2

k−1δk/δk−1

δk−1

= δk −εk−1δk

δk−1

Let εj = β2j δj+1/(δj δj)

The diagonal entries of Lk are√

δj , the subdiagonal entries are√εj and

εj = εjδj+1

δj

The expression for δj can be written as

δj = δj + εj − εj−1

The QD algorithm is the following; given δj and εj :

ε0 = 0for j=1:k-1δj = (δj − εj−1) + εjεj = εjδj+1/δj

endδk = δk − εk−1

Then, in the LR algorithm, we do δj = δj , εj = εj and we iterateuntil the off-diagonal elements are small enough

The differential QD algorithm (dqd)The QD algorithm can be modified by removing subtractions

δj =δj

δj−1

(δj−1 − εj−1) + εj

Let tj = δj − εj . Then

tj = tj−1δj

δj−1

, δj = tj + εj

t = δ1

for j=1:k-1δj = t + εjf = δj+1/δj

εj = f εjt = ft

endδk = t

The differential QD algorithm with shift

The algorithm with shift dqds(µ) is:

t = δ1 − µfor j=1:k-1δj = t + εjf = δj+1/δj

εj = f εjt = ft − µ

endδk = t

The Cholesky factorization of a shifted matrix

Assume we know the factorization of Jk = LDLT . We would liketo compute the factorization of Jk − µI = LDLT

δ1 = α1−µ, δj = αj−µ−β2

j−1

δj−1

, j = 2, . . . , k, lj =βj

δj

, j = 1, . . . , k−1

Butα1 = δ1, αj = δj + δj−1lj−1, j = 2, . . . , k

βj = δj lj , j = 1, . . . , k − 1

We can eliminate αj and βj from the formulas giving δj and lj

δ1 = δ1 − µ, δj = δj − µ + δj−1l2j−1 −

δ2j−1l

2j−1

δj−1

, j = 2, . . . , k

and

lj =δj lj

δj

, j = 1, . . . , k − 1

Finally

δj = δj − µ + δj−1l2j−1 − δj−1lj−1 lj−1, j = 2, . . . , k

This algorithm has been named stqds by Dhillon and Parlett

The dstqds algorithm

Let sj = δj − δj ; then

sj = lj−1 lj−1

(lj−1δj−1

lj−1

− δj−1

)− µ

= lj−1 lj−1(δj−1 − δj−1)− µ

= lj−1 lj−1sj−1 − µ

s = −µfor j=1:k-1δj = s + δj

lj = (δj lj)/δj

s = lj ljs − µendδk = s + δk

Historical note

I Augustin Louis Cauchy, (1789 Paris, France-1857 Sceaux,France), French mathematician

I Issai Schur, (1875 Mogilev, Russia-1941 Tel Aviv, Israel),German mathematician

I Andre-Louis Cholesky, (1875 Montguyon, France-1918Bagneux (Soissons), France), French geodesist

I Eduard Ludwig Stiefel, (1909-1978 Zurich, Switzerland),Swiss mathematician

I Heinz Rutishauser, (1918 Weinfelden, Switzerland-1970Zurich, Swtizerland), Swiss mathematician

I Peter Henrici, (1923 Basel, Switzerland-1987 Zurich,Switzerland), Swiss mathematician

I Jack Sherman, (?), American statistician

I Winifred J. Morrison, (?), American statistician

I Beresford Neill Parlett, (? ?-), American mathematician

A.L. Cauchy I. Schur

1789-1857 1875-1941

A-L. Cholesky

1875-1918

H. Rutishauser E. Stiefel

1918-1970 1909-1978

P. Henrici B.N. Parlett

1923-1987

J. Baranger and M. Duc-Jacquet, Matricestridiagonales symetriques et matrices factorisables, RIRO, v 3(1971), pp. 61–66

I.S. Dhillon and B.N. Parlett, Relatively robustrepresentations of symmetric tridiagonals, Linear Alg. Appl.,v 309 (2000), pp. 121–151

K.V. Fernando and B.N. Parlett, Accurate singularvalues and differential qd algorithms, Numer. Math., v 67(1994), pp. 191–229

G.H. Golub and C. Van Loan, Matrix Computations,Third Edition, Johns Hopkins University Press, (1996)

P. Henrici, The quotient-difference algorithm, NBSAppl. Math. Series, v 49 (1958), pp. 23–46

G. Meurant, A review of the inverse of tridiagonal andblock tridiagonal matrices, SIAM J. Matrix Anal. Appl., v 13n 3 (1992), pp. 707–728

B.N. Parlett, The new qd algorithms, Acta Numerica, v 15(1995), pp. 459-491

H. Rutishauser, Der Quotienten-Differenzen-Algorithmus,Zeitschrift fur Angewandte Mathematik und Physik (ZAMP),v 5 n 3 (1954), pp. 233-251

H. Rutishauser, Solution of eigenvalue problems with theLR-transformation , Nat. Bur. Standards Appl. Math. Ser.,v 49 (1958), pp. 47–81