EIGENVALUES OF SYMMETRIC MATRICES, AND GRAPH THEORY

Last week we saw how to use the eigenvalues of a matrix to study the properties of a graph. If ourgraph is undirected, then the adjacency matrix is symmetric. There are many special properties ofeigenvalues of symmetric matrices, as we will now discuss.

Let A be a symmetric matrix. Let λ and µ be eigenvalues of A, with corresponding eigenvectorsu and v. We claim that, if λ and µ are distinct, then u and v are orthogonal. Proof: We haveuTAv = λ(uT v). But, also, uTAv = (Au)T v = µuT v. So λuT v = µuT v and we deduce thatuT v = 0.

Thus, if A has n distinct eigenvalues, with n real eigenvectors vi, then the vi’s are orthogonal andcan be normalized to be orthonormal. In fact, more is true. As you should have learned in yourlinear algebra class, we have

The Spectral Theorem: If A is a symmetric real matrix, then the eigenvalues of A are real andRn has an orthonormal basis of eigenvectors for A.

Let v1, v2, . . . , vn be the promised orthogonal basis of eigenvectors for A. Let S be the matrixwhich takes the standard basis vector ei to vi; explicitly, the columns of S are the vi. As we learnedbefore, we have A = S−1diag(λ1, . . . , λn)S. However, in this case, things get even better. Since thevi are orthonormal, the matrix S is orthogonal and we have S−1 = ST . In other words, the rowsof S are again the vi:

A =

vT1vT2· · ·vTn

λ1

λ2. . .

λn

v1 v2 · · · vn

.

This result is so important that we write it in several equivalent ways:

A =n∑

i=1

λivivTi .

A(u) =

n∑i=1

λivi〈vi, u〉.

And, most importantly for our current purposes,

〈u,Au〉 =∑

λi〈vi, u〉2 (∗).

Write λ1 ≥ λ2 ≥ · · · ≥ λn. From equation (∗), we have the following consequences:

For any vector u 6= 0, we have

λ1 ≥〈Au, u〉〈u, u〉

≥ λn.

For any vector u which is orthogonal to v1, we have

λ2 ≥〈Au, u〉〈u, u〉

≥ λn.1

2 EIGENVALUES OF SYMMETRIC MATRICES, AND GRAPH THEORY

The proof of the first equation is simple enough: if u =∑civi then 〈Au, u〉/〈u, u〉 =

(∑λic

2i

)/(∑

c2i),

which is between λ1 and λn. The second result is similar, just noting that the condition that u isorthogonal to v1 means that λ1 = 0.

* * *

Let G be a d regular graph.1 Let A be the adjacency matrix, and let λ1 ≥ λ2 ≥ · · · ≥ λn be theeigenvalues of A. As a first application of these ideas, we show that all the λi’s lie between −d andd. By the spectral theorem, we know that the λi are real.

Proof: Let L be the Laplacian matrix, L = d · Id−A. So the eigenvalues of L are d−λ1, . . . , d−λn.Note that, for any vector u, we have 〈u, Lu〉 =

∑(i,j)∈Edge(G)(u(i) − u(j))2 ≥ 0. In particular, if

vi are the orthonormal eigenvectors of A, then we have 〈vi, Lvi〉 = (d − λi)〈vi, vi〉 = d − λi. Sod− λi ≥ 0 and we see that d ≥ λi. A similar argument, using d · Id +A, shows that −d ≤ λi.Exercise Show that λ2 = d if and only if G is disconnected. Show that λn = −d if and only if Gis bipartite.

* * *

We now return to our study of how well random walks mix on G. Let ` be the largest absolutevalue of any of λ2, λ3, . . . , λn, so ` is either λ2 or −λn.

As we noted last time, the number of walks from r to s of length k is of the form c1dk + c2λ

k2 + · · ·

for various constants c2, . . . , cn which we were not able to compute at that time. We now remedythis. We have

Ak = ST

λk1

λk2. . .

λkn

S.

So the (r, s) entry of Ak isn∑

i=1

(vi)rλki (vi)s.

Let’s start with the i = 1 term, which will give us the coefficient of dn. The eigenvector witheigenvalue d is (1, 1, . . . , 1). That’s before normalizing to become orthonormal. The all onesvector has length

√n, so v1 = (1/

√n, 1/

√n, . . . , 1/

√n). Thus, the leading term of our sum is

1/√n1/√ndn = dn/n. Notice that, if the dn paths from r were distributed at random, we’d expect

dn/n of them to land at s.

For a crude bound for the other terms, since 〈vi, vi〉 = 1, every coordinate of vi is at most 1. So wededuce that

|#(paths from r to s of length k)− dn/n| ≤ `n + `n + · · ·+ `n = (n− 1)`k.

We can do a little better if we think about the fact that the vi are orthonormal. Although anyindividual component in a vi might be near 1, we can’t have many of them near 1 all at once. Morespecifically, we have the equality of matrices∑

vivTi = Id

son∑

i=1

(vi)2r = 1.

1Almost all of these ideas can be generalized to non-regular graphs, but the notation gets worse.

EIGENVALUES OF SYMMETRIC MATRICES, AND GRAPH THEORY 3

By Cauchy-Scwartz,∑|(vi)r||(vi)s| ≤ 1 and we have

|#(paths from r to s of length k)− dn/n| ≤ `k(

n∑i=2

|(vi)r||(vi)s|

)≤ `k.

Let’s see what this implies about the diameter of G. Let K + 1 be the greatest distance betweenany two vertices of G. We must have that dK/n ≤ `K , so that it is possible there can be no pathsof length ≤ K from r to s. In other words,

K ≤ log n

log d− log `.

Notice that log n/ log d is an obvious lower bound for K. So, if ` is significantly less than d, thenwe come within a constant factor of this lower bound.

This seems like a good point to tell the definition of an expander sequence: For fixed d, an expandersequence is a sequence of graphs Gn, all d regular, such that the size of Gn goes to ∞ there is aconstant R < d so that `(Gn) < R for all n.

In particular, for an expander sequence, the diameter is close to the minimum possible, and we seethat signals sent through an expander graph distribute very fast.

* * *

As a second, longer, application, we will show that, if λ2 is significantly less than d, then anyseparation of G into two pieces must cut many edges.

Let’s partition the vertices of G into two sets, X and Y . We will build a vector u which is orthogonalto v1. Namely, u(w) = 1/|X| if w ∈ X and u(w) = −1/|Y | if w ∈ Y . Notice that 〈< u, v1〉 isnothing but

∑u(w) = |X|/|X| − |Y |/|Y | = 0.

We thus know that〈u,Au〉〈u, u〉

≤ λ2.

Although it isn’t too bad to compute 〈u,Au〉, it is even easy to work with the Laplacian. Theeigenvalues of L are 0, d− λ2, . . . , d− λn, and the eigenvectors are the same as for A. So

〈u, Lu〉〈u, u〉

≥ d− λ2.

The numerator is∑

(i,j)∈Edge(G)(u(i)− u(j))2 This is

#(Edges between X and Y )(1/|X|+ 1/|Y |)2.The denominator, meanwhile, is

∑u(w)2 = |X|/|X|2 + |Y |/|Y |2 = 1/|X|+ 1/|Y |.

Putting it all together,

#(Edges between X and Y )(1/|X|+ 1/|Y |)2

1/|X|+ 1/|Y |≥ d− λ2.

We have 1/|X|+ 1/|Y | = |X| · |Y |/(|X|+ |Y |) = |X| · |Y |/n so we have

#(Edges between X and Y )(1/|X|+ 1/|Y |)2 ≥ d− λ2n|X| · |Y |.

In other words, of the |X| · |Y | possible edges between X and Y , at least (d − λ2)/n of them arepresent. For a random pair of vertices, the odds that there is an edge between them is d/n. So,when λ2 is significantly less than d, the number of edges between X and Y is always within aconstant ratio of what you would expect at random.