Introduction to Matrix Algebra

George H Olson, Ph. D.

Doctoral Program in Educational Leadership

Appalachian State University

September 2010

What is a matrix? Dimensions and order of a matrix. A p by q dimensioned matrix is a p (rows) by q (columns) array of numbers, or symbols, and will itself be represented by an upper-case, bold, italic symbol; e.g, A , β . For instance, a 2 by 3 matrix, A , can

be represented, as a symbolic array,

23

13

22

12

21

11

a

a

a

a

a

aA ,

or as an actual array of numbers

3

6

4

5

2

3A .

In either case, the order of A is said to be 2 3. On the other hand, the matrix,

434241

333231

232221

131211

ddd

ddd

ddd

ddd

D ,

is a 4 3 matrix. That is, its order is 4 3. A matrix having only one column or one row is called a vector and is represented by a lower-case, bold, italic symbol; e.g., a, β . Hence the 1 4 and 3 1 matrices,

4321 aaaaa , and

3

2

1

β ,

are both vectors. A is a row vector; β is a column vector.

The numbers (or symbols) inside a matrix are called elements. Thus, in the first matrix, A, given above, a11 and a23 are elements of the matrix A. Similarly, in the second matrix, A, given earlier, the numbers 4 and 6 are elements. Elements in a matrix are indexed (or referred to) by subscripts that give their row and column locations. For, instance, in the first matrix, A, above, a23 is the element in the second row of the third column. Similarly, in the matrix, D, above, element d41 is found in the fourth row, first column. In general, aij is the i,j’th element of A. When referring to elements of vectors, however, the first row (or column) is assumed. Hence, in the vector, a, above, a2 is its second element; in

β , 3 is the third element. In general, elements of vectors, are indicated by the

j’th element of row (i.e., 1 x q) vectors and the i’th element of column (i.e., p x 1) vectors.

A matrix in which the number of rows (p) equals the number of columns (q) is called a square matrix.; otherwise, providing it is not a vector, it is called a rectangular matrix.

The natural order of a rectangular matrix has more rows than columns (i.e., p > q). Hence,

23

22

21

13

12

11

a

a

a

a

a

a

A ,

a 3 x 2 matrix, is presented in its natural order.

The transpose of a matrix is obtained by interchanging its rows and columns. Thus, the transpose of A, represented as A , is

23

13

22

12

21

11

a

a

a

a

a

aA ,

a 3 x 2 matrix. As a concrete example, consider

622

761

475

342

X ,

which, when transposed, becomes

6

2

2

7

6

1

4

7

5

3

4

2

X .

The natural order of a vector is a p x 1 column vector. Here, the 4 x 1 column vector,

4

3

2

1

a

a

a

a

a ,

is presented in its natural order. The transpose of a,

4321 aaaaa ,

is a 1 x 4 row vector.

As mentioned earlier a matrix where p = q is a square matrix. At least two particular types of square matrices are particularly important, symmetric matrices and diagonal matrices. A symmetric matrix is a square matrix in which the elements above the main diagonal are mirror images of the elements below the main diagonal (the main diagonal is that set of elements running from the upper left-hand side of a square matrix to the lower right-hand. In the symmetric matrix V, given below, the elements in bold represent the main diagonal. Note that the elements above the main diagonal are mirror images of the elements below the main diagonal.

59

64

56

6138

6152

3852

V .

Diagonal matrices are square matrices in which all elements except the main diagonal are zero (0). For instance,

61000

04900

00520

00037

D

is a symmetric diagonal matrix.

An important diagonal matrix is the identity matrix. The identity matrix is a diagonal matrix in which all the elements along the main diagonal are unity (1). For example,

100

010

001

I

is a 3 x 3 identity matrix.

Symmetric matrices have important properties. For instance, a symmetric matrix is equal to its transpose. That is, for the matrix given a little earlier, V = V .

Matrix Operations

Addition

The addition of two matrices, A and B, is accomplished by adding corresponding

elements, e.g., aij + bij.; hence, for the 4 x 3 matrices, andA B .

434241

333231

232221

131211

434342424141

333332323131

232322222121

131312121111

434241

333231

232221

131211

434241

333231

232221

131211

ccc

ccc

ccc

ccc

bababa

bababa

bababa

bababa

bbb

bbb

bbb

bbb

aaa

aaa

aaa

aaa

BAC

It is obvious (or should be obvious) that the orders of the two matrices being added need to be identical. Here, for instance, both matrices are of order 4 x 3. When the orders of the two matrices are not identical, addition is not impossible.

Addition of matrices is commutative. Hence A + B = B + A.

As a concrete example, let

431

854

113

857

A , and

313

214

331

302

B .

Then, C = A + B = B + A

=

341331

281544

313113

380527

=

744

1068

444

1159

.

Multiplication

Multiplication is somewhat more complicated. When multiplying matrices we need to distinguish among multiplication by a scalar, vector multiplication, pre- multiplication, and post-multiplication, inner-products and outer-products. But first, let us define a scalar. A scalar is simply a single number, such as 2, 17.6, or -300.335. Symbolically, we typically use the symbol, c, to represent a scalar.

Scalar multiplication

Scalar multiplication is easily shown by example. Given the 3 x 2 matrix X and the scalar, c, the product, cX, is given by

Note that all the elements in X are multiplied by the scalar, c.

.

3231

22

3231

22

3231

22

3231

22

cxcx

cxc

cc

c

xx

x

cxcx

cx

xx

xc

21

1211

21

1211

21

1211

21

1211

x

xx

x

xx

cx

cxcx

x

xx

cX

As a concrete example, let

21

03

74

X

and c = 5. Then

105

015

3520

21

03

74

21

03

74

5

5Xc

In the above expression, the product, cX, results from pre-multiplying X by the scalar c; whereas the product Xc results from post-multiplying X by the scalar, c. Since scalar multiplication is commutative, cX will always be equal to Xc. This is not the case with matrix multiplication, however. Only in certain special cases will the products formed by pre-multiplying and post-multiplying two matrices, X and

Y be equal. In general XY ≠ YX.

Matrix Multiplication

First, note that given two matrices, e.g., A of order n x m, and B of order p x q, multiplication is only possible when the number of columns in the pre-multiplier is equal to the number of rows in the post-multiplier. Hence the product, AB, is only possible when m = p. Similarly, the product BA is only possible when n = q.

When m = p the product, C = AB will have order n x q. Similarly, when q = n, the product, C = BA will have order p x m.

We can illustrate this by defining the two matrices,

,

434241

333231

232221

131211

xxx

xxx

xxx

xxx

X and

23

13

22

12

21

11

y

y

y

y

y

yY .

Note that X has order 4 x 3, and Y’ has order 2 x 3. Here, only the products, XY (not XY’) and Y’X’ (not YX) are possible. Post-multiplying X by Y, yields

234322422141134312421141

233322322131133312321131

232322222121132312221121

231322122111131312121111

2313

2212

2111

434241

333231

232221

131211

yxyxyxyxyxyx

yxyxyxyxyxyx

yxyxyxyxyxyx

yxyxyxyxyxyx

yy

yy

yy

xxx

xxx

xxx

xxx

XY

where the product is of order 4 x 2. As a more concrete example, let

231

012

131

102

X and

2

1

0

1

3

2Y .

Then

,

77

65

56

95

403232

006014

203132

306104

21

01

32

231

012

131

102

XY

a 4 x 2 matrix. The only other product possible between these two matrices is Y’X’:

,7658

7563

403

232

006

014

203

132

206

102

2

3

1

0

1

2

1

3

1

1

0

2

432342224121332343223121232322222121132312222121

431342124111331343123111231322122111131312121111

43332313

42322212

41312111

23

13

22

12

21

11

2

1

0

1

3

2

xyxyxyxyxyxyxyxyxyxyxyxy

xyxyxyxyxyxyxyxyxyxyxyxy

xxxx

xxxx

xxxx

y

y

y

y

y

yXY

which is a 2 x 4 matrix.

When A and B are both square matrices of the same order, then all the products, AB, BA, A’B, AB’, B’A, BA’, A’B’, and B’A’ are possible. Furthermore, except is certain special cases, the matrices formed by these various products will be different from each other. As an exercise, try computing all possible products of the following two square matrices:

120

132

321

A and

112

013

122

B .

Vector Multiplication

Having learned how to compute matrix multiplication, vector multiplication is easy. It follows the same rules of matrix multiplication except that here one of the matrices is a vector. For instance, given the 4 x 1 column vector,

4

3

2

1

n

n

n

n

n , and the 4 x 2 matrix,

4241

3231

2221

1211

yy

yy

yy

yy

Y ,

the products n Y and Y n are the only possible products. Hence,

.21

424323222121414313212111

4241

3231

2221

1211

4321

iiii ynyn

ynynynynynynynyn

yy

yy

yy

yy

nnnnYn

To put this in concrete terms, let

5213n and .

4

2

3

1

4

2

2

3

Y

Then

25321543320426

3

2

3

1

4

2

2

3

5213

Yn .

Some special cases of vector and matrix multiplication are particularly important. For instance, let the vector, 1 be defined as a vector with all elements equal to 1:

1

1

1

1

1

1 and .

24

13

04

21

32

21

yyY {Note, y1 and y2 are column vectors}.

Then,

81421 yyY1 .

If we let

21

21

31

11

21

21 xxX and

34

13

04

21

32

21 yyY .

Then,

.1631

914

21

21

xyxy

yyYX

Note also that

.2210

1052

2

2

22

212

211

x

x

x

n

xxx

xxxXX

Inner-products and outer-products

Given the two matrices,

3231

2221

1211

aa

aa

aa

A and

232221

131211

bbb

bbbB ,

Verify that the following two products are possible, AB and BA (of course, A’B’ and B’A’ also are possible). The orders of the two products are not identical. AB has order, 3 x 3, while BA has order 2 x 2. The product AB is an outer-product while the product BA is an inner-product. In general, whenever both an inner-product and an outer-product exist for two matrices, the order of the inner-product will be less than the outer-product. Furthermore, in general, the inner-product will not be identical to the outer-product.

For example, let

13

23

12

A and

123

421B .

Then

1386

14109

965

AB and

815

920BA .

Inner- and outer-products are particularly important in vector multiplication. Let

1111a , ,111b and

112

234

432

321

X .

Then

3211099 iii xxxXa , {Note the summation is over

rows} and

j

j

j

j

x

x

x

x

4

3

2

1

4

9

9

6

Xb . {Here the summation is over columns}.

Also, it is easy to verify that

ijx28Xba where the summation is over rows and columns.