• date post

17-Oct-2020
• Category

## Documents

• view

1

0

Embed Size (px)

### Transcript of Section 1.5 - Exploiting partitioning of matrices and · PDF file Section 1.5 - Exploiting...

• Section 1.5 - Exploiting partitioning of matrices and vectors

Maggie Myers Robert A. van de Geijn

The University of Texas at Austin

Practical Linear Algebra – Fall 2009

http://z.cs.utexas.edu/wiki/pla.wiki/ 1

• Example: Partitioning vectors

Given a vector

x =

 4 −1 3 1

 one can think of this as its four elements

x =

 χ0 χ1 χ2 χ3

 where χ0 = 4, χ1 = −1, etc. Note

The parentheses are only there to delimit (outline) the vector. They have no particular other meaning.

http://z.cs.utexas.edu/wiki/pla.wiki/ 2

• Example: Partitioning vectors

Given a vector

x =

 4 −1 3 1

 one can think of this as two subvectors:

x = ( x0 x1

) =

 (

4 −1

) (

3 1

)  =

 4 −1 3 1

 so that

x0 = (

4 −1

) and x1 =

( 3 1

)

http://z.cs.utexas.edu/wiki/pla.wiki/ 3

• Example: Partitioning vectors

Given a vector

x =

 4 −1 3 1

 one can think of this as two subvectors:

x = ( x0 x1

) =

 (

4 ) −13

1

  =

 4 −1 3 1

 so that

x0 = (

4 )

and x1 =

 −13 1

 http://z.cs.utexas.edu/wiki/pla.wiki/ 4

• Example: Partitioning vectors

Given a vector

x =

 4 −1 3 1

 one can think of this as two subvectors:

x = ( x0 x1

) =

 ( )

4 −1 3 1

  =

 4 −1 3 1

 so that

x0 = ( )

and x1 =

 4 −1 3 1

 http://z.cs.utexas.edu/wiki/pla.wiki/ 5

• Example: Inner product with partitioned vectors

Given vector

x = ( x0 x1

) =

 4 −1 3 1

 and y = ( y0y1 )

=

 1 −2 3 −4

 We find that

xT y = (4)× (1) + (−1)× (−2)︸ ︷︷ ︸ = (

4 −1

)T ( 1 −2

) = xT0 y0

+ (3)× (−3) + (1)×−4)︸ ︷︷ ︸ = (

3 1

)T ( 3 −4

) = xT1 y1︸ ︷︷ ︸

= xT0 y0 + x T 1 y1

http://z.cs.utexas.edu/wiki/pla.wiki/ 6

• Theorem

Let x, y ∈ Rn and partition

x =

 x0 x1 ...

xN−1

 and y = 

y0 y1 ...

yN−1

 , where xi and yi have the same size, for i = 0, . . . , N − 1.

http://z.cs.utexas.edu/wiki/pla.wiki/ 7

• Theorem (continued)

Then

xT y =

 x0 x1 ...

xN−1

 T 

y0 y1 ...

yN−1



= ( xT0 x

T 1 · · · xTN−1

) 

y0 y1 ...

yN−1

 = xT0 y0 + x

T 1 y1 + · · ·+ xTN−1yN−1

http://z.cs.utexas.edu/wiki/pla.wiki/ 8

• Example: axpywith partitioned vectors

Given

x = ( x0 x1

) =

 4 −1 3 1

 , y = ( y0y1 )

=

 1 −2 3 −4

 , and α = 4 We find that

αx+ y = 4

 4 −1 3 1

+ 

1 −2 3 −4

 = 

(4)× (4) + (1) (4)× (−1) + (−2)

(4)× (3) + (3) (4)× (1) + (−4)



=

 4 (

4 −1

) + (

1 −2

) 4 (

3 1

) + (

3 −4

)  = ( αx0 + y0αx1 + y1

)

http://z.cs.utexas.edu/wiki/pla.wiki/ 9

• Theorem

Let x, y ∈ Rn, α ∈ R, and partition

x =

 x0 x1 ...

xN−1

 and y = 

y0 y1 ...

yN−1

 , where xi and yi have the same size, for i = 0, . . . , N − 1.

http://z.cs.utexas.edu/wiki/pla.wiki/ 10

• Theorem (continued)

Then

αx+ y = α

 x0 x1 ...

xN−1

+ 

y0 y1 ...

yN−1



=

 αx0 + y0 αx1 + y1

...

αxN−1 + yN−1



http://z.cs.utexas.edu/wiki/pla.wiki/ 11

• Partitioning matrices

A =

 A00 a01 A02aT10 α11 aT12 A20 a21 A22

 =  −1 2 4 1 0

1 0 −1 −2 1 2 −1 3 1 2 1 2 3 4 3 −1 −2 0 1 2

Notice how the labels “A”, “a”, “α” are used.

Why do we use the label “aT10”?

http://z.cs.utexas.edu/wiki/pla.wiki/ 12

• Example: blocked matrix-vector multiplication

Consider

A =

 A00 a01 A02aT10 α11 aT12 A20 a21 A22

 =  −1 2 4 1 0

1 0 −1 −2 1 2 −1 3 1 2 1 2 3 4 3 −1 −2 0 1 2

 ,

x =

 x0χ1 x2

 = 

1 2 3 4 5

 , and y =  y0ψ1

y2

 ,

where y0, y2 ∈ R2.

http://z.cs.utexas.edu/wiki/pla.wiki/ 13

• Example (continued)

Then Ax =0BBBB@ −1 2 4 1 0

1 0 −1 −2 1 2 −1 3 1 2 1 2 3 4 3 −1 −2 0 1 2

1CCCCA 0BBBB@

1 2

3

4 5

1CCCCA = 0BBBB@

(−1)× (1) + (2)× (2) + (4)× (3) + (1)× (4) + (0)× (5) (1)× (1) + (0)× (2) + (−1)× (3) + (−2)× (4) + (1)× (5) (2)× (1) + (−1)× (2) + (3)× (3) + (1)× (4) + (2)× (5) (1)× (1) + (2)× (2) + (3)× (3) + (4)× (4) + (3)× (5)

(−1)× (1) + (−2)× (2) + (0)× (3) + (1)× (4) + (2)× (5)

1CCCCA

http://z.cs.utexas.edu/wiki/pla.wiki/ 14

• Example (continued)

0BBBB@ (−1)× (1) + (2)× (2) + (4)× (3) + (1)× (4) + (0)× (5)

(1)× (1) + (0)× (2) + (−1)× (3) + (−2)× (4) + (1)× (5) (2)× (1) + (−1)× (2) + (3)× (3) + (1)× (4) + (2)× (5) (1)× (1) + (2)× (2) + (3)× (3) + (4)× (4) + (3)× (5)

(−1)× (1) + (−2)× (2) + (0)× (3) + (1)× (4) + (2)× (5)

1CCCCA

=

0BBBBBB@

„ −1 2

1 0

«„ 1 2

« +

„ 4 −1

« 3 +

„ 1 0 −2 1

«„ 4 5

« `

2 −1 ´„ 1

2

« + (3)3 +

` 1 2

´„ 4 5

« „

1 2 −1 −2

«„ 1 2

« +

„ 3 0

« 3 +

„ 4 3 1 2

«„ 4 5

«

1CCCCCCA

=

0BBBB@ „

3 1

« +

„ 12 −3

« +

„ 4 −3

« 0 + 9 + 14„

5 −5

« +

„ 9 0

« +

„ 31 14

« 1CCCCA =

0BBBB@ 19 −5 23 45 9

1CCCCA

http://z.cs.utexas.edu/wiki/pla.wiki/ 15

• Blocked matrix-vector multiplication

Let A ∈ Rm×n, x ∈ Rn, and y ∈ Rn. Let m = m0 +m1 + · · ·mM−1, mi ≥ 0 for i = 0, . . . ,M − 1; and n = n0 + n1 + · · ·nN−1, nj ≥ 0 for j = 0, . . . , N − 1; and

Partition

A =

0BBB@ A0,0 A0,1 · · · A0,N−1 A1,0 A1,1 · · · A1,N−1

... ...

. . . ...

AM−1,0 AM−1,1 · · · AM−1,N−1

1CCCA ,

x =

0BBB@ x0 x1 ...

xN−1

1CCCA , and y = 0BBB@

y0 y1 ...

yM−1

1CCCA with Ai,j ∈ Rmi×nj , xj ∈ Rnj , and yi ∈ Rmi .

http://z.cs.utexas.edu/wiki/pla.wiki/ 16

• Theorem (continued)

Then0BBB@ y0 y1 ...

yM−1

1CCCA = 0BBB@

A0,0 A0,1 · · · A0,N−1 A1,0 A1,1 · · · A1,N−1

... ...

. . . ...

AM−1,0 AM−1,1 · · · AM−1,N−1

1CCCA 0BBB@

x0 x1 ...

xN−1

1CCCA

=

0BBB@ A0,0x0 +A0,1x1 + · · ·+A0,N−1xN−1 A1,0x0 +A1,1x1 + · · ·+A1,N−1xN−1

...

AM−1,0x0 +AM−1,1x1 + · · ·+AM−1,N−1xN−1

1CCCA

In other words...

yi = N−1∑ j=0

Ai,jxj .

http://z.cs.utexas.edu/wiki/pla.wiki/ 17

• Example (revisited)

Consider

A =

0@ A00 a01 A02aT10 α11 aT12 A20 a21 A22

1A = 0BBBB@ −1 2 4 1 0

1 0 −1 −2 1 2 −1 3 1 2 1 2 3 4 3 −1 −2 0 1 2

1CCCCA ,

x =

0@ x0χ1 x2

1A = 0BBBB@

1 2

3

4 5

1CCCCA , and y = 0@ y0ψ1

y2

1A ,

where y0, y2 ∈ R2.

http://z.cs.utexas.edu/wiki/pla.wiki/ 18

• Example (continued)

Then

y =

0@ y0ψ1 y2

1A = 0@ A00 a01 A02aT10 α11 aT12

A20 a21 A22

1A0@ x0χ1 x2

1A =

0@ A00x0 + a01χ1 +A02x2aT10x0 + α11χ1 + aT12x2 A20x0 + a21χ1 +A22x2

1A

=

0BBBBBB@

„ −1 2

1 0

«