Section 1.5 - Exploiting partitioning of matrices and · PDF file Section 1.5 - Exploiting...

Click here to load reader

  • date post

    17-Oct-2020
  • Category

    Documents

  • view

    1
  • download

    0

Embed Size (px)

Transcript of Section 1.5 - Exploiting partitioning of matrices and · PDF file Section 1.5 - Exploiting...

  • Section 1.5 - Exploiting partitioning of matrices and vectors

    Maggie Myers Robert A. van de Geijn

    The University of Texas at Austin

    Practical Linear Algebra – Fall 2009

    http://z.cs.utexas.edu/wiki/pla.wiki/ 1

  • Example: Partitioning vectors

    Given a vector

    x =

     4 −1 3 1

     one can think of this as its four elements

    x =

     χ0 χ1 χ2 χ3

     where χ0 = 4, χ1 = −1, etc. Note

    The parentheses are only there to delimit (outline) the vector. They have no particular other meaning.

    http://z.cs.utexas.edu/wiki/pla.wiki/ 2

  • Example: Partitioning vectors

    Given a vector

    x =

     4 −1 3 1

     one can think of this as two subvectors:

    x = ( x0 x1

    ) =

     (

    4 −1

    ) (

    3 1

    )  =

     4 −1 3 1

     so that

    x0 = (

    4 −1

    ) and x1 =

    ( 3 1

    )

    http://z.cs.utexas.edu/wiki/pla.wiki/ 3

  • Example: Partitioning vectors

    Given a vector

    x =

     4 −1 3 1

     one can think of this as two subvectors:

    x = ( x0 x1

    ) =

     (

    4 ) −13

    1

      =

     4 −1 3 1

     so that

    x0 = (

    4 )

    and x1 =

     −13 1

     http://z.cs.utexas.edu/wiki/pla.wiki/ 4

  • Example: Partitioning vectors

    Given a vector

    x =

     4 −1 3 1

     one can think of this as two subvectors:

    x = ( x0 x1

    ) =

     ( )

    4 −1 3 1

      =

     4 −1 3 1

     so that

    x0 = ( )

    and x1 =

     4 −1 3 1

     http://z.cs.utexas.edu/wiki/pla.wiki/ 5

  • Example: Inner product with partitioned vectors

    Given vector

    x = ( x0 x1

    ) =

     4 −1 3 1

     and y = ( y0y1 )

    =

     1 −2 3 −4

     We find that

    xT y = (4)× (1) + (−1)× (−2)︸ ︷︷ ︸ = (

    4 −1

    )T ( 1 −2

    ) = xT0 y0

    + (3)× (−3) + (1)×−4)︸ ︷︷ ︸ = (

    3 1

    )T ( 3 −4

    ) = xT1 y1︸ ︷︷ ︸

    = xT0 y0 + x T 1 y1

    http://z.cs.utexas.edu/wiki/pla.wiki/ 6

  • Theorem

    Let x, y ∈ Rn and partition

    x =

     x0 x1 ...

    xN−1

     and y = 

    y0 y1 ...

    yN−1

     , where xi and yi have the same size, for i = 0, . . . , N − 1.

    http://z.cs.utexas.edu/wiki/pla.wiki/ 7

  • Theorem (continued)

    Then

    xT y =

     x0 x1 ...

    xN−1

     T 

    y0 y1 ...

    yN−1

    

    = ( xT0 x

    T 1 · · · xTN−1

    ) 

    y0 y1 ...

    yN−1

     = xT0 y0 + x

    T 1 y1 + · · ·+ xTN−1yN−1

    http://z.cs.utexas.edu/wiki/pla.wiki/ 8

  • Example: axpywith partitioned vectors

    Given

    x = ( x0 x1

    ) =

     4 −1 3 1

     , y = ( y0y1 )

    =

     1 −2 3 −4

     , and α = 4 We find that

    αx+ y = 4

     4 −1 3 1

    + 

    1 −2 3 −4

     = 

    (4)× (4) + (1) (4)× (−1) + (−2)

    (4)× (3) + (3) (4)× (1) + (−4)

    

    =

     4 (

    4 −1

    ) + (

    1 −2

    ) 4 (

    3 1

    ) + (

    3 −4

    )  = ( αx0 + y0αx1 + y1

    )

    http://z.cs.utexas.edu/wiki/pla.wiki/ 9

  • Theorem

    Let x, y ∈ Rn, α ∈ R, and partition

    x =

     x0 x1 ...

    xN−1

     and y = 

    y0 y1 ...

    yN−1

     , where xi and yi have the same size, for i = 0, . . . , N − 1.

    http://z.cs.utexas.edu/wiki/pla.wiki/ 10

  • Theorem (continued)

    Then

    αx+ y = α

     x0 x1 ...

    xN−1

    + 

    y0 y1 ...

    yN−1

    

    =

     αx0 + y0 αx1 + y1

    ...

    αxN−1 + yN−1

    

    http://z.cs.utexas.edu/wiki/pla.wiki/ 11

  • Partitioning matrices

    A =

     A00 a01 A02aT10 α11 aT12 A20 a21 A22

     =  −1 2 4 1 0

    1 0 −1 −2 1 2 −1 3 1 2 1 2 3 4 3 −1 −2 0 1 2

     Pronounce a21 as a-two-one instead of a-twentyone, please.

    Notice how the labels “A”, “a”, “α” are used.

    Why do we use the label “aT10”?

    http://z.cs.utexas.edu/wiki/pla.wiki/ 12

  • Example: blocked matrix-vector multiplication

    Consider

    A =

     A00 a01 A02aT10 α11 aT12 A20 a21 A22

     =  −1 2 4 1 0

    1 0 −1 −2 1 2 −1 3 1 2 1 2 3 4 3 −1 −2 0 1 2

     ,

    x =

     x0χ1 x2

     = 

    1 2 3 4 5

     , and y =  y0ψ1

    y2

     ,

    where y0, y2 ∈ R2.

    http://z.cs.utexas.edu/wiki/pla.wiki/ 13

  • Example (continued)

    Then Ax =0BBBB@ −1 2 4 1 0

    1 0 −1 −2 1 2 −1 3 1 2 1 2 3 4 3 −1 −2 0 1 2

    1CCCCA 0BBBB@

    1 2

    3

    4 5

    1CCCCA = 0BBBB@

    (−1)× (1) + (2)× (2) + (4)× (3) + (1)× (4) + (0)× (5) (1)× (1) + (0)× (2) + (−1)× (3) + (−2)× (4) + (1)× (5) (2)× (1) + (−1)× (2) + (3)× (3) + (1)× (4) + (2)× (5) (1)× (1) + (2)× (2) + (3)× (3) + (4)× (4) + (3)× (5)

    (−1)× (1) + (−2)× (2) + (0)× (3) + (1)× (4) + (2)× (5)

    1CCCCA

    http://z.cs.utexas.edu/wiki/pla.wiki/ 14

  • Example (continued)

    0BBBB@ (−1)× (1) + (2)× (2) + (4)× (3) + (1)× (4) + (0)× (5)

    (1)× (1) + (0)× (2) + (−1)× (3) + (−2)× (4) + (1)× (5) (2)× (1) + (−1)× (2) + (3)× (3) + (1)× (4) + (2)× (5) (1)× (1) + (2)× (2) + (3)× (3) + (4)× (4) + (3)× (5)

    (−1)× (1) + (−2)× (2) + (0)× (3) + (1)× (4) + (2)× (5)

    1CCCCA

    =

    0BBBBBB@

    „ −1 2

    1 0

    «„ 1 2

    « +

    „ 4 −1

    « 3 +

    „ 1 0 −2 1

    «„ 4 5

    « `

    2 −1 ´„ 1

    2

    « + (3)3 +

    ` 1 2

    ´„ 4 5

    « „

    1 2 −1 −2

    «„ 1 2

    « +

    „ 3 0

    « 3 +

    „ 4 3 1 2

    «„ 4 5

    «

    1CCCCCCA

    =

    0BBBB@ „

    3 1

    « +

    „ 12 −3

    « +

    „ 4 −3

    « 0 + 9 + 14„

    5 −5

    « +

    „ 9 0

    « +

    „ 31 14

    « 1CCCCA =

    0BBBB@ 19 −5 23 45 9

    1CCCCA

    http://z.cs.utexas.edu/wiki/pla.wiki/ 15

  • Blocked matrix-vector multiplication

    Let A ∈ Rm×n, x ∈ Rn, and y ∈ Rn. Let m = m0 +m1 + · · ·mM−1, mi ≥ 0 for i = 0, . . . ,M − 1; and n = n0 + n1 + · · ·nN−1, nj ≥ 0 for j = 0, . . . , N − 1; and

    Partition

    A =

    0BBB@ A0,0 A0,1 · · · A0,N−1 A1,0 A1,1 · · · A1,N−1

    ... ...

    . . . ...

    AM−1,0 AM−1,1 · · · AM−1,N−1

    1CCCA ,

    x =

    0BBB@ x0 x1 ...

    xN−1

    1CCCA , and y = 0BBB@

    y0 y1 ...

    yM−1

    1CCCA with Ai,j ∈ Rmi×nj , xj ∈ Rnj , and yi ∈ Rmi .

    http://z.cs.utexas.edu/wiki/pla.wiki/ 16

  • Theorem (continued)

    Then0BBB@ y0 y1 ...

    yM−1

    1CCCA = 0BBB@

    A0,0 A0,1 · · · A0,N−1 A1,0 A1,1 · · · A1,N−1

    ... ...

    . . . ...

    AM−1,0 AM−1,1 · · · AM−1,N−1

    1CCCA 0BBB@

    x0 x1 ...

    xN−1

    1CCCA

    =

    0BBB@ A0,0x0 +A0,1x1 + · · ·+A0,N−1xN−1 A1,0x0 +A1,1x1 + · · ·+A1,N−1xN−1

    ...

    AM−1,0x0 +AM−1,1x1 + · · ·+AM−1,N−1xN−1

    1CCCA

    In other words...

    yi = N−1∑ j=0

    Ai,jxj .

    http://z.cs.utexas.edu/wiki/pla.wiki/ 17

  • Example (revisited)

    Consider

    A =

    0@ A00 a01 A02aT10 α11 aT12 A20 a21 A22

    1A = 0BBBB@ −1 2 4 1 0

    1 0 −1 −2 1 2 −1 3 1 2 1 2 3 4 3 −1 −2 0 1 2

    1CCCCA ,

    x =

    0@ x0χ1 x2

    1A = 0BBBB@

    1 2

    3

    4 5

    1CCCCA , and y = 0@ y0ψ1

    y2

    1A ,

    where y0, y2 ∈ R2.

    http://z.cs.utexas.edu/wiki/pla.wiki/ 18

  • Example (continued)

    Then

    y =

    0@ y0ψ1 y2

    1A = 0@ A00 a01 A02aT10 α11 aT12

    A20 a21 A22

    1A0@ x0χ1 x2

    1A =

    0@ A00x0 + a01χ1 +A02x2aT10x0 + α11χ1 + aT12x2 A20x0 + a21χ1 +A22x2

    1A

    =

    0BBBBBB@

    „ −1 2

    1 0

    «