Post on 02-Jan-2016
Matrix Chain Multiplication
a
b
b
c
=
A = a x b matrixB = b x c matrix
matrix multiplication
Multiplying the Matrix
a
b
b
c
=
Time used = Θ(abc)
Naïve Method
for (i = 1; i <= a;i++) { for (j = 1; i <= c;j++) { sum = 0; for (k = 1;k <= b;k++) { sum += A[i][k] * B[k][j]; } C[i][j] = sum; }}
O(abc)
N x N matrix
N x N matrix
A B C
N x
1 m
atrix
Matrix Chain Multiplication
How to compute ABC ?
Matrix Multiplication
• ABC = (AB)C = A(BC)• (AB)C differs from A(BC)?– Same result, different efficiency
• What is the cost of (AB)C?• What is the cost of A(BC)?
(AB)C
N x N N x N
N x1
N x N
N*N*N
N*N*1
N x1
A(BC)
N x N
N x N N x 1
N*N*1
N*N*1
N x 1
N x 1
The Problem
• Input: a1,a2,a3,….,an
– n-1 matrices of sizes– a1 x a2 B1
– a2 x a3 B2
– a3 x a4 B3
– ….– an-1 x an Bn-1
• Output– The order of multiplication– How to parenthesize the chain
Example
• a1 a2 a3 a4 a5 a6
• 10 x 5 x 1 x 5 x 10 x 2B1 B2 B3 B4 B5
INPUT
Possible Output
((B1B2)(B3B4))B5
(B1((B2B3)B4))B5
(B1B2)((B3B4)B5)
And much more…
Consider the Output
(B1B2)((B3B4)B5)
(B1B2)(B3(B4B5))
What do
((B1B2)(B3B4))B5
(((B1B2)B3)B4))B5
have in common?
What do
have in common?
Min cost of
B1 (B2 B3 B4 … Bn-1 )(B1 B2) (B3 B4 … Bn-1 )(B1 B2 B3) (B4 … Bn-1 )…(B1 B2 B3 B4 …) Bn-1
Solving B1 B2 B3 B4 … Bn-1
Min cost of
B1 (B2 B3 B4 … Bn-1 )(B1 B2) (B3 B4 … Bn-1 )(B1 B2 B3) (B4 … Bn-1 )…(B1 B2 B3 B4 …) Bn-1
Solving B1 B2 B3 B4 … Bn-1
Sub problem
Sub problem
Sub problemSub problem
Sub problem
Sub problem
Matrix Chain Multiplication
B1…BN-1
(B1)(B2…BN-1) (B1B2)(B3…BN-1) (B1…)(BN-1)
(B…)(B2…BN-2)… … … …
…
(B2…BN-2) (BN-1)
Deriving the Recurrent
• mcm(l,r)– The least cost to multiply Bl … Br
• The solution is mcm(1,n-1)
The Recurrence
• Initial Case– mcm(x,x) = 0– mcm(x,x+1) = a[x] * a[x+1] * a[x+2]
The Recurrence
min cost of Bl (Bl+1 Bl+2 Bl+3 … Br ) + al•al+1•ar+1
min cost of (Bl Bl+1) (Bl+2 Bl+3 … Br ) + al•al+2•ar+1
min cost of (Bl Bl+1 Bl+2) (Bl+3 … Br ) + al•al+3•ar+1
…
min cost of (Bl Bl+1 Bl+2 Bl+3 …) Br + al•ar•ar+1
mcm(l,r) = min of
The Recurrence
Bl (Bl+1 Bl+2 Bl+3 … Br ) + al•al+1•ar+1
(Bl Bl+1) (Bl+2 Bl+3 … Br ) + al•al+2•ar+1
(Bl Bl+1 Bl+2) (Bl+3 … Br ) + al•al+3•ar+1
…
(Bl Bl+1 Bl+2 Bl+3 …) Br + al•ar•ar+1
mcm(l,r) = min of
mcm(l+1,r)
mcm(l+2,r)
mcm(l+3,r)mcm(l,l+2)
mcm(l,l+1)
mcm(l,r-1)
0
0
+
+
+
+ +
+
+
+
Matrix Chain Multiplication
int mcm(int l,int r) { if (l < r) { minCost = MAX_INT; for (int i = l;i < r;i++) { my_cost = mcm(l,i) + mcm(i+1,r) + (a[l] * a[i+1] * a[r+1]);
minCost = min(my_cost,minCost); } return minCost; } else { return 0; }}
Using bottom-up DP
• Design the table
• M[i,j] = the best solution (min cost) for multiplying Bi…Bj
• The solution is at M[i,n-1]
What is M[i,j]?
• Trivial case– What is m[x,x] ?– No multiplication, m[x,x] = 0
What is M[i,j]?
• Simple case– What is m[x,x+1] ?– BxBx+1
– Only one solution = ax * ax+1 * ax+2
What is M[i,j]?
• General case– What is m[x,x+k] ?– BxBx+1Bx+2…Bx+k
Bx (Bx+1 Bx+2 Bx+3 … Bx-k ) + ax•ax+1•ax+k+1
(Bx Bx+1) (Bx+2 Bx+3 … Bx+k ) + ax•ax+2•ax-k+1
(Bx Bx+1 Bx+2) (Bx+3 … Bx+k ) + ax•ax+3•ax+k+1
…
(Bx Bx+1 Bx+2 Bx+3 …) Bx+k + ax•ax+k•ax+k+1
min of
Filling the Table
1 2 3 4 5 6
1
2
3
4
5
6
M[1,1]
M[1,6] (our
solution)
Filling the Table
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
Trivial case
Filling the Table
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
Arbitrary case
Filling the Table
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
Arbitrary case
Plus a1•a2•a6
Filling the Table
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
Arbitrary case
Plus a1•a3•a6
Filling the Table
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
Arbitrary case
Plus a1•a4•a6
Filling the Table
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
Arbitrary case
Plus a1•a5•a6
Filling the Table
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
Filling the Table
1 2 3 4 5 6
1 0
2 0
3 0
4 0
5 0
6 0
12
34
5
Example
• a1 a2 a3 a4 a5 a6
• 10 x 5 x 1 x 5 x 10 x 2B1 B2 B3 B4 B5
Example
1 2 3 4 5
1 0
2 0
3 0
4 0
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50
2 0
3 0
4 0
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50
2 0 25
3 0
4 0
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50
2 0 25
3 0 50
4 0
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50
2 0 25
3 0 50
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50
2 0 25
3 0 50
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50
2 0 25
3 0 50
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Option 1 = 0 + 25 + 10 x 5 x 5 = 275
Example
1 2 3 4 5
1 0 50
2 0 25
3 0 50
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Option 2 = 50 + 25 + 10 x 1 x 5 = 100 minimal
Example
1 2 3 4 5
1 0 50 100(2)
2 0 25
3 0 50
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Option 2 = 50 + 25 + 10 x 1 x 5 = 100 minimal
(2) means that the minimal
solution is by dividing at B2
Example
1 2 3 4 5
1 0 50 100(2)
2 0 25
3 0 50
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Option 1 = 0+ 50 + 5x 1 x 10 = 100
Example
1 2 3 4 5
1 0 50 100(2)
2 0 25
3 0 50
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Option 1 = 25+ 0 + 5x 5 x 10 = 275
Example
1 2 3 4 5
1 0 50 100(2)
2 0 25 100(2)
3 0 50
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Option 1 is better
Example
1 2 3 4 5
1 0 50 100(2)
2 0 25 100(2)
3 0 50 70(4)
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50 100(2)
200(2)
2 0 25 100(2)
3 0 50 70(4)
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50 100(2)
200(2)
2 0 25 100(2)
80(2)
3 0 50 70(4)
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2
Example
1 2 3 4 5
1 0 50 100(2)
200(2)
140(2)
2 0 25 100(2)
80(2)
3 0 50 70(4)
4 0 100
5 0
a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2