Divide and Conquer (Part II) Multiplication of two ... and Conquer (Part II) Multiplication of two...

Divide and Conquer (Part II)

Multiplication of two numbersMultiplication of two numbersLet U = (u2n-1u2n-2 … u1u0)2 and V = (v2n-1v2n-2…v1v0)2, and our

l i fi d U i Vgoal is to find U times V.

Ordinary multiplication requires execution time α (n2).

Alternatively, let U = 2nU1 + U0 and V = 2nV1 + V0 where

U1 = (u2n-1u2n-2 … un)2 and U0 = (un-1un-2 … u0)2

Likewise, V1 = (v2n-1v2n-2 … vn)2 and V0 = (vn-1vn-2 … v0)2

U X V = 22nU1V1 + 2n(U1V0 + U0V1) + U0V0.

Apparently there is no saving, this multiplication also requires execution time proportional to n2.

1/21/2010


Multiplication: Divide and ConquerMultiplication: Divide and ConquerU X V = (22n+ 2n)U1V1 + 2n(U1 – U0 )( V0 – V1) + (2n+ 1)

U0V0.

which requires only three multiplications and some extra ddi iaddition.

If T(n) denotes time to multiply two binary integers of size h thn each, then

T(2n) = 3 T(n) + cn, and T(1) =1,

where cn denotes the cost associated with additions and shifting of binary integers etc..

1/21/2010


Multiplication: Divide and ConquerMultiplication: Divide and Conquer

Solution of this recurrence equation is (by Master Theorem) or q ( y )repeated substitution, for n = 2m, is

T(2m) = 3mT(1) + c(3m-2m)

In general,

T(n) ≤ 3⎡lg n⎤ T(1) + c(3⎡lg n⎤ - 2⎡lg n⎤ )T(n) ≤ 3 T(1) + c(3 2 )

≤ 3c X 3lg n

3 X lg 3 3 X 1 585= 3c X nlg 3 ≈ 3c X 1.585

1/21/2010


Matrix MultiplicationMatrix MultiplicationLet C = A B = (cij) be the product of two n x n matrices A and B,

hj

wherea11 a12 … a1n b11 b12 … b1n

A d B b b bA = a21 a22 … a2n and B = b21 b22 … b2n

… … … … … … … …b b ban1 an2 … ann bn1 bn2 … bnn

• We calculate n2 terms, cij for i =1,2,…,n, , j =1,2,…,n. • Each c requires n scalar multiplications• Each cij requires n scalar multiplications.• Therefore, total complexity of this matrix multiplication is of

the order n3.

1/21/2010


Suppose n = 2k for some integer value of k. Letpp gr … s a … b e … g

C = … … … = A X B = … … … X … … …

t … u c … d f … h

Where r,s,t,u are all n/2 X n/2 matrices

• In this way of obtaining C, by first partitioning A and B in y g , y p gsubmatrices, we calculate 4 matrices r,s,t, and u.

1/21/2010


Suppose n = 2k for some integer value of k. Letpp g

• Each submatrix has two matrix multiplications, of size n/2 X n/2 each.

Hence, the recurrence equation associated with this divide and conquer approach would satisfy:

T(n) = 8T(n/2) + (n2)

T(n) =Θ(n3)( ) ( )

Thus, there is no saving.

1/21/2010


Matrix Multiplication: Strassen’s AlgorithmMatrix Multiplication: Strassen s Algorithm

P1 = a (g-h) = ag - ah1 (g ) gP2 = (a+b)h = ah + bhP = (c+d) e = ce + deP3 (c+d) e ce + deP4 = d (f-e) = df - deP ( +d)( +h) + h + d + dhP5 = (a+d)(e+h)= ae + ah + de + dhP6 = (b-d)(f+h) = bf + bh - df – dhandP7 = (a-c) (e+g) = ae + ag – ce - cg

1/21/2010


Then, we can obtain r,s,t, and u as follows:r = P5+P4 - P2+P6

s = P1 + P2

t = P3 +P4

u = P5+P1 - P3-P7

For example, P +P P +PP5+P4 - P2+P6

= (ae+ah+de+dh)+(df - de) - (ah + bh) + (bf + bh - df -dh)= ae+bf ae+bf= r

1/21/2010


Hence, by Master Theorem,, y ,

T(n) = 7T(n/2) + Θ(n2)l 7= Θ(nlg 7)

= Θ(n2.81)

In other words, it is possible to multiply two matrices at a rate faster than Θ(n3).matrices at a rate faster than Θ(n ).

1/21/2010


27 99 0 8 13 64 86 16 7 ↑ ↑ ↑ i x j j

27 99 0 8 13 64 86 16 7 ↑ ↑ i j

7 99 0 8 13 64 86 16 277 99 0 8 13 64 86 16 27↑ ↑

i j

7 99 0 8 13 64 86 16 27↑ ↑ i j

7 16 0 8 13 64 86 99 27↑ ↑↑ ↑

i j

7 16 0 8 13 64 86 99 27↑ ↑

1/21/2010

i j


Method 2:Method 2:

1. Choose x = A[1].1. Choose x A[1].

2. Set i ← 2, and set j ← n.

3. If A[j] > x, then j ← j -1, else go to step 4.

4. If A[i] < x, then i ← i +1, else go to step 5.[ ] , , g p

5. If i < j, then interchange A[i] and A[j], otherwise set q ← j and interchange A[1] and A[q]q ← j and interchange A[1] and A[q].

1/21/2010


27 99 0 8 13 64 86 16 7 ↑ ↑ ↑↑ ↑ ↑

i x j

27 7 0 8 13 64 86 16 9927 7 0 8 13 64 86 16 99 ↑ ↑ i j

27 7 0 8 13 16 86 64 99↑ ↑ ↑ x j i

16 7 0 8 13 27 86 64 99↑

1/21/2010

↑ x


Method 3:Method 3:

1. To partition A[1],…,A[n] this method proceeds as follows:p [ ], , [ ] p

2. Set i =1, j = n, X = A[1]

3 Compare A[i] and A[j] and exchange if A[i] ≥ A[j]3. Compare A[i] and A[j], and exchange if A[i] ≥ A[j].

4. If no exchange is required then decrease j by 1 and continue. If exchange is required then increase i by 1 after theIf exchange is required then increase i by 1 after the exchange and continue. If i = j, then stop, else

5. Go to step 2.p

To see how this method works suppose we wish to partition 503, 087, 512, 061, 908, 170, and 897.

1/21/2010


503 087 512 061 908 170 897↑ ↑

x,i j After 1st interchange - g

170 087 512 061 908 503 897 ↑ ↑ i j Aft 2nd i t hAfter 2nd interchange -

170 087 503 061 908 512 897 ↑ ↑

i j

After 3rd interchange -

170 087 061 503 908 512 897 ↑ ↑ i j

Since i = j, the algorithm stops to give the following partition 170 087 061 503 908 512 897

1/21/2010

170, 087, 061, 503, 908, 512, 897left sub-aray x right sub-array


Quick-Sort A[1], … , A[n] algorithm is:

1. Find a partition of A[1], … , A[n] such that A[1], … , A[q] are all smaller than (new) A[q+1], … , A[n] for 1 ≤ q ≤ (n 1)A[n] for 1 ≤ q ≤ (n -1)

2. Quick-Sort A[1], … ,A[q].

3. Quick-Sort A[q + 1], … , A[n].

1/21/2010


Comments:

1. All comparisons are made with the same element. Amount of data movement is reasonable. This makes these partition proced res reasonable for large al es ofprocedures reasonable for large values of n.

2. Other choices for X are A[n], A[ ⎣n/2⎦ ] and randomly chosen value i.e., A[i].c ose va ue .e., [i].

3. It makes sense to sort an array by special procedures such as Bubble-Sort, for small values of n, say n ≤ M. Some

l d h i h M 9analyses and other supporting arguments suggest that M ≈ 9 is a reasonable bound for shifting from Quick-Sort to Bubble-Sort.

1/21/2010


Analysis of Quick-sort:y Q

• Exact analysis of Quick-sort is not possible --- we cannot control the size of the sub-arrays.

• We will consider the average analysis of Quick-sort.

Assumptions:

• A[1], … , A[n] are all distinct.

• A[i]'s are randomly distributed.

• A[i] ∈ {1,2, … , n} for i = 1, … , n.

• The partition Method #3 is used.p

1/21/2010


Observations:

1. All n! permutations of A[1], … , A[n] are equally likely.

2. Suppose A[i] = k. Then after one application of the partition method, size of the left sub-array is (k - 1),

3. size of the right sub-array is (n - k) and k is in its correct positionposition.

4. Pr (A[i] = k) = 1/n.

5 E tl ( + 1) i d i i th5. Exactly (n + 1) comparisons are made in arranging the given array A[1], …, A[n] in partition method 2. [Reason - Each time i is increased by 1 or j is decreased b 1 i i d til i b l t j ]by 1 a comparison is made until i becomes equal to j.]

1/21/2010


Suppose AC(n) = Average number of comparisons required to sort A[1], … , A[n] by Quick-sort.

ThThen,

AC(n) = (n + 1) + (1/n){ [AC(k - 1) + AC(n - k)]}

= (n + 1) + (1/n){ [AC(k) + AC(n k 1)]}

∑=

n

k 1

∑−1n

= (n + 1) + (1/n){ [AC(k) + AC(n - k - 1)]}

= (n + 1) + AC(k)

Consequently

∑= 0k

∑−

=

1

0

2 n

knConsequently,

n AC(n) = n (n + 1) + 2 [AC(0) + … + AC(n - 1)].

Substituting (n + 1) for n in the above equation gives,g ( ) q g ,

(n + 1) AC(n + 1)

= (n + 1) (n + 2) + 2 [AC(0) + … + AC(n - 1) + AC(n)],

1/21/2010


ibhfdiffhd

lyConsequent).(2)1(2)()1()1(

:istwoabovetheofdifferencetheand++=−++ nACnnnACnACn

or,),()2()1(2)1()1(

ly,Consequent+++=++ nACnnnACn

)1(

.)1()(

22

)2()1(

++

+=

++

ACn

nACnn

nAC

)(2)1(

,Then.)2(

)1()1(Let+

+=+n

nACng

,givesonsubstitutiRepated

.)(2

2)1( ++

=+ ngn

ng

1/21/2010

)1(1

22

2)1( −++

++

=+ ngnn

ng


)2(21

22

2 −+++

++

= ngnnn

M

h f3)2(0)1(1)0(B

)2(422

12

22 ++++

++

+=

ACACAC

gnnn

L

Hence,.1)2(

therefore,3)2(,0)1(,1)0(But

3

3

3

)2( ===

===

g

ACACACAC

33

41

11

212)1(

33

+⎭⎬⎫

⎩⎨⎧ ++

++

+=+

nnng L

11

121

312 2

⎞⎛

+⎟⎠⎞

⎜⎝⎛ −−= +nH

1/21/2010

16

112 2 +⎟⎠⎞

⎜⎝⎛ −= +nH


Fi ll

16

112)1()(

Finally,

1HnnAC n

⎭⎬⎫

⎩⎨⎧ +⎟

⎠⎞

⎜⎝⎛ −+= +

382)1(

6

1Hn n ⎥⎦⎤

⎢⎣⎡ −+=

⎭⎩ ⎠⎝

+

( ) ( )13812

3

1 nHn n +−+=

⎥⎦⎢⎣

+

( ) .log)(,logSince3

nnnACnHn θ=≈

1/21/2010


Selection ProblemSelection ProblemFind the i-th smallest element of a given array A[1], A[2], …,A[n]}

The algorithm described below borrows an idea from Quick-Sort---how to partition an array into two subarrays.

Al ithAlgorithm1. Call A[1] the pivotal element.

2. Use a partition procedure, as in Quick-Sort, to find the location of the pivotal element, k, in the partitioned array (in which all elements to the left of the pivotal element are smaller than it pand to the right are larger than it).

3. If i = k, then we have found the i-th smallest element; the i ( ld) A[1]

1/21/2010

answer is (old) A[1].


4. If 1 ≤ i ≤ k-1, then find the i-th smallest element among (new) A[1] A[k-1] else(new) A[1], … , A[k 1], else

5. if k<i ≤ n, then find the i-th smallest element among new A[k+1], … ,A[n]. (This case can be seen as [ ], , [ ] (equivalent to finding the (i-k)-th smallest element out of (n-k) of the right subarray after renaming A[k+1], … , A[n] as A[1] A[n k])A[n] as A[1], … ,A[n-k]).

Analysis of the above algorithm

Please see the book

1/21/2010

Divide and ConquerDivide and Conquer

Questions

1/21/2010

Divide and Conquer (Part II) Multiplication of two ... and Conquer (Part II) Multiplication of two...

Documents

Transcript of Divide and Conquer (Part II) Multiplication of two ... and Conquer (Part II) Multiplication of two...