Randomization, Derandomization, and Parallelization --- take the MIS problem as a demonstration

26
1 Randomization, Randomization, Derandomization, and Derandomization, and Parallelization Parallelization --- take the MIS problem as a --- take the MIS problem as a demonstration demonstration Speaker: Hong Chih-Duo Speaker: Hong Chih-Duo Advisor: Chao Kuen-Mao Advisor: Chao Kuen-Mao National Taiwan University National Taiwan University Department of Information Engineering Department of Information Engineering

description

Randomization, Derandomization, and Parallelization --- take the MIS problem as a demonstration. Speaker: Hong Chih-Duo Advisor: Chao Kuen-Mao National Taiwan University Department of Information Engineering. The MIS problem. Finding a maximum independent set is NPC - PowerPoint PPT Presentation

Transcript of Randomization, Derandomization, and Parallelization --- take the MIS problem as a demonstration

Page 1: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

11

Randomization, Derandomization, Randomization, Derandomization, and Parallelization and Parallelization

--- take the MIS problem as a demonstration--- take the MIS problem as a demonstration

Speaker: Hong Chih-DuoSpeaker: Hong Chih-Duo

Advisor: Chao Kuen-MaoAdvisor: Chao Kuen-Mao

National Taiwan UniversityNational Taiwan UniversityDepartment of Information EngineeringDepartment of Information Engineering

Page 2: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

2

The MIS problemThe MIS problem

►Finding a maximum independent set is NPC►The decision version of MIS is to decide the inde

pendence number α(G) of G -- still NPC►However, there are many good algorithms to app

roximate it, in various settings.►A naive greedy algorithm guarantees a (Δ/(Δ+

1))|OPT| solution. (cf. your textbook!)►Could tossing coins help in this respect?

Page 3: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

3

A simple randomized algorithmA simple randomized algorithm

►For each vertex v, add v to I’ with probability p. (p is to be determined later)

►For each edge vu that both v, u I’, remove one ∊of them from I’ uniformly at random.

►The resulting I” is an independent set of G.

Page 4: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

4

Step 1Step 1

Step 2Step 2

Page 5: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

5

Good luck Bad luck Step 3Step 3

Page 6: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

6

Average performance analysisAverage performance analysis

i

( ) ( )

2

2

1, if i 'Let random variable X

0, o.w.

Let

be a pessimistic estimate for the size of ".

Then [ ] .

Choose to maximize [ ] : [ ] .2 4

i i ji V G ij E G

i i j

I

Z X X X

I

E Z nE X mE X X np mp

n np E Z E Z

m m

Page 7: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

7

Average performance analysisAverage performance analysis

2

:

For any graph G with n vertices and m edges,

G has an independent set of size n 4 .

:

Could we find this set efficiently?

Theorem

m

Question

Page 8: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

8

Some refreshers from probability TheorySome refreshers from probability Theory

1 1

|

|

| , st. | .

| ,..., ,...,

|

x

n n

P X x A P X x A P A

E X A xP X x A

E Y X f X f x E Y X x

E Y X X f X X

X Y E X Y E X

Page 9: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

9

Some refreshers from probability TheorySome refreshers from probability Theory

1 1 1 1

1

1 1

1 1

Corollary:

| ,..., | ,..., | ,...,

Since is fully determined by rvs ,..., ,

we may define ,..., | ,..., .

It then follows from the corollay that

,..., ,

r r r

n

n n

r r

E E Z X X X X E Z X X

Z X X

f X X E Z X X Z

E E f x x X

1,..., .rE f x x

► Theorem

Page 10: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

10

1 1

1

1 11,0

1 11,0

1 1

1

Let denote the event ,..., .

|

| , | (by corollary)

| , |

| , (independency)

| , 1 | , 0 1

max | , 1 , | ,

r r r

r

r r r

r r r ri

r r ri

r r r r

r r r

A X x X x

E Z A

E E Z A X A

E Z A X i P X i A

E Z A X i P X i

E Z A X p E Z A X p

E Z A X E Z A

1 0rX

Effect of a single toss on E[Z]Effect of a single toss on E[Z]

Page 11: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

11

1

1

1 10 1

21 1 1 0

1

(recall that )

But | , could be computed

in time!

If for each 0, we select 0,1 such that

| max | , | ,

then

| | 4 ,

r r

v v uv V vu E

r

r r r ri

E Z A X i

O m n Z X X X

n r x

E Z A E Z A X i E Z A

E f X x E Z A E Z A E Z n m

E f X

21 2 2 2 1

21 1 1

, | | 4 ,...

,..., | | 4 .n n n n

x X x E Z A E Z A n m

E f X x X x E Z A E Z A n m

Page 12: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

12

A derandomized resultA derandomized result

►We derived a deterministic procedure to find an assignment {x1, x2,..., xn} that guarantees an indepen

dent set of size f(x1,...,xn) ≥ n2/4m.

►Note that we may have to prune I’ = {i : xi =1}

in order to get an independent set. (why?)►The argument is in fact from a general scheme call

ed the conditional probability method, which is very powerful in derandomizing probabilistic proofs.

Page 13: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

13

Notes on conditional probability methodsNotes on conditional probability methods

► In general, it is hard to compute conditional expectations.

►There are many instance where there is no efficient way to compute the required conditional expectation.

►Moreover, the conditional probability method is inherently sequential: the variables xi’s are determined in a fixed order. As a result, the time complexity is Ω(n) even if we have an unbounded number of processors.

►Example

►PRAM

Page 14: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

14

The length of computation path is The length of computation path is ΩΩ(n)(n)

X1=0

X2=1

Xn-1=1Xn=0

X3=0

Good points

height = n

Page 15: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

15

““Compress” the computation pathCompress” the computation path

► It appears that the bottleneck lies in the number of random variables {Xi}. Do we really need n variables?

►Recall the definition of Z:

wherein X1,...,Xn are i.i.d..

►Note that if X1,...,Xn are pairwise independent, then also E[Z] = n2/4m.

►We may generate pairwise independent X1,...,Xn with fewer i.i.d. random variables!

i i ji V ij E

Z X X X

Page 16: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

16

““Compress” the computation pathCompress” the computation path

1

i

1

Let ,..., be i.i.d. rvs, where lg ,

and each W is uniform distributed on [1..2m].

Redefine

1, if (mod 2 )

0, o.w.

where H is a subset of [1..2m] with size n

k

k it tt

i

W W k n

W m HIX

1

, and

,..., is the binary representation of .i ikI I i

1 1We may write ( ,..., ) as ( ,...., ).n kZ f X X Z h W W

Page 17: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

17

1 1

1

1 1[1..2 ]

1 1[1..2 ]

1[1..2 ]

[1..2 ]

Let denote the event ,..., .

|

| , | (by corollary)

| , |

| , (independency)

1| ,

2

max | ,

r r r

r

r r r

r r r ri m

r r ri m

r ri m

ri m

A W W

E Z A

E E Z A W A

E Z A W i P W i A

E Z A W i P W i

E Z A W im

E Z A

1rW i

““Compress” the computation pathCompress” the computation path

Page 18: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

18

1

1

1 1[1..2 ]

But | , could be computed in

( ) lg time with a single processor,

lg time with ( ) processors.

If for each 0, we select [1..2 ] such that

| max | ,

r r

r

r r ri m

E Z A W i

O m n n

O n O m n

n r m

E Z A E Z A W i

21 1 1 0

21 1 2 2 2 1

21 1 1

| ,

then

| | 4 ,

, | | 4 ,...

,..., | | 4 .

r

k k k k

E Z A

E h W E Z A E Z A E Z n m

E h W W E Z A E Z A n m

E h W W E Z A E Z A n m

Page 19: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

19

The length of computation path is now O(lgThe length of computation path is now O(lgn)n)

W1=ω1

W2=ω2

Wk-1= ωk-1Wk=ωk

W3=ω3

Good points

height = k

= ceiling(lg n)

Page 20: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

20

A parallelized resultA parallelized result

►We derived a deterministic parallel algorithm to find an independent set of size h(ω1,..., ωk) ≥ n2/4m, for a graph with n vertices and m edges.

►This algorithms can be implemented on an EREW-PRAM in O(lg2n) time with O(m2) processors.

►There is a high-level theorem indicating this fact:

If an RNC-algorithm works properly when the proIf an RNC-algorithm works properly when the probabilities are babilities are suitablysuitably approximated, then it can be approximated, then it can be converted to an equivalent NC-algorithm.converted to an equivalent NC-algorithm.

Page 21: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

21

ReferenceReference

Fast Parallel Algorithms for Graph Matching Problems, M.Karpinski, W.Ryttter, p.104~115.

The Probabilistic Method, 2nd Edition, Noga Alon, J.Spencer, p.249 ~ 257.

Randomized Algorithms, R.Motwani, P.Raghavan, p.335~346.

Page 22: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

22

Example: looking for the biggest determinantExample: looking for the biggest determinant

►An ∊ Mn[{+1,-1}]

►How big can | det(An) | be ??

►This is a famous (and unsolved) problem of Hadamard.

►Fact: |det(An)| ≦ nn/2. a corollary of Hadamard’s determinant theorem

Page 23: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

23

Example: looking for the biggest determinantExample: looking for the biggest determinant

►Let’s toss coins!

►(M. Kac) A random matrix An of {+1, -1} has

E[ |det(AE[ |det(Ann)|)|22 ] = n! ] = n!

►So there exists an n×n matrix An that satisfies |det(An)| ≧ (n!)1/2.

►However, no one knows how to construct one efficiently.

Back

Page 24: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

24

:

| | |

:

| |

| , |

, |

( , | )

| .

x

y x

y x

x y

x

Theorem

E E X Y Z E X Z

proof

LHS E xP X x Y Z

xP X x Y y Z P Y y Z

xP X x Y y Z

x P X x Y y Z

xP X x Z RHS

► Corollary

Page 25: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

25

Some notes on parallel computation modelsSome notes on parallel computation models

►PRAM: a very general model that is interested mostly in exposing the parallel nature of problems.

►PRAM consists of a number of RAMs that work synchronous and communicate through a common random access memory.

►Typically, technical details such as synchroniza-tion and communication problems are ignored.

►The most “realistic” PRAM variant is EREW, allowing only exclusive-read and exclusive-write on the shared memory.

Page 26: Randomization, Derandomization, and Parallelization  --- take the MIS problem as a demonstration

26

Some notes on parallel complexity classesSome notes on parallel complexity classes

►The main aim of parallel computing is the decrease of computation time. The main class of interests is NC = { problems computable in polylogarithmic time and polynomially many processors. }

►Perhaps the most important question in the theory of parallel computations is: is P = NC ?

► It is strongly believed that the answer is negative. However, this question could be of similar difficulty to the P = NP problem.

Back