An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and...

39
An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with : An Introduction to Parallel Computation and Ρ-Completeness Theory by Raymond Greenlaw Jim Hoover Computing Science Department University of Alberta Larry Ruzzo Department of Computer Science University of Washington

Transcript of An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and...

Page 1: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

1

Joint work with:

An Introduction toParallel Computation

and Ρ-Completeness Theory

by

Raymond Greenlaw

An Introduction toParallel Computation

and Ρ-Completeness Theory

by

Raymond Greenlaw

Jim HooverComputing Science Department

University of Alberta

Larry Ruzzo Department of Computer

ScienceUniversity of Washington

Page 2: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

2

OutlineOutline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems

Page 3: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

3

OutlineOutline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems

Page 4: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

4

IntroductionIntroduction

• Sequential computation:

Feasible ~ n O(1) time(polynomial time).

• Parallel computation:

Feasible ~ n O(1) time and ~ n O(1) processors.

Page 5: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

5

IntroductionIntroduction

• Goal of parallel computation:Develop extremely fast ((log n)O(1) ) time algorithms using a reasonable number of processors.

• Speedup equation: (best sequential time)(number of processors)

Allow a polynomial number of processors.

(parallel time).

Page 6: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

6

IntroductionIntroduction

Roughly,• A problem is feasible if it can be solved by a

parallel algorithm with worst case time and processor complexity n

O(1).• A problem is feasible highly parallel if it can be

solved by an algorithm with worst case time complexity (log n)

O(1) and processor complexity n

O(1).• A problem is inherently sequential if it is

feasible but has no feasible highly parallel algorithm for its solution.

Page 7: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

7

OutlineOutline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems

Page 8: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

8

Parallel Models of ComputationParallel Models of Computation

• Parallel Random Access Machine Model• Boolean Circuit Model• Circuits and PRAMs

Page 9: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

9

Parallel Random Access MachineParallel Random Access Machine

RAM Processors

Global Memory Cells

P0 P1 P2

C0 C1 C2

Page 10: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

10

Boolean Circuit ModelBoolean Circuit Model

1 0 1 10

AND OR

AND

ANDOR

OR

NOT

Page 11: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

11

Circuits and PRAMSCircuits and PRAMS

Theorem:

A function f from {0,1}* to {0,1}* can be computed by a logarithmic space uniform Boolean circuit family {n } with depth (n) є (logn)O(1) and size (n ) є

n O(1)

if and only if

f can be computed by a CREW-PRAM M on inputs of length n in time t(n) є (logn)O(1) using p(n) є n

O(1).

Page 12: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

12

OutlineOutline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems

Page 13: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

13

Basic ComplexityBasic Complexity

• Decision, Function, and Search Problems• Complexity Classes• Reducibility• Completeness

Page 14: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

14

Decision, Function, and Search ProblemsDecision, Function, and Search Problems

4 4

4 4

3 3 366

Spanning Tree-DGiven: An undirected graph G = (V,E) with weights from N labeling edges in E and a natural number k.Problem: Is there a spanning tree of G with cost less than or equal to k ?

Spanning Tree-FGiven: Same (no k ).Problem: Compute the weight of a minimum cost spanning tree.

Spanning Tree-SGiven: Same.Problem: Find a minimum cost spanning tree.

Page 15: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

15

Complexity ClassesComplexity Classes

Definitions:

P is the set of all languages L that are decidable in sequential time n O(1).

NC is the set of all languages L that are decidable in parallel time (logn)O(1) and processors n O(1).

FP is the set of all functions from {0,1}* to {0,1}* that are computable in sequential time n O(1).

FNC is the set of all functions from {0,1}* to {0,1}* that are computable in parallel time (logn)O(1) and processors n O(1).

NC k, k 1, is the set of all languages L such that L is recognized by a uniform Boolean circuit family {n } with size(n

) = n O(1) and depth (n ) = O((logn)k ).

Page 16: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

16

Definitions:

A language L is many-one reducible to a language L’, written L P L’ , if there is a function f such that x є L if and only if f(x) є L’.

L is P many-one reducible to L’, written L P L’ , if the function f is in FP.

For k 1, L is NC k many-one reducible to L’, written L NC kL’, if the function f is in FNC k.

L is NC many-one reducible to L’, written L NC L’, if the function f is in FNC.

Many-One ReducibilityMany-One Reducibility

m

m

m

m

Page 17: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

17

Many-One ReducibilityMany-One Reducibility

m

Lemma:

The reducibilities m, P, NC k(k 1), and

NC are transitive.

– If L’ є P and L P L’, L

NC k L’ (k > 1), or L

NC L’ then L є P.

– If L’ є NC k (k > 1), and L NC kL’ then L є NC k.

– If L’ є NC and L NC L’ then L є NC .

m m m

mm m

m

Page 18: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

18

Turing ReducibilityTuring Reducibility

Definition:

L is NC Turing reducible to L’, written L NC L’, if and only if

the L’-oracle PRAM on inputs of length n uses time (logn)O(1) and processors n O(1).

Lemma:

The reducibility NC is transitive.

– If L NC L’ then L

NC L’.

– If L NC L’ and L’ є NC then L є NC.

– If L NC L’ and L’ є P then L є P.

T

T

m T

T

T

Page 19: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

19

CompletenessCompleteness

Definitions:

A language L is P-hard under NC reducibility if L’ NC L for

every L’ є P.

A language L is P-complete under NC reducibility if L є P and L is P-hard.

Theorem:If any P-complete problem is in NC then NC equals P.

T

Page 20: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

20

OutlineOutline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems

Page 21: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

21

Evidence That NC PEvidence That NC P

• General Simulations Are Not Fast• Fast Simulations Are Not General• Natural Approaches Provably Fail

Page 22: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

22

Evidence That NC PEvidence That NC P

Gaps in Simulations

ModelResource

1Resource

2Max R2C NC

Min R2C P

DTMDTMATMPDAUCPRAM

Time = n O(1)

Time = n O(1)

2Space = n O(1)

2Space = n O(1)

Size = n O(1)

Procs = n O(1)

SpaceReversalslog(Treesize)log(Time)DepthTime

log n(log n)O(1)

(log n)O(1)

(log n)O(1)

(log n)O(1)

(log n)O(1)

n O(1)

n O(1)

n O(1)

n O(1)

n O(1)

n O(1)

Page 23: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

23

OutlineOutline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems

Page 24: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

24

P-Complete ProblemsP-Complete Problems

There are approximately 175 P-complete problems (500 with variations).

Categories:– Circuit complexity– Graph theory– Searching graphs– Combinatorial

optimization and flow– Local optimality– Logic

– Formal languages– Algebra– Geometry– Real analysis– Games– Miscellaneous

Page 25: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

25

Circuit Value ProblemCircuit Value Problem

Given: An encoding of a Boolean circuit , inputs x1,…,xn, and a designated output y.

Problem:Is output y of TRUE on input x1,…,xn?

Theorem: [Ladner 75]The Circuit Value Problem is P-complete under

NC

1

reductions.

m

Page 26: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

26

P-Complete Variations of CVPP-Complete Variations of CVP

– Topologically Ordered [Folklore]– Monotone [Goldschlager 77]– Alternating Monotone Fanin 2, Fanout 2

[Folklore]– NAND [Folklore]– Topologically Ordered NOR [Folklore]– Synchronous Alternating Monotone Fanout

2 CVP [Greenlaw, Hoover, and Ruzzo 87]– Planar [Goldschlager 77]

Page 27: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

27

NAND Circuit Value ProblemNAND Circuit Value Problem

Given: An encoding of a Boolean circuit that consists solely of

NAND gates, inputs x1,…,xn, and a designated output y.

Problem: Is output y of TRUE on input x1,…,xn?

Theorem: The NAND Circuit Value Problem is P-complete.

Page 28: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

28

NAND Circuit Value ProblemNAND Circuit Value Problem

Proof:Reduce AM2CVP to NAND CVP. Complement all inputs. Relabel all gates as NAND.

01 0 0 1

1

1

OR

AND

OROR

1

1

10 1 1 0

1

1

NAND

NAND

NANDNAND

1

0

Page 29: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

29

Graph TheoryGraph Theory

– Lexicographically First Maximal Independent Set [Cook 85]

– Lexicographically First ( + 1)-Vertex Coloring [Luby 84]

– High Degree Subgraph [Anderson and Mayr 84]

– Nearest Neighbor Traveling Salesman Heuristic [Kindervater, Lenstra, and Shmoys 89]

Page 30: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

30

Lexicographically First Maximal Independent Set

Lexicographically First Maximal Independent Set

Theorem: [Cook 85]LFMIS is P-complete.

Proof:Reduce TopNOR CVP to LFMIS. Add new vertex 0. Connect to all false inputs.

1 2 3 4 5

NOR9

NOR8

NOR7

NOR6

1 1 0 0 0

0

1

1 2 3 4 5

6 7

0

8

9

0

0

X X

X

X

Page 31: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

31

Searching GraphsSearching Graphs

– Lexicographically First Depth-First Search Ordering [Reif 85]

– Stack Breadth-First Search [Greenlaw 92]

– Breadth-Depth Search [Greenlaw 93]

Page 32: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

32

Context-Free Grammar EmptyContext-Free Grammar Empty

Given: A context-free grammar G=(N,T,P,S).Problem: Is L(G) empty?

Theorem: [Jones and Laaser 76], [Goldschlager 81], [Tompa 91]CFGempty is P-complete.

Proof: Reduce Monotone CVP to CFGempty. Given construct G=(N,T,P,S) with N, T, S, and P as follows:

Page 33: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

33

Context-Free Grammar EmptyContext-Free Grammar Empty

N = {i | vi is a vertex in }

T = {a }S = n, where vn is the output of .

P as follows:

1. For input vi, i a if value of vi is 1,

2. i jk if vi vj Λ vk, and

3. i j | k if vi vj V vk .

Then the value of vi is 1 if and only if i , where є {a }+.*

Page 34: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

34

CFGempty ExampleCFGempty Example

x1 = 0, x2 = 0, x3 = 1, and x4 = 1.

G = (N, T, S, P), whereN = {1, 2, 3, 4, 5, 6, 7 }T = {a }S = 7P = {3 a, 4 a, 5 1 | 2, 6 34, 7 56}

1 2 3 4

5 6

7

x3 x4x1 x2

OR

AND

AND

Page 35: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

35

OutlineOutline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems

Page 36: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

36

Comparator Circuit ClassComparator Circuit Class

Definition: [Cook 82]Comparator Circuit Value Problem (CCVP)

Given: An encoding of a circuit composed of comparator gates, plus inputs x1,…,xn, and a designated output y.Problem: Is output y of TRUE on input x1,…,xn?

Definition: CC is the class of languages NC reducible to CCVP.

Page 37: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

37

CC-Complete ProblemsCC-Complete Problems

– Lexicographically First Maximal Matching [Cook 82]

– Telephone Connection [Ramachandran and Wang 91]

– Stable Marriage [Mayr and Subramanian 92]

Page 38: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

38

OutlineOutline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems

Page 39: An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and Larry Ruzzo 1 Joint work with: An Introduction to Parallel.

An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo

39

Open ProblemsOpen Problems

Find an NC algorithm or classify as P-complete.

– Edge Ranking– Edge-Weighted Matching– Restricted Lexicographically First Maximal

Independent Set– Integer Greatest Common Divisor– Modular Powering