CSC 413/513: Intro to Algorithms

NP Completeness

Problems and Instances

● Problem: Sort(A)● Problem instance: Sort([4,2,7,1,6,9])● Input encoding

■ Bin(4) concat Bin(2) concat Bin(7) …■ Size: n lg x

○ n: size of A○ lg x: max. number of bits required to represent A[i], for

any i○ We say size = ϴ(n), or simply n

NP-Completeness

● Some problems are intractable: as the input size increases, we are unable to solve them in reasonable time

● What constitutes reasonable time? Standard working definition: polynomial time■ On an input of size n the worst-case running time is

O(nk) for some constant k■ Polynomial time: O(n2), O(n3), O(1), O(n lg n) ■ Not in polynomial time: O(2n), O(nn), O(n!)

Polynomial-Time Problems

● Are some problems solvable in polynomial time?■ Of course: every algorithm we’ve studied provides

polynomial-time solution to some problem○ Except DP algorithm to 0-1 knapsack

■ We define P to be the class of problems solvable in polynomial time

○ That is, all instances of such problems are polynomially solvable (e.g., sorting)

■ These are the “easy” problems

Polynomial-Time Problems

● Are all problems solvable in polynomial time?■ No: Turing’s “Halting Problem” is not solvable by any

computer, no matter how much time is given○ But we often detect infinite loops in a program by inspection, and

hence “solve” the halting problem○ However, there are instances that are not solvable, even by our brains

■ Such problems are clearly intractable, not in P■ Similarly, there are problems that can be solved, but we

haven’t found algorithms that will take any instance of the problem and solve it in polynomial time

NP-Complete Problems

● The NP-Complete problems are an interesting class of problems whose status is unknown ■ No polynomial-time algorithm has been discovered

for an NP-Complete problem, for all instances of it■ No suprapolynomial lower bound has been proved for

some instance of any NP-Complete problem, either

● We call this the P = NP question■ The biggest open problem in CS

An NP-Complete Problem:Hamiltonian Cycles

● An example of an NP-Complete problem:■ A hamiltonian cycle of an undirected graph is a

simple cycle that contains every vertex■ The hamiltonian-cycle problem: given a graph G,

does it have a hamiltonian cycle?

Describe a naïve algorithm for solving the hamiltonian-cycle problem. Running time?

P and NP

● As mentioned, P is set of problems that can be solved in polynomial time

● NP (nondeterministic polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computer■ What the hell is that?

Nondeterminism

● Think of a non-deterministic computer as a computer that magically “guesses” a solution, and then has to verify that it is correct■ If a solution exists, n.d. computer always guesses it■ One way to imagine it: a parallel computer that can freely

spawn an infinite number of processes○ Have one processor work on each possible solution○ All processors attempt to verify that their solution works, in parallel○ If a processor finds it has a working solution

■ So: NP = problems verifiable in polynomial time■ Note: P NP (think of verifying the output of sorting)

P and NP

● Summary so far:■ P = problems that can be solved in polynomial time■ NP = problems for which a solution can be verified in

polynomial time■ P NP■ Unknown whether P = NP (most suspect not)

● Hamiltonian-cycle problem is in NP:■ Don’t know how to solve in polynomial time■ But, easy to verify solution in polynomial time (How?)

Intuition behind NP-Complete problems

● Many problems exhibit structure that is exploited■ E.g., recursive structure in sorting, optimal

substructure in problems solved by DP, greedy etc.

● But, some problems seem to have no exploitable structure ■ E.g., longest simple path in a graph

○ Contrast with shortest paths in graphs, which has the optimal substructure

■ Typically, these are the NP-Complete problems

Does not have optimal substructure

What about 0-1 Knapsack?

● Has optimal substructure■ But input contains a number (W), and the optimal

substructure depends on the magnitude of this number, not its size

■ Algorithm turns out to be exponential in the size of W (although polynomial in the magnitude of W)

■ Its DP Algorithm is called pseudo-polynomial

● 0-1 knapsack is called weakly NP-complete

NP-Complete Problems

● We will see that NP-Complete problems are the “hardest” problems in NP:■ If any one NP-Complete problem can be solved in

polynomial time…○ …then every NP-Complete problem can be solved in

polynomial time…○ …and in fact every problem in NP can be solved in

polynomial time (which would show P = NP)

■ Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

A Simplification

● Henceforth, we will only talk about decision problems, with boolean answers■ Most problems of interest can be posed as decision

problems■ E.g., SHORTEST_PATH(G,u,v)■ Its decision problem is PATH(G,u,v,k): is there a path

between u and v involving no more than k edges?■ Can solve SHORTEST_PATH by calling PATH

repeatedly with decreasing k, until the answer is “No”

● So, “solve” ≡ “decide”

How to Verify?

● How do you verify the answer to a decision problem (in polynomial time)?

● Need a certificate■ E.g., certificate for PATH(G,u,v,k) would be the path:

<u,v1,v2,…,v>

● How do you verify the certificate?■ Check length ≤ k■ Check first vertex is u, last vertex is v■ Check all path-edges actually exist

● How much time?

Why only Decision Problems?

● So that problems can be considered as languages■ Then, “algorithm” ≡ Turing machine■ Consider the binary encoding of a problem instance (i.e., the

input)■ Collect all input binary strings for which the answer/output is

“Yes”■ This set is a language

○ A machine that decides the language also solves the problem

● So, “decision problem” ≡ “language”

Reduction

● The crux of NP-Completeness is reducibility■ Informally, a problem P can be reduced to another

problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P

○ What do you suppose “easily” means? ○ This rephrasing is called transformation

■ Intuitively: If P reduces to Q, P is “no harder to solve” than Q

Polynomial time

Polynomial Reduction

• Polynomial reduction can be used for polynomial time solution • If A polynomially reduces to B, and we know a polynomial algorithm to B, then we can solve A in polynomial time as well.

Reducibility

● An example:■ A: Given a set of Booleans, is at least one TRUE?■ B: Given a set of integers, is their sum > 0?

■ Transformation: Given (x1, x2, …, xn) construct (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

● Another example: ■ Solving linear equations is reducible to solving quadratic

equations○ How can we easily use a quadratic-equation solver to solve linear

equations?

NP-Hard and NP-Complete

● If P is polynomial-time reducible to Q, we denote this P p Q

● Definition of NP-Hard and NP-Complete: ■ If all problems R NP are reducible to P, then P is

NP-Hard■ We say P is NP-Complete if P is NP-Hard

and P NP

● If P p Q and P is NP-Complete, Q is alsoNP- Complete

Why Prove NP-Completeness?

● Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable

● Therefore it can be important to prove that a problem is NP-Complete■ Don’t need to come up with an efficient algorithm■ Can instead work on approximation algorithms

Proving NP-Completeness

● What steps do we have to take to prove a problem P is NP-Complete?■ Pick a known NP-Complete problem Q■ Reduce Q to P

○ Describe a transformation that maps an arbitrary instance of Q to some instance of P, s.t. “yes” for P = “yes” for Q

○ Prove the transformation works○ Prove the transformation takes polynomial time

■ Oh yeah, prove P NP (What if you can’t?)

Coming Up

● Given one NP-Complete problem, we can prove many interesting problems NP-Complete■ Graph coloring (recall the wrestler-rivalry HW problem)■ Hamiltonian cycle■ Hamiltonian path■ Knapsack problem■ Traveling salesman■ Job scheduling with penalties■ Many, many more