CSC 413/513: Intro to Algorithms
NP Completeness
Problems and Instances
● Problem: Sort(A)● Problem instance: Sort([4,2,7,1,6,9])● Input encoding
■ Bin(4) concat Bin(2) concat Bin(7) …■ Size: n lg x
○ n: size of A○ lg x: max. number of bits required to represent A[i], for
any i○ We say size = ϴ(n), or simply n
NP-Completeness
● Some problems are intractable: as the input size increases, we are unable to solve them in reasonable time
● What constitutes reasonable time? Standard working definition: polynomial time■ On an input of size n the worst-case running time is
O(nk) for some constant k■ Polynomial time: O(n2), O(n3), O(1), O(n lg n) ■ Not in polynomial time: O(2n), O(nn), O(n!)
Polynomial-Time Problems
● Are some problems solvable in polynomial time?■ Of course: every algorithm we’ve studied provides
polynomial-time solution to some problem○ Except DP algorithm to 0-1 knapsack
■ We define P to be the class of problems solvable in polynomial time
○ That is, all instances of such problems are polynomially solvable (e.g., sorting)
■ These are the “easy” problems
Polynomial-Time Problems
● Are all problems solvable in polynomial time?■ No: Turing’s “Halting Problem” is not solvable by any
computer, no matter how much time is given○ But we often detect infinite loops in a program by inspection, and
hence “solve” the halting problem○ However, there are instances that are not solvable, even by our brains
■ Such problems are clearly intractable, not in P■ Similarly, there are problems that can be solved, but we
haven’t found algorithms that will take any instance of the problem and solve it in polynomial time
NP-Complete Problems
● The NP-Complete problems are an interesting class of problems whose status is unknown ■ No polynomial-time algorithm has been discovered
for an NP-Complete problem, for all instances of it■ No suprapolynomial lower bound has been proved for
some instance of any NP-Complete problem, either
● We call this the P = NP question■ The biggest open problem in CS
An NP-Complete Problem:Hamiltonian Cycles
● An example of an NP-Complete problem:■ A hamiltonian cycle of an undirected graph is a
simple cycle that contains every vertex■ The hamiltonian-cycle problem: given a graph G,
does it have a hamiltonian cycle?
Describe a naïve algorithm for solving the hamiltonian-cycle problem. Running time?
P and NP
● As mentioned, P is set of problems that can be solved in polynomial time
● NP (nondeterministic polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computer■ What the hell is that?
Nondeterminism
● Think of a non-deterministic computer as a computer that magically “guesses” a solution, and then has to verify that it is correct■ If a solution exists, n.d. computer always guesses it■ One way to imagine it: a parallel computer that can freely
spawn an infinite number of processes○ Have one processor work on each possible solution○ All processors attempt to verify that their solution works, in parallel○ If a processor finds it has a working solution
■ So: NP = problems verifiable in polynomial time■ Note: P NP (think of verifying the output of sorting)
P and NP
● Summary so far:■ P = problems that can be solved in polynomial time■ NP = problems for which a solution can be verified in
polynomial time■ P NP■ Unknown whether P = NP (most suspect not)
● Hamiltonian-cycle problem is in NP:■ Don’t know how to solve in polynomial time■ But, easy to verify solution in polynomial time (How?)
Intuition behind NP-Complete problems
● Many problems exhibit structure that is exploited■ E.g., recursive structure in sorting, optimal
substructure in problems solved by DP, greedy etc.
● But, some problems seem to have no exploitable structure ■ E.g., longest simple path in a graph
○ Contrast with shortest paths in graphs, which has the optimal substructure
■ Typically, these are the NP-Complete problems
Does not have optimal substructure
What about 0-1 Knapsack?
● Has optimal substructure■ But input contains a number (W), and the optimal
substructure depends on the magnitude of this number, not its size
■ Algorithm turns out to be exponential in the size of W (although polynomial in the magnitude of W)
■ Its DP Algorithm is called pseudo-polynomial
● 0-1 knapsack is called weakly NP-complete
NP-Complete Problems
● We will see that NP-Complete problems are the “hardest” problems in NP:■ If any one NP-Complete problem can be solved in
polynomial time…○ …then every NP-Complete problem can be solved in
polynomial time…○ …and in fact every problem in NP can be solved in
polynomial time (which would show P = NP)
■ Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.
A Simplification
● Henceforth, we will only talk about decision problems, with boolean answers■ Most problems of interest can be posed as decision
problems■ E.g., SHORTEST_PATH(G,u,v)■ Its decision problem is PATH(G,u,v,k): is there a path
between u and v involving no more than k edges?■ Can solve SHORTEST_PATH by calling PATH
repeatedly with decreasing k, until the answer is “No”
● So, “solve” ≡ “decide”
How to Verify?
● How do you verify the answer to a decision problem (in polynomial time)?
● Need a certificate■ E.g., certificate for PATH(G,u,v,k) would be the path:
<u,v1,v2,…,v>
● How do you verify the certificate?■ Check length ≤ k■ Check first vertex is u, last vertex is v■ Check all path-edges actually exist
● How much time?
Why only Decision Problems?
● So that problems can be considered as languages■ Then, “algorithm” ≡ Turing machine■ Consider the binary encoding of a problem instance (i.e., the
input)■ Collect all input binary strings for which the answer/output is
“Yes”■ This set is a language
○ A machine that decides the language also solves the problem
● So, “decision problem” ≡ “language”
Reduction
● The crux of NP-Completeness is reducibility■ Informally, a problem P can be reduced to another
problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P
○ What do you suppose “easily” means? ○ This rephrasing is called transformation
■ Intuitively: If P reduces to Q, P is “no harder to solve” than Q
Polynomial time
Polynomial Reduction
• Polynomial reduction can be used for polynomial time solution • If A polynomially reduces to B, and we know a polynomial algorithm to B, then we can solve A in polynomial time as well.
Reducibility
● An example:■ A: Given a set of Booleans, is at least one TRUE?■ B: Given a set of integers, is their sum > 0?
■ Transformation: Given (x1, x2, …, xn) construct (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE
● Another example: ■ Solving linear equations is reducible to solving quadratic
equations○ How can we easily use a quadratic-equation solver to solve linear
equations?
NP-Hard and NP-Complete
● If P is polynomial-time reducible to Q, we denote this P p Q
● Definition of NP-Hard and NP-Complete: ■ If all problems R NP are reducible to P, then P is
NP-Hard■ We say P is NP-Complete if P is NP-Hard
and P NP
● If P p Q and P is NP-Complete, Q is alsoNP- Complete
Why Prove NP-Completeness?
● Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable
● Therefore it can be important to prove that a problem is NP-Complete■ Don’t need to come up with an efficient algorithm■ Can instead work on approximation algorithms
Proving NP-Completeness
● What steps do we have to take to prove a problem P is NP-Complete?■ Pick a known NP-Complete problem Q■ Reduce Q to P
○ Describe a transformation that maps an arbitrary instance of Q to some instance of P, s.t. “yes” for P = “yes” for Q
○ Prove the transformation works○ Prove the transformation takes polynomial time
■ Oh yeah, prove P NP (What if you can’t?)
Coming Up
● Given one NP-Complete problem, we can prove many interesting problems NP-Complete■ Graph coloring (recall the wrestler-rivalry HW problem)■ Hamiltonian cycle■ Hamiltonian path■ Knapsack problem■ Traveling salesman■ Job scheduling with penalties■ Many, many more
Top Related