CSC 413/513: Intro to Algorithms NP Completeness.
Embed Size (px)
Transcript of CSC 413/513: Intro to Algorithms NP Completeness.
CSC 413/513: Intro to AlgorithmsNP Completeness
Problems and InstancesProblem: Sort(A)Problem instance: Sort([4,2,7,1,6,9])Input encodingBin(4) concat Bin(2) concat Bin(7) Size: n lg xn: size of Alg x: max. number of bits required to represent A[i], for any iWe say size = (n), or simply n
NP-CompletenessSome problems are intractable: as the input size increases, we are unable to solve them in reasonable timeWhat constitutes reasonable time? Standard working definition: polynomial timeOn an input of size n the worst-case running time is O(nk) for some constant kPolynomial time: O(n2), O(n3), O(1), O(n lg n) Not in polynomial time: O(2n), O(nn), O(n!)
Polynomial-Time ProblemsAre some problems solvable in polynomial time?Of course: every algorithm weve studied provides polynomial-time solution to some problemExcept DP algorithm to 0-1 knapsackWe define P to be the class of problems solvable in polynomial timeThat is, all instances of such problems are polynomially solvable (e.g., sorting)These are the easy problems
Polynomial-Time ProblemsAre all problems solvable in polynomial time?No: Turings Halting Problem is not solvable by any computer, no matter how much time is givenBut we often detect infinite loops in a program by inspection, and hence solve the halting problemHowever, there are instances that are not solvable, even by our brainsSuch problems are clearly intractable, not in PSimilarly, there are problems that can be solved, but we havent found algorithms that will take any instance of the problem and solve it in polynomial time
NP-Complete ProblemsThe NP-Complete problems are an interesting class of problems whose status is unknown No polynomial-time algorithm has been discovered for an NP-Complete problem, for all instances of itNo suprapolynomial lower bound has been proved for some instance of any NP-Complete problem, eitherWe call this the P = NP questionThe biggest open problem in CS
An NP-Complete Problem:Hamiltonian CyclesAn example of an NP-Complete problem:A hamiltonian cycle of an undirected graph is a simple cycle that contains every vertexThe hamiltonian-cycle problem: given a graph G, does it have a hamiltonian cycle?
Describe a nave algorithm for solving the hamiltonian-cycle problem. Running time?
P and NPAs mentioned, P is set of problems that can be solved in polynomial timeNP (nondeterministic polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computerWhat the hell is that?
NondeterminismThink of a non-deterministic computer as a computer that magically guesses a solution, and then has to verify that it is correctIf a solution exists, n.d. computer always guesses itOne way to imagine it: a parallel computer that can freely spawn an infinite number of processesHave one processor work on each possible solutionAll processors attempt to verify that their solution works, in parallelIf a processor finds it has a working solutionSo: NP = problems verifiable in polynomial timeNote: P NP (think of verifying the output of sorting)
P and NPSummary so far:P = problems that can be solved in polynomial timeNP = problems for which a solution can be verified in polynomial timeP NPUnknown whether P = NP (most suspect not)Hamiltonian-cycle problem is in NP:Dont know how to solve in polynomial timeBut, easy to verify solution in polynomial time (How?)
Intuition behind NP-Complete problemsMany problems exhibit structure that is exploitedE.g., recursive structure in sorting, optimal substructure in problems solved by DP, greedy etc.But, some problems seem to have no exploitable structure E.g., longest simple path in a graphContrast with shortest paths in graphs, which has the optimal substructureTypically, these are the NP-Complete problemsDoes not have optimal substructure
What about 0-1 Knapsack?Has optimal substructureBut input contains a number (W), and the optimal substructure depends on the magnitude of this number, not its sizeAlgorithm turns out to be exponential in the size of W (although polynomial in the magnitude of W)Its DP Algorithm is called pseudo-polynomial0-1 knapsack is called weakly NP-complete
NP-Complete ProblemsWe will see that NP-Complete problems are the hardest problems in NP:If any one NP-Complete problem can be solved in polynomial timethen every NP-Complete problem can be solved in polynomial timeand in fact every problem in NP can be solved in polynomial time (which would show P = NP)Thus: solve hamiltonian-cycle in O(n100) time, youve proved that P = NP. Retire rich & famous.
A SimplificationHenceforth, we will only talk about decision problems, with boolean answersMost problems of interest can be posed as decision problemsE.g., SHORTEST_PATH(G,u,v)Its decision problem is PATH(G,u,v,k): is there a path between u and v involving no more than k edges?Can solve SHORTEST_PATH by calling PATH repeatedly with decreasing k, until the answer is NoSo, solve decide
How to Verify? How do you verify the answer to a decision problem (in polynomial time)?Need a certificateE.g., certificate for PATH(G,u,v,k) would be the path: How do you verify the certificate?Check length kCheck first vertex is u, last vertex is vCheck all path-edges actually existHow much time?
Why only Decision Problems?So that problems can be considered as languagesThen, algorithm Turing machineConsider the binary encoding of a problem instance (i.e., the input)Collect all input binary strings for which the answer/output is YesThis set is a language A machine that decides the language also solves the problemSo, decision problem language
ReductionThe crux of NP-Completeness is reducibilityInformally, a problem P can be reduced to another problem Q if any instance of P can be easily rephrased as an instance of Q, the solution to which provides a solution to the instance of PWhat do you suppose easily means? This rephrasing is called transformationIntuitively: If P reduces to Q, P is no harder to solve than QPolynomial time
Polynomial Reduction Polynomial reduction can be used for polynomial time solution If A polynomially reduces to B, and we know a polynomial algorithm to B, then we can solve A in polynomial time as well.
ReducibilityAn example:A: Given a set of Booleans, is at least one TRUE?B: Given a set of integers, is their sum > 0?Transformation: Given (x1, x2, , xn) construct (y1, y2, , yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSEAnother example: Solving linear equations is reducible to solving quadratic equationsHow can we easily use a quadratic-equation solver to solve linear equations?
NP-Hard and NP-CompleteIf P is polynomial-time reducible to Q, we denote this P p QDefinition of NP-Hard and NP-Complete: If all problems R NP are reducible to P, then P is NP-HardWe say P is NP-Complete if P is NP-Hard and P NPIf P p Q and P is NP-Complete, Q is also NP- Complete
Why Prove NP-Completeness?Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractableTherefore it can be important to prove that a problem is NP-CompleteDont need to come up with an efficient algorithmCan instead work on approximation algorithms
Proving NP-CompletenessWhat steps do we have to take to prove a problem P is NP-Complete?Pick a known NP-Complete problem QReduce Q to PDescribe a transformation that maps an arbitrary instance of Q to some instance of P, s.t. yes for P = yes for QProve the transformation worksProve the transformation takes polynomial timeOh yeah, prove P NP (What if you cant?)
Coming UpGiven one NP-Complete problem, we can prove many interesting problems NP-CompleteGraph coloring (recall the wrestler-rivalry HW problem)Hamiltonian cycleHamiltonian pathKnapsack problemTraveling salesmanJob scheduling with penaltiesMany, many more