Introduction to Algorithms

Author
portiaroman 
Category
Documents

view
30 
download
0
Embed Size (px)
description
Transcript of Introduction to Algorithms

Introduction to Algorithms
Jiafen LiuSept. 2013

Todays taskDevelop more asymptotic notationsHow to solve recurrences

notation
Math: (g(n)) = { f(n) : there exist positive constants c1, c2, and n0 such that 0 c1g(n) f (n) c2g(n) for all n n0}Engineering:Drop loworder terms; ignore leading constants.

OnotationAnother asymptotic symbol used to indicate upper bounds.Math:
Ex : 2n2= O(n3)(c= 1, n0= 2)= means oneway equality

Set definition of Onotation
Math:
Looks better?EX : 2n2 O(n3)Onotation corresponds roughly to less than or equal to.

Usage of OnotationMacro substitution: A set in a formula represents an anonymous function in the set, if Onotation only occurs on the right hand of formula.Ex:f(n) = n3+ O(n2) Means f(n) = n3+ h(n)for some h(n) O(n2), here we can think of h(n) as error term.

Usage of Onotation
If Onotation occurs on the left hand of formula, such as n2+ O(n) = O(n2), which means for any f(n) O(n):there exists some h(n) O(n2), makes n2+ f(n) = h(n)Onotation is an upperbound notation. It makes no sense to say f(n) is at least O(n2).

notation How can we express a lower bound?
Ex:

notation
Onotation is like notation is like .And notation is like =Now we can give another definition of notation
We also call notation as tight bound.

A Theorem on Asymptotic Notations Theorem 3.1 For any two functions f(n) and g(n), we say f(n)=(g(n)) if and only if f(n)=O(g(n)) and f(n)=(g(n)).
Ex: , how can we prove that?

Two More Strict NotationsWe have just said that:Onotation and notation are like and .onotation and notation are like < and >.
Difference with Onotation : This inequality must hold for all c instead of just for 1.

What does it mean? for any constant c With onotation, we mean that: no matter what constant we put in front of g(x), f(x) will be still less than cg(x) for sufficiently large n. No matter how small c is.Ex: 2n2= o(n3)
An counter example:1/2n2 = (n2) does not hold for each c.(n0= 2/c)

Two More Strict NotationsWe have just said that:Onotation and notation are like and .onotation and notation are like < and >.
Difference with notation : This inequality must hold for all c instead of just for 1.

Solving recurrencesThe analysis of merge sort from Lecture 1 required us to solve a recurrence. Lecture 3: Applications of recurrences to divideandconquer algorithms.We often omit some details inessential while solving recurrences:n is supposed to be an integer because the size of input is an integer typically .Ignore the boundary conditions for convenience.

Substitution methodThe most general method : Substitution methodGuess the form of the solution.Verify by induction.Solve for constants.Substitution method is used to determine the upper or lower bounds of recurrence.

Example of SubstitutionEXAMPLE: T(n) = 4T(n/2) + n (n1)(Assume that T(1) = (1) )Can you guess the time complexity of it?Guess O(n3). (Prove O and separately.)We will prove T(n) cn3 by induction.First ,we assume that T(k) ck3 for k < n T(k) ck3 holds while k=n/2 by assumption.

Example of Substitution
All we need is this holds as long as c 2 for n 1

Base Case in InductionWe must also handle the initial conditions, that is, ground the induction with base cases.Base: T(n) = (1) for all n < n0, where n0 is a suitable constant.For 1 n< n0, we have (1) cn3, if we pick c big enough.
Here we are!
BUT this bound is not tight !

A tighter upper bound?We shall prove that T(n) = O(n2) .Assume that T(k) ck2 for k < n.
Anybody can tell me why? Now we need n 0 But it seems impossible for n 1

A tighter upper bound!IDEA: Strengthen the inductive hypothesis. Subtract a loworder term.Inductive hypothesis: T(k) c1k2 c2k for k< n.
Now we need (c21) n 0, it holds if c2 1

For the Base CaseWe have proved now that for any value of c1, and provided c2 1.Base: We need T(1) c1 c2Assumed T(1) is some constant.We need to choose c1 to be sufficiently larger than c2, and c2 has to be at least 1.To handle the initial conditions, just pick c1 big enough with respect to c2.

About Substitution methodWe have worked for upper bounds, and the lower bounds are similar.Try it yourself.Shortcomings: We had to know the answer in order to find it, which is a bit of a pain. It would be nicer to just figure out the answer by some procedure, and that will be the next two techniques.

Recursiontree methodA recursion tree models the costs (time) of a recursive execution of an algorithm.The recursiontree method can be unreliable, just like any method that uses dot,dot,dots ().The recursiontree method promotes intuition, however. The recursion tree method is good for generating guesses for the substitution method.

Example of recursion tree
Solve T(n) = T(n/4) + T(n/2)+ n2:

Example of recursion tree
Solve T(n) = T(n/4) + T(n/2)+ n2:

Example of recursion tree
Solve T(n) = T(n/4) + T(n/2)+ n2:

Example of recursion tree
Solve T(n) = T(n/4) + T(n/2)+ n2:?How many leaves?We just need an upper bound.

Example of recursion tree
Solve T(n) = T(n/4) + T(n/2)+ n2:
2n2 So T(n) = (n2)
????Recall 1+1/2+1/4+1/8+

The Master MethodIt looks like an application of the recursion tree method but with more precise.The sad part about the master method is it is pretty restrictive. It only applies to recurrences of the form: T(n) = a T(n/b) + f(n) , where a 1, b >1, and f(n) is asymptotically positive.aT(n/b) means every problem you recurse on should be of the same size.

Three Common Cases

Three Common Cases
Case 1:

Three Common Cases
Case 1:
Case 2:

Three Common Cases
Case 3:

ExamplesEx: T(n) = 4T(n/2) + n a = 4, b= 2 and

ExamplesEx: T(n) = 4T(n/2) + n2 a = 4, b= 2 and

ExamplesEx: T(n) = 4T(n/2) + n3 a = 4, b= 2 and

HomeworkRead Chapter 3 and 4 to be prepared for applications of recurrences.

Proof of Master MethodHeight = logbn#(leaves) = ?

Proof of Master MethodHeight = logbn#(leaves) = ?
CASE1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.

Proof of Master MethodHeight = logbn#(leaves) = ?
CASE2(k= 0) :The weight is approximately the same on each of the logbn levels.

Proof of Master MethodHeight = logbn#(leaves) = ?
CASE3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.

**Recall 1+1/2+1/4+1/8+*