Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2...

55
Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation O-, -, and Θ-notation Recurrences Substitution method Iterating the recurrence Recursion tree Master method Prof. Charles E. Leiserson

Transcript of Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2...

Page 1: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Introduction to Algorithms6.046J/18.401J

LECTURE 2Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences• Substitution method • Iterating the recurrence • Recursion tree • Master method

Prof. Charles E. Leiserson

Page 2: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Asymptotic notation

O-notation (upper bounds):

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.2

Page 3: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Asymptotic notation

O-notation (upper bounds):

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

EXAMPLE: 2n2 = O(n3) (c = 1, n0 = 2)

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.3

Page 4: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Asymptotic notation

O-notation (upper bounds):

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

EXAMPLE: 2n2 = O(n3) (c = 1, n0 = 2)

functions,not values

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.4

Page 5: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Asymptotic notation

O-notation (upper bounds):

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

EXAMPLE: 2n2 = O(n3) (c = 1, n0 = 2) funny, “one-way”

functions, equalitynot values

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.5

Page 6: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Set definition of O-notation

O(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n)for all n ≥ n0

O(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n)for all n ≥ n0

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.6

Page 7: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Set definition of O-notation

O(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n)for all n ≥ n0

O(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n)for all n ≥ n0

EXAMPLE: 2n2 ∈ O(n3)

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.7

Page 8: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Set definition of O-notation

O(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n)for all n ≥ n0

O(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n)for all n ≥ n0

EXAMPLE: 2n2 ∈ O(n3)(Logicians: λn.2n2 ∈ O(λn.n3), but it’sconvenient to be sloppy, as long as weunderstand what’s really going on.)

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.8

Page 9: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Macro substitution

Convention: A set in a formula representsan anonymous function in the set.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.9

Page 10: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Macro substitution

Convention: A set in a formula representsan anonymous function in the set. EXAMPLE: f(n) = n3 + O(n2)

means f(n) = n3 + h(n) for some h(n) ∈ O(n2) .

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.10

Page 11: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Ω−notation (lower bounds)

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2).

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.11

Page 12: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Ω−notation (lower bounds)

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2).

Ω(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ cg(n) ≤ f(n)for all n ≥ n0

Ω(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ cg(n) ≤ f(n)for all n ≥ n0

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.12

Page 13: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Ω−notation (lower bounds)

Ω(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ cg(n) ≤ f(n)for all n ≥ n0

Ω(g(n)) = f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ cg(n) ≤ f(n)for all n ≥ n0

EXAMPLE: n )(lg n= Ω © 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.13

Page 14: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Θ-notation (tight bounds)

Θ(g(n)) = O (g(n)) ∩ Ω(g(n))Θ(g(n)) = O (g(n)) ∩ Ω(g(n))

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.14

Page 15: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Θ-notation (tight bounds)

Θ(g(n)) = O (g(n)) ∩ Ω(g(n))Θ(g(n)) = O (g(n)) ∩ Ω(g(n))

2EXAMPLE: 1 n − 2n Θ= (n2)2

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.15

Page 16: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Θ-notation (tight bounds)

Θ(g(n)) = O (g(n)) ∩ Ω(g(n))Θ(g(n)) = O (g(n)) ∩ Ω(g(n))

1 2EXAMPLE: n − 2n Θ= (n2 )2

Theorem. The leading constant and low-order terms don’t matter.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.16

Page 17: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Solving recurrences

• The analysis of merge sort from Lecture 1 required us to solve a recurrence.

• Recurrences are like solving integrals, differential equations, etc. o Learn a few tricks.

• Lecture 3: Applications of recurrences to divide-and-conquer algorithms.

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.17

Page 18: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Substitution method

The most general method: 1. Guess the form of the solution. 2. Verify by induction. 3. Solve for constants.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.18

Page 19: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Substitution method

The most general method: 1. Guess the form of the solution.2. Verify by induction. 3. Solve for constants. EXAMPLE: T(n) = 4T(n/2) + n • [Assume that T(1) = Θ(1).] • Guess O(n3) . (Prove O and Ω separately.) • Assume that T(k) ≤ ck3 for k < n . • Prove T(n) ≤ cn3 by induction.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.19

Page 20: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Example of substitution

T (n) = 4T (n / 2) + n(≤ 4 n c / 2)3 + n

= (c / 2)n3 + n= cn3 − ((c / 2)n3 − n) desired – residual

3 desired≤ cn whenever (c/2)n3 – n ≥ 0, for example, if c ≥ 2 and n ≥ 1.

residual

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.20

Page 21: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Example (continued)• We must also handle the initial conditions,

that is, ground the induction with base cases.

• Base: T(n) = Θ(1) for all n < n0, where n0 is a suitable constant.

• For 1 ≤ n < n0, we have “Θ(1)” ≤ cn3, if we pick c big enough.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.21

Page 22: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Example (continued)• We must also handle the initial conditions,

that is, ground the induction with base cases.

• Base: T(n) = Θ(1) for all n < n0, where n0 is a suitable constant.

• For 1 ≤ n < n0, we have “Θ(1)” ≤ cn3, if we pick c big enough.

This bound is not tight! © 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.22

Page 23: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

A tighter upper bound?

We shall prove that T(n) = O(n2).

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.23

Page 24: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

A tighter upper bound?

We shall prove that T(n) = O(n2).

Assume that T(k) ≤ ck2 for k < n: T (n) = 4T (n / 2) + n

(≤ 4 n c / 2)2 + n2= cn + n

(= n O 2 )

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.24

Page 25: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.25

A tighter upper bound?

We shall prove that T(n) = O(n2).

Assume that T(k) ≤ ck2 for k < n:

)(

)2/(4)2/(4)(

2

2

2

nOncn

nncnnTnT

=+=

+≤+=

Wrong! We must prove the I.H.

© 2001–4 by Charles E. Leiserson

Page 26: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

A tighter upper bound?

We shall prove that T(n) = O(n2).

Assume that T(k) ≤ ck2 for k < n:T (n) = 4T (n / 2) + n

(≤ 4 n c / 2)2 + n2= cn + n

= n O ( 2 ) Wrong! We must prove the I.H.= cn2 − (− n) [ desired – residual ]≤ cn2 for no choice of c > 0. Lose!

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.26

Page 27: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

A tighter upper bound!IDEA: Strengthen the inductive hypothesis.• Subtract a low-order term.Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.27

Page 28: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

A tighter upper bound!IDEA: Strengthen the inductive hypothesis.• Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n.

T(n) = 4T(n/2) + n = 4(c1(n/2)2 – c2(n/2) + n = c1n2 – 2c2n + n = c1n2 – c2n – (c2n – n)≤ c1n2 – c2n if c2 ≥ 1.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.28

Page 29: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

A tighter upper bound!IDEA: Strengthen the inductive hypothesis.• Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n.

T(n) = 4T(n/2) + n = 4(c1(n/2)2 – c2(n/2) + n = c1n2 – 2c2n + n = c1n2 – c2n – (c2n – n)≤ c1n2 – c2n if c2 ≥ 1.

Pick c1 big enough to handle the initial conditions.© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.29

Page 30: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Recursion-tree method

• A recursion tree models the costs (time) of a recursive execution of an algorithm.

• The recursion-tree method can be unreliable, just like any method that uses ellipses (…).

• The recursion-tree method promotes intuition, however.

• The recursion tree method is good for generating guesses for the substitution method.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.30

Page 31: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Example of recursion tree

Solve T(n) = T(n/4) + T(n/2) + n2:

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.31

Page 32: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Example of recursion tree

Solve T(n) = T(n/4) + T(n/2) + n2:

T(n)

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.32

Page 33: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.33

Example of recursion tree

T(n/4) T(n/2)

n2

Solve T(n) = T(n/4) + T(n/2) + n2:

© 2001–4 by Charles E. Leiserson

Page 34: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.34

Example of recursion treeSolve T(n) = T(n/4) + T(n/2) + n2:

n2

(n/4)2 (n/2)2

T(n/16) T(n/8) T(n/8) T(n/4)

© 2001–4 by Charles E. Leiserson

Page 35: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.35

Example of recursion tree

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2 (n/2)2

Θ(1)

Solve T(n) = T(n/4) + T(n/2) + n2:n2

© 2001–4 by Charles E. Leiserson

Page 36: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.36

Example of recursion treeSolve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2 (n/2)2

Θ(1)

2nn2

© 2001–4 by Charles E. Leiserson

Page 37: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.37

Example of recursion treeSolve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2 (n/2)2

Θ(1)

2165 n

2nn2

© 2001–4 by Charles E. Leiserson

Page 38: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.38

Example of recursion treeSolve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2

Θ(1)

2165 n

2n

225625 n

n2

(n/2)2

© 2001–4 by Charles E. Leiserson

Page 39: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.39

Example of recursion treeSolve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2

Θ(1)

2165 n

2n

225625 n

( ) ( )( ) 1 31652

165

1652 L++++n

Total == Θ(n2)

n2

(n/2)2

geometric series© 2001–4 by Charles E. Leiserson

Page 40: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

The master method

The master method applies to recurrences of the form

T(n) = a T(n/b) + f (n) , where a ≥ 1, b > 1, and f is asymptotically positive.

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.40

Page 41: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Three common cases

Compare f (n) with nlogba: 1. f (n) = O(nlogba – ε) for some constant ε > 0.

• f (n) grows polynomially slower than nlogba

(by an nε factor). Solution: T(n) = Θ(nlogba) .

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.41

Page 42: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Three common cases

Compare f (n) with nlogba: 1. f (n) = O(nlogba – ε) for some constant ε > 0.

• f (n) grows polynomially slower than nlogba

(by an nε factor). Solution: T(n) = Θ(nlogba) .

nlogba lgkn2. f (n) = Θ( ) for some constant k ≥ 0.

n• f (n) and nlogba grow at similar rates.

logba lgk+1nSolution: T(n) = Θ( ) . © 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.42

Page 43: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Three common cases (cont.)Compare f (n) with nlogba:

3. f (n) = Ω(nlogba + ε) for some constant ε > 0. • f (n) grows polynomially faster than nlogba (by

an nε factor), and f (n) satisfies the regularity condition that a f (n/b) ≤ c f (n) for some constant c < 1. Solution: T(n) = Θ( f (n)) .

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.43

Page 44: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Examples

EX. T(n) = 4T(n/2) + n a = 4, b = 2 ⇒ nlogba = n2; f (n) = n. CASE 1: f (n) = O(n2 – ε) for ε = 1. ∴ T(n) = Θ(n2).

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.44

Page 45: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Examples

EX. T(n) = 4T(n/2) + n a = 4, b = 2 ⇒ nlogba = n2; f (n) = n. CASE 1: f (n) = O(n2 – ε) for ε = 1. ∴ T(n) = Θ(n2).

EX. T(n) = 4T(n/2) + n2

a = 4, b = 2 ⇒ nlogba = n2; f (n) = n2. CASE 2: f (n) = Θ(n2lg0n), that is, k = 0. ∴ T(n) = Θ(n2lg n).

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.45

Page 46: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Examples

EX. T(n) = 4T(n/2) + n3

a = 4, b = 2 ⇒ nlogba = n2; f (n) = n3. CASE 3: f (n) = Ω(n2 + ε) for ε = 1 and 4(n/2)3 ≤ cn3 (reg. cond.) for c = 1/2. ∴ T(n) = Θ(n3).

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.46

Page 47: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Examples

EX. T(n) = 4T(n/2) + n3

a = 4, b = 2 ⇒ nlogba = n2; f (n) = n3. CASE 3: f (n) = Ω(n2 + ε) for ε = 1 and 4(n/2)3 ≤ cn3 (reg. cond.) for c = 1/2. ∴ T(n) = Θ(n3).

EX. T(n) = 4T(n/2) + n2/lg n a = 4, b = 2 ⇒ nlogba = n2; f (n) = n2/lg n. Master method does not apply. In particular, for every constant ε > 0, we have nε = ω(lg n).

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.47

Page 48: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

Τ (1)

Recursion tree:

… f (n) a

f (n/b2)f (n/b2) f (n/b2)… a

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.48

Page 49: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

Τ (1)

Recursion tree:

… f (n) a

f (n/b2)f (n/b2) f (n/b2)… a

f (n)

a f (n/b)

a2 f (n/b2)

© 2001–4 by Charles E. Leiserson

September 13, 2004 Introduction to Algorithms L2.49

Page 50: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.50

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

Τ (1)

…Recursion tree:

…f (n) a

f (n/b2)f (n/b2) f (n/b2)…ah = logbn

f (n)

a f (n/b)

a2 f (n/b2)

© 2001–4 by Charles E. Leiserson

Page 51: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.51

nlogbaΤ (1)

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

Τ (1)

…Recursion tree:

…f (n) a

f (n/b2)f (n/b2) f (n/b2)…ah = logbn

f (n)

a f (n/b)

a2 f (n/b2)

#leaves = ah

= alogbn

= nlogba

© 2001–4 by Charles E. Leiserson

Page 52: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.52

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

Τ (1)

…Recursion tree:

…f (n) a

f (n/b2)f (n/b2) f (n/b2)…ah = logbn

f (n)

a f (n/b)

a2 f (n/b2)

CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.

CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.

Θ(nlogba)

nlogbaΤ (1)

© 2001–4 by Charles E. Leiserson

Page 53: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.53

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

Τ (1)

…Recursion tree:

…f (n) a

f (n/b2)f (n/b2) f (n/b2)…ah = logbn

f (n)

a f (n/b)

a2 f (n/b2)

CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels.

CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels.

Θ(nlogbalg n)

nlogbaΤ (1)

© 2001–4 by Charles E. Leiserson

Page 54: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

September 13, 2004 Introduction to Algorithms L2.54

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

Τ (1)

…Recursion tree:

…f (n) a

f (n/b2)f (n/b2) f (n/b2)…ah = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.

CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.

nlogbaΤ (1)

Θ( f (n))© 2001–4 by Charles E. Leiserson

Page 55: Introduction to Algorithms · 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution method

Appendix: geometric series

n1 + x + x2 + L + x = 1 − xn+1

for x ≠ 11 − x

1 + x + x2 + L = 1

for |x| < 11 − x

Return to last slide viewed.

© 2001–4 by Charles E. Leiserson September 13, 2004 Introduction to Algorithms L2.55