1

Recurrences

Algorithms

Jay Urbain, PhD

urbain@msoe.edu

Credits: Discrete Mathematics and Its Applications, by Kenneth RosenThe Design and Analysis of Algorithms, by Levitan

2

OverviewWe are interested in determining the running time behavior

of algorithms expressed as bounds using Θ, O, or Ω representation.

3

Recursive Algorithms• Recurrence relations result naturally from the analysis of

recursive algorithms.• Solving recurrence relations yields a closed-end formula

for calculation of run time.

• Example: the recursive n! algorithm:

int factorial (int n)

if (n == 1) return 1;

return factorial (n-1) * n;

4

n!int factorial (int n)

if (n == 1) return 1;

return factorial (n-1) * n;

• The running time, T(n), can be defined as:

T(1) = 1*T(n) = T(n-1) + 1 for all n>1

• where 1 is the running time for each execution of the factorial function.

5

n!• Solving the recurrence by recognizing the summation leads

to closed-end formula:

T(n) = T(1) + 1 + ... + 1 = 1 + 1 + ... + 1 = n

• The run time for execution of factorial can now be directly computed for a given n.

T(10) = 10.

• More importantly, when expressing the worst case run time bounds we can then write:

T(n) = O(1) if n=1;

T(n) = O(n) if n>1

6

Sequences and Recurrence Relations

Sequence - a numerical, ordered list of numbers• T - a recurrence function defining a sequence• n - a parameter to function T• T(n) - the sequence term generated by function T for parameter n

7

Sequences and Recurrence Relations

Is T(1) = 1 necessary?

8

Sequences and Recurrence Relations

• Inductive proofs on the recurrence:

T(n) = T(n-1) + 1

Typically assume the case for parameter n that generates a sequence term for recurrence T(n) and prove: T(n+1)=T((n+1)-1)+1.

• For above recurrence: T(n) = T(floor(n/4)) + n

What parameter value generates the next sequence term following T(n)?

9

Defining a sequence

• Either Tn or T(n) which stresses that a sequence is a function where T(n) is the generic nth term.

• Defining a sequence (two ways)– Closed-end formula of generic term as a function of n,

T(n) = 2n for n>=0– Recurrence equation relating generic term to one or more other

sequence terms combined with one or more explicit values of first term(s).

T(n) = T(n-1) + n for n>1 // generic term defined by other sequence terms, i.e., recurrence

T(0) = 0 //initial condition

10

Example: Fibonacci

• Fibonacci sequence definition:

int fibonacci (int n)

if( n==0 || n==1 ) return n;

return fibonacci(n-1) + fibonacci(n-1);

• has the corresponding recurrence:

F(0)=0 InitialF(1)=1 Initial F(n) = F(n-1)+F(n-2) for n>1 Recurrence

11

Solve Recurrence Relation

Find closed-end formula for generic nth term of sequence that satisfies both the recurrence term and initial condition or prove that the sequence does not exist.

• Example

T(n) = T(n-1) + n for n>1 B1

T(0) = 0 B2

• Solution of B1 subject to initial condition B2 is:

T(n) = n(n+1) for n>=0 B3 2

12

Verify closed-end solution

• Substitute solution into recurrence formula to check that equality holds or use induction to prove. B1 must hold for every n>0:

13

Solution to Recurrence

• Particular solution to recurrence - a specific sequence that satisfies recurrence.– B3 is a particular solution for B1 and B2.

• General solution to recurrence - formula that specifies all sequences, typically includes arbitrary constants. A general solution for recurrence B1 is:

T(n) = c + n(n+1) for n>=0 2

• choosing different values for c gives all solutions to B1.

• No universal method but common techniques for a variety of recurrences!

14

Forward substitution• Forward substitution for finding exact closed-end equation for T(n)

- limited to simple recurrences.

• Start with initial term(s) given by initial conditions, generate first few terms in hope of seeing a pattern that can be expressed by a closed-end formula.

• Example

T(n) = 2T(n-1)+1 for n>1 B5

T(1)=1 B6

• Generate

T(1) = 1 = 21-1

T(2) = 2T(2-1)+1 = 2T(1)+1 = 2*1+1 = 3 = 22-1

T(3) = 2T(3-1)+1 = 2T(2)+1 = 2*3+1 = 7 = 23-1

T(4) = 2T(4-1)+1 = 2T(3)+1 = 2*7+1 = 15 = 24-1

• Hope

Observe pattern that suggests T(n) = 2n-1 for n=1,2,3,4

15

Forward substitution• Prove T(n) = 2n-1 correct by substitution or induction

Backward Substituion

• express T(n-1) as a function of T(n-2)• substitute result to give T(n) as a function of T(n-2)• repeating for T(n-2) gives T(n) as function of T(n-3)• hope to see a pattern to express T(n) as function of T(n-i)

for i=1,2, ...

• select i to make n-i reach the initial condition• closed-end formula

– use standard summation formula from Appendix A will often lead to closed-end formula

– when lucky, get form similar to: T(n) = Base case + f(n), and can solve by substitution

16

Backward Substituion

17

Backward Substitution

18

19

Common Recurrence Types in Algorithmic Analysis

• Decrease-by-one algorithms exploits relationship between instance of size n and smaller instance of size n-1.

• Decrease-by-a-constant factor algorithms reduce instance of size n to an instance of size n/b (b=2 for many such algorithms), solving the smaller instance recursively. If necessary, the solution of the smaller instance is combined to a solution of the given instance.

• Divide-and-conquer divides a given instance into smaller instances, solving each recursively. If necessary solutions to smaller instances are combined to give a solution to a given instance.

20

Decrease-by-one algorithms

• Decrease-by-one algorithms exploits relationship between instance of size n and smaller instance of size n-1.

21

Decrease-by-one algorithms

• Typical form of recurrence equation, where f(n) is the time to reduce an instance to a smaller one is:

22

Specific functions - f(n)

23

Decrease-by-a-constant factor algorithms

• Reduce instance of size n to an instance of size n/b (b=2 for many such algorithms), solving the smaller instance recursively.

• If necessary, the solution of the smaller instance is extended to a solution of the given instance.

24

Decrease-by-a-constant factor algorithms – Binary Search

25

Divide and Conquer

Divides a given instance into smaller instances, solving each recursively. If necessary solutions to smaller instances are combined to give a solution to a given instance.

26

Merge Sort

27

Merge Sort

28

Merge Sort

29

General form of recurrence equation

• n is the problem size• B>=2 the number of problem divisions• n/b is the size of the smaller problem after division• A>=1 is the number of sub-problem solutions• f(n) is the time to divide an instance to a smaller ones and combine

the solutions

T(n) = aT(n/b) + f(n)

30