CS420 lecture threeLowerbounds

wim bohm, cs CSU

Big O, big omega, big theta• f(x) = O(g(x)) iff there are positive integers c and n0 such that

f(x) < c.g(x) for all x > n0

• f(x) = Ω(g(x)) iff there are positive integers c and n0 such that

f(x) > c.g(x) for all x > n0

• f(x) = Θ(g(x)) iff f(x) = O(g(x)) and f(x) = Ω(g(x))

Big O etc.

• Big O used in upper bounds, ie the worst or average case complexity of an algorithm. Big theta is a tight bound, eg stronger than upper bound.

• Big omega used in lower bounds, ie the complexity of a problem.

lower bounds

• An easy lower bound on a problem is the size of the output it needs to produce, or the number of inputs it has to access– Generate all permutations of size n, lower bound?– Towers of Hanoi, lower bound?– Sum n input integers, lower bound? but... sum integers 1 to n?

some Ω(n) problems

• Given an unsorted array A and a number p, rearrange A such that all A[i]<x occur before all A[i]>x

some Ω(n) problems

• Given an unsorted array A and a number x, rearrange A such that all A[i]<=x occur before all A[i]>=x

L=1; R=n; while(L<R){ while((A[L]<=x)&&(L<R)L++ while((A[R]>=x)&&(L<R))R- - if(L<R)swap(A,L,R) }

Dutch National Flag

• Red, blue and white tulips, rearrange them so that all reds precede all whites precede all blues

r=1; b=n; i=1; while(i<=b){ case S[i] red: swap(S,i,r);r++;i++; white: i++; blue: swap(S,i,b); b- -; } //How does it work?

flag

while(i<=b){ case S[i] red: swap(S,i,r);r++;i++; white: i++; blue: swap(S,i,b); b- -; }

€

∀j < r S[ j] = red

∀ j > b S[ j] = blue

∀ j : r < j < i S[ j] = white

flag

while(i<=b){ case S[i] red: swap(S,i,r);r++;i++; white: i++; blue: swap(S,i,b); b- -; }

but does it terminate? what happens if S[i] = S[b] = blue

Comparison problems

• We have seen search.

Comparison problems

• We have seen search.

Why is it closed?

Comparison problems

• We have seen search.

Why is it Ω(logn)?

Comparison problems

• We have seen search.

What are the allowed operations?

Comparison problems

• We have seen search.

Why is the decision tree binary?

Comparison problems

• We have seen search.

At least how many leaves?

Comparison problems

• We have seen search.

At least how high? Why?

Comparison problems

• Let's try the same for sort we have an O(n logn) upper bound

what would be the allowed steps?

Comparison problems

• Let's try the same for sort we have an O(n logn) upper bound

comparison, array element assignment, index arithmetic, establish sorted

Comparison problems

• Let's try the same for sort we have an O(n logn) upper bound

is the decision tree binary again?

Comparison problems

• Let's try the same for sort we have an O(n logn) upper bound

how many leaves this time?

Comparison problems

• Let's try the same for sort we have an O(n logn) upper bound

every leaf represents a specific permutation of the array

Comparison problems

• Let's try the same for sort we have an O(n logn) upper bound

how many different permutations?

Comparison problems

• Let's try the same for sort we have an O(n logn) upper bound

n! how high will the decision tree at least be?

Comparison problems

• Let's try the same for sort we have an O(n logn) upper bound

the decision tree will be at least log(n!)

high log(n!) = O(nlogn) so sort is Ω(nlogn), and thus closed.

Tournaments• Selecting the maximum element of an array. View it

as a tournament. Comparison is a match, winning is transitive:– A>B and B>C implies A>C, so A does not need to play C, and B and C can never be the winner of the tournament

• Many possible O(n) algorithms tennis(S): if (|S|==1) S1

else max(tennis(topHalf(S)),

tennis(bottomHalf(S))) sweep(S): winner= S1

for i=2 to n winner = max(winner,Si)

Runner up

ab c d

e fg h

ba

dc

fe

hg

tennis

sweep

which gives more info about runner up?

Assumptions about initial input

• Transitive and total ordering– transitive: a>b and b>c implies a>c (no roshambo)– total: for all a and b: either a>b or b>a– ie the elements can be ranked

• Any assignment of ranks to names is feasible – in previous example we could assign any of the 8! permutations of

rankings 1 to 8 to a to h

• Because we look for the winner, anyone who loses stops playing. Therefore there will be n-1 (n is number of inputs) games / comparisons for any tournament– notice: both sweep and tennis have 7 matches

Simpler case: n=4

• Any of the 24 assignments of 1..4 to a..d is valid• Build a decision tree for sweep (for n = 4), which is

sometimes called "caterpillar"

Decision tree for sweep

• Any of the 24 assignments of 1..4 to a..d is valid• Build a decision tree for sweep (for n = 4)

a:b ba

a:c ca b:c cb

c:d

dc

b:d

db

c:d

dca:d

da

Decision tree for sweep

• Any of the 24 assignments of 1..4 to a..d is valid• Build a decision tree for sweep (for n = 4)

a:b ba

a:c ca b:c cb

c:d

dc

b:d

db

c:d

dca:d

da

what do we know about the runner up for this outcome?

Decision tree for sweep

• Any of the 24 assignments of 1..4 to a..d is valid• Build a decision tree for sweep (for n = 4)

a:b ba

a:c ca b:c cb

c:d

dc

b:d

db

c:d

dca:d

da

what do we know about the runner up for this outcome?

tennis

• Build a decision tree for tennis (for n = 4)

Decision tree for tennis

• Build a decision tree for tennis (for n = 4)

a:b ba

c:d dc c:d dc

b:d

db

b:c

cb

a:d

daa:c

ca

Decision tree for tennis

• Build a decision tree for tennis (for n = 4)

a:b ba

c:d dc c:d dc

b:d

db

b:c

cb

a:d

daa:c

ca

what do we know about the runner up for this outcome?

Decision tree for tennis

• Build a decision tree for tennis (for n = 4)

a:b ba

c:d dc c:d dc

b:d

db

b:c

cb

a:d

daa:c

ca

what do we know about the runner up for this outcome?

Runner up

• In “The Art of Computer Programming, vol 3”, Donald Knuth proved that the tennis tournament algorithm provides the lower bound to determine the runner up:

In the case of tennis we need lg n more comparisons to find the runner up