Lower Bounds in Greedy Model

Sashka DavisAdvised by Russell Impagliazzo

(Slides modified by Jeff)

UC San DiegoOctober 6, 2006

Suppose you have to solve a problem Π…

Is there a Greedy algorithm that solves Π?Is there a Backtracking

algorithm that solves Π?

Is there a Dynamic Programming algorithm

that solves Π?

Eureka! I have a DP Algorithm!No Backtracking agl.

exists? Or I didn’t think of one?

Is my DP algorithm optimal or a better one

exists?

No Greedy alg. exists? Or I didn’t think of one?

Suppose we a have formal model of each algorithmic

paradigmIs there a Greedy

algorithm that solves Π?

No Greedy algorithm can solve Π exactly. Is there a Backtracking

algorithm that solves Π?No Backtracking algorithm

can solve Π exactly.

Is there a Dynamic Programming alg. that

solves Π?

DP helps!

Is my algorithm optimal, or a better DP

algorithm exists?

Yes, it is! Because NO DP alg. can solve Π more

efficiently.

The goal

• To build a formal model of each of the basic algorithmic design paradigms which should capture the strengths of the paradigm.

• To develop lower bound technique, for each formal model, that can prove negative results for all algorithms in the class.

Using the framework we can answer the following questions

1. When solving problems exactly:What algorithmic design paradigm can help?• No algorithm within a given formal model can solve the problem

exactly.• We find an algorithm that fits a given formal model.

2. Is a given algorithm optimal?• Prove a lower bound matching the upper bound for all algorithms

in the class.

3. Solving the problems approximately:• What algorithmic paradigm can help?• Is a given approximation scheme optimal within the formal

model?

Some of our results

ADAPTIVEPRIORITY

FIXED

GreedyGreedy

Backtracking & Simple DP

(tree)

Backtracking & Simple DP

(tree)

DynamicProgramming

DynamicProgramming

pBT

pBP

Online

is a set of data items; is a set of options

Input: instance I={1 ,2 ,…,n }, I

Output: solution S={(i , i) | i= 1,2,…,d}; i

1. Order: Objects arrive in worst case order chosen by adversary.

2. Loop considering i in order.

– Make a irrevocable decision i

On-line algorithms

is a set of data items; is a set of options

Input: instance I={1 ,2 ,…,n }, I

Output: solution S={(i , i) | i= 1,2,…,d}; i

1. Order: Algorithm chooses fixed π : →R+

without looking at I.

2. Loop considering i in order.

– Make a irrevocable decision i

Fixed priority algorithms

is a set of data items; is a set of options

Input: instance I={1 ,2 ,…,n }, I

Output: solution S={(i , i) | i= 1,2,…,d}; i

2. Loop

- Order: Algorithm reorders π : →R+

without looking at rest of I.

- Considering next i in current order.

– Make a irrevocable decision i

Adaptive priority algorithms

is a set of data items; is a set of options

Input: instance I={1 ,2 ,…,n }, I

Output: solution S={(i , i) | i= 1,2,…,d}; i

1. Order: Algorithm chooses π : →R+

without looking at I.

2. Loop considering i in order.

– Make a set of decisions i

(one of which will be the final decision.)

Fixed priority “Back Tracking”

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Shortest Path in negative graphs no cycles

Bellman-Ford

Shortest Path in no-negative graphs

Dijkstra’s

Online

Maximum Matching in Bipartite graphs

Flow Algorithms

Maximum Matching in Bipartite graphs

Minimum Spanning Tree

Prim’sKruskal’sKruskal’s

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Dijkstra’s

Shortest Path in no-negative graphs

OnlinePrim’sKruskal’s

Minimum Spanning Tree

Kruskal’s

Kruskal algorithm for MST is a Fixed priority algorithm

Input (G=(V,E), ω: E →R)1. Initialize empty solution T2. L = Sorted list of edges in non-decreasing order

according to their weight3. while (L is not empty)

– e = next edge in L– Add the edge to T, as long as T remains a forest

and remove e from L4. Output T

Prim’s algorithm

Input G=(V,E), w: E →R

1. Initialize an empty tree T ← ; S ← 2. Pick a vertex u; S={u};

3. for (i=1 to |V|-1)– (u,v) = min(u,v)cut(S, V-S)w(u,v)

– S←S {v}; T←T{(u,v)}

4. Output T

Prims algorithm for MST is an adaptive priority algorithm

Dijkstra’s Shortest Paths Alg is an adaptive priority algorithm

• Dijkstra algorithm (G=(V,E), s V)• T← ; S←{s};∅• Until (S≠V)• Find e=(u,x) | e = mineCut(S, V-S){path(s, u)+ω(e)}• T← T{e}; S ← S {x}

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Dijkstra’s

Shortest Path in no-negative graphs

OnlinePrim’sKruskal’s

Minimum Spanning Tree

Kruskal’s

ShortPath Problem: Given a graph G=(V,E),

ω: E →R+; s, t V. Find a directed tree of edges,

rooted at s, such that the combined weight of the

path from s to t is minimal

• Theorem: No Fixed priority algorithm can achieve any constant approximation ratio for the ShortPath problem

Some of our results

• Data items are edges of the graph• Decision options = {accept, reject}

Fixed priority game

Solver Adversaryγd

γi

γ3

γj γk

γ2γ1

Γ0

S_sol = {(γi2,σi2)}

σi2

S_sol = {(γi2,σi2), (γi4,σi4)}

γi2 γi9,…γi1 γi3 γi4 γi5 γi6 γi7 γi8Γ0Γ1 Γ2

σi4

Γ3

End GameS_adv = {(γi2,σ*

i2), (γi4,σ*i4)}

Solver is awarded(S_sol)

(S_adv)

f

f

=∅

Adversary selects 0

t

b

s

a u(k)

w(k)

x(1)

v(1)

y(1)

z(1)

0 , , , , ,u v w x y z

Solver selects an order on 0

If then the Adversary presents: ( ) ( )y z

t

b

s

a u(k)

w(k)

x(1)

v(1)

y(1)

z(1)

1 , , ,u x y z

Adversary’s strategy

Waits until Solver considers edge y(1)

Solver will consider y(1) before z(1)

Event 1σy=accept

Event 2σy=reject

Event 1: Solver accepts y(1)

t

u(k)

x(1)

y(1)

z(1)

b

a

s

The Solver constructs a path {u,y}The Adversary outputs solution {x,z}

1

2

Alg k

Adv

Event 2: Solver rejects y(1)

The Solver fails to construct a path.The Adversary outputs a solution {u,y}.

t

u(k)

x(1)

y(1)

z(1)

b

a

s

The outcome of the game:

• The Solver either fails to output a solution or achieves an approximation ratio (k+1)/2

• The Adversary can set k arbitrarily large and thus can force the Algorithm to claim arbitrarily large approximation ratio

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Dijkstra’s

Shortest Path in no-negative graphs

Online

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Interval Schedulingvalue is width

Factor of 3Online

Factor of 3

Interval scheduling on a single machine

• Instance:

Set of intervals I=(i1, i2,…,in), j ij=[rj, dj]

• Problem: schedule intervals on a single machine

• Solution: S I• Objective function: maximize iS(dj - rj)

A simple solution (LPT)

Longest Processing Time algorithm input I=(i1, i2,…,in)

1. Initialize S ← 2. Sort the intervals in decreasing order (dj – rj)

3. while (I is not empty) Let ik be the next in the sorted order

If ik can be scheduled then S ← S U {ik};

I ← I \ {ik}

4. Output S

LPT is a 3-approximation

• LPT sorts the intervals in decreasing order according to their length

• 3 LPT ≥ OPT

OPT OPT OPT

LPT

ri di

Example lower bound [BNR02]

• Theorem1: No adaptive priority algorithm can achieve an approximation ratio better than 3 for the interval scheduling problem with proportional profit for a single machine configuration

Proof of Theorem 1• Adversary’s move

• Algorithm’s move: Algorithm selects an ordering

• Let i be the interval with highest priority

1

2

3

q

q-1q-11

2

3

e

Adversary’s strategy

• If Algorithm decides not to schedule i • During next round Adversary removes all

remaining intervals and schedules interval i

1

2

3

i

jk1

2

3

i

Alg’s value = 0Adv’s value = i

Adversary’s strategy

• If i = and Algorithm schedules i• During next round the Adversary restricts

the sequence:

i

jk1

2

3

i

i

i+1i-1

Alg’s value = iAdv’s value = (i-1)+3(i/3)+(i+1)=3i

Adversary’s strategy

• If i = and Algorithm schedules i• During next round the Adversary restricts

the sequence:

1

Alg’s value = 1Adv’s value = 3(1/3)+(2)=3

1

2

3

i

jk1

2

31

2

Adversary’s strategy

• If i = and Algorithm schedules i• During next round the Adversary restricts

the sequence:

1

2

3

i

jk1

2

3

q

q

q-1q-1

Alg’s value = qAdv’s value = (q-1)+3(q/3)+(q-1)=3q-1

But q is big

Adversary’s strategy

• If i = and Algorithm schedules i• During next round Adversary restricts the

sequence:

1

2

3

m

jk1

2

3

i

i

m

Alg’s value = iAdv’s value = (3i) =3i

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Interval Schedulingvalue is width

Factor of 3Online

Factor of 3

The algorithm was missed up beforeit got a chance to reorder things.

?

Some of our results

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Weighted Vertex Cover

Factor of 2

Online

[Joh74] greedy 2-approximation for WVC

Input: instance G with weights on nodes.

Output: solution S V covers all edges and minimizes weight taken nodes.

Repeat until all edges covered.• Take v minimizing ω(v)/(# uncovered adj edges)

Weighted Vertex Cover

• With Shortest Path,a data item is an edge of the graph – = (<u,v>, ω(<u,v>) )

• With weighted vertex cover,– A data item is a vertex of the graph

= (v, ω(v), adj_list(v))

• (Stronger than having the items be edges,because the alg gets more info from nodes.

Weighted Vertex Cover

Theorem: No Adaptive priority algorithm can

achieve an approximation ration better than 2

Adaptive priority gameSolver Adversary

γ3

γ5

γ6

γ1 γ4

γ7

γ2

S_sol = {(γ7,σ7)}σ4S_sol = {(γ7,σ7), (γ4,σ4)}

Γ3Γ1Γ2

σ7

The Game Ends:1. S_adv = {(γ7,σ*

7), (γ4,σ*4),(γ2,σ*

2)}2. Solver is awarded payoff

f(S_sol)/f(S_adv)

γ8

γ9γ10

γ11

γ12

Γ0

σ2 S_sol = {(γ7,σ7), (γ4,σ4),(γ2,σ2)}

The Adversary chooses instances to be graphs Kn,n

The weight function ω:V→ {1, n2}

n2

1n2

n2

n2

11

1

The game

• Data items– each node appears in o as two separate data

items with weights 1, n2 • Solver moves

– Choses a data item, and commits to a decision

• Adversary move– Removes from the next t the data item,

corresponding to the node just committed and..

Adversary’s strategy is to wait unitl

Event 1: Solver accepts a node of weight n2

Event 2: Solver rejects a node of any weight

Event 3: Solver has committed to all but one nodes on either side of the bipartite

1

1

1

11

Event 1: Solver accepts a node ω(v)=n2

A B

The Adversary chooses part B of the bipartite as a cover, and incurs cost n

The cost of a cover for the Solver is at least n2+n-12lg

2A n

Adv n

1

1

n2

1

1

1

1

Event 2: Solver rejects a node of any weight

A B

The Adversary chooses part A of the bipartite as a cover.The Solver must choose part B of the bipartite as a cover.

2

2

lg 22

A n

Adv n n

n2

n2

Event 3: Solver commits to n-1 nodes w(v)=1, on either side of Kn,n

A B

The Adversary chooses part B of the bipartite as a cover, and incurs cost n

The cost of a cover for the Solver is 2n-1lg 2 1

2A n

Adv n

1

1

1

1

1

1

1

n2

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Weighted Vertex Cover

Factor of 2

Online

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Facility Location

Factor of logn

Online

Facility location problem

• Instance is a set of cities and set of facilities– The set of cities is C={1,2,…,n}– Each facility fi has an opening cost cost(fi) and

connection costs for each city: {ci1, ci2,…, cin}

• Problem: open a collection of facilities such that each city is connected to at least one facility

• Objective function: minimize the opening and connection costs min(ΣfScost(fi) + ΣjCmin fiScij )

[AB02] result

• Theorem: No adaptive priority algorithm can achieve an approximation ratio better than log(n) for facility location in arbitrary spaces

Adversary presents the instance:

• Cities: C={1,2,…,n}, where n=2k

• Facilities: – Each facility has opening cost n– City connection costs are 1 or ∞ – Each facility covers exactly n/2 cities– cover(fj) = {i | i C,cji=1}

Cu denotes the set of cities not yet covered by the solution of the Algorithm

Adversary’s strategy

At the beginning of each round t – The Adversary chooses St to consist of

facilities f such that fSt iff |cover(f) ∩ Cu| = n/(2t)

– The number of uncovered cities Cu is n/(2t-1)

Two facilities are complementary if together they cover all cities in C. For any round t St consists of complementary facilities

The game

Uncovered cities Cu

End of the game

• Either Algorithm opened log(n) facilities or failed to produce a valid solution

• Cost of Algorithm’s solution is n.log(n)+n

• Adversary opens two facilities incurs total cost 2n+n

Some of our results

PRIORITY

pBT

pBP

ADAPTIVEPRIORITY

FIXED

Facility Location

Factor of logn

Online