Discrete Optimization 2010 Lecture 2 Matroids & Shortest Pathsuetzm/do/DO_Lecture2.pdf · 2019. 11....

Matroids Shortest Paths

Discrete Optimization 2010Lecture 2

Matroids & Shortest Paths

Marc UetzUniversity of Twente

[email protected]

Lecture 2: sheet 1 / 25 Marc Uetz Discrete Optimization

mailto:[email protected]


Prim’s Algoritm (1957)

Algorithm 1: Prim

input : G = (V ,E , c)output: T ⊆ E , minimum spanning tree of Glet W = v0 arbitrary start vertex;T = ∅;for (i = 1, . . . , n − 1) do

ei = {v , u} := argmin{ce | e ∈ δ(W )}; // ei is

cheapest edge leaving W ; v ∈W and u 6∈W

T = T ∪ ei ;W = W ∪ u

Correctness follows directly from Cut Condition

Computation time can be shown in O( n2 )



More remarks about MST

Read literature

With a little more care (data structure & analysis), Kruskal’salgorithm can be implemented to run in O( m log m )

Krukal’s algorithm: Grows a forest (collection of several trees)on V

Prim’s algorithm: Grows one (minimal) spanning tree only, onsubset W of nodes of V

Take away message

A good bound on computation time may depend on clever datastructures (& tricks); just some (polynomial) bound is often easy



Outline

1 Matroids

2 Shortest PathsGeneral GraphsNonnegative Arc Lengths



Independence Systems and Matroids

Definition (Matroid)

Let S = {1, . . . , n} be a finite set and I ⊆ 2S be a family ofsubsets of S (the independent sets). Then M = (S , I) is anindependence system if:

1 ∅ ∈ I (∅ is independent)

2 If J ∈ I and I ⊆ J then I ∈ I (I is closed for “⊆”)

Moreover, M = (S , I) is a matroid if in addition:

3 For all A ⊆ S , all maximal independent subsets of A have thesame cardinality

if elements s of S have weights ws , call it weighted matroid

maximal independent subsets of A ⊆ S are called bases of A

rank r(A) = size of any basis of A (well-defined for matroids)



The Greedy Algorithm

Algorithm 2: Greedy

input : weighted matroid M = (S , I,w), any A ⊆ Soutput: minimum [maximum] weight basis F of Alet F = ∅; // note ∅ ∈ Iwhile (∃s ∈ A such that F ∪ {s} ∈ I) do

choose such s ∈ A with minimal [maximal] weight ws ;F ← F ∪ {s}; // greedily add s to F

Note: For any particular matroid, need an “oracle” (an algorithm)that tells us if F ∪ {s} ∈ I (i.e., if F ∪ {s} is independent)



Kruskal’s Algorithm Revisited

Kruskal’s Algorithm

Let T = ∅ (T is a forest)

while (∃ edge e ∈ E \ T , s.t. T ∪ {e} is forest)

pick such e with minimal cost ce

T ← T ∪ {e}

Observations

The set of forests of a graph are a matroid, and the bases arethe spanning trees (Exercise)

Kruskal’s algorithm is just the greedy algorithm applied tosome particular matroid



The Greedy Algorithm

Theorem

Greedy algorithm computes a min [max] weight basis of any A ⊆ S .

Proof

F = output of Greedy, T = minimal weight base, note |F | = |T |.Let F = {f1, . . . , f`},T = {t1, . . . , t`} both sorted by weights ↗Let k be minimal with wfk > wtk

Iteration k , why didn’t Greedy pick any ti of t1, . . . , tk?

ti ∈ {f1, . . . , fk−1}{f1, . . . , fk−1, ti} 6∈ I

In any case, we have that {f1, . . . , fk−1} is basis of{f1, . . . , fk−1, t1, . . . , tk} but {t1, . . . , tk} ∈ I, contradiction



Greedy Works only for Matroids

Theorem

Given an independence system M = (S , I), the greedy algorithmcomputes min [max] weight bases for all possible weights functionsws if and only if M is a matroid.

Proof (”only if ”)

Assume two bases F ,T of different cardinality, say |F | ≥ |T |+ 1Define weights

2 for all s ∈ S \ (F ∪ T )

1− ε for all s ∈ F for some ε < 1/|S |1 for all s ∈ T \ F (6= ∅)

and Greedy must fail to compute optimum, which has weight ≤ |T |



Greedy or Not Greedy

Matroids

forests of undirected graphs (your Exercise)

linear matroids: linearly independent vectors of a vector field(Steinitz exchange lemma ⇒ all bases have same length)

Independence systems, but no matroids

matchings in undirected graphs(but there exists a polytime algorithm - Edmonds 1965)

independent sets in graph G = (V ,E )(no polytime algorithm, unless P=NP)



Side Remark: Polymatroids

Let M = (S , I) be a matroid with rank function r

Consider Linear Program

maximize∑s∈S

wsxs

subject to∑s∈A

xs ≤ r(A), A ⊆ S

x ≥ 0

For A ⊆ S , let xA be the characteristic vector of A

If T ∈ I, xT is feasible solution (obvious)

If Greedy computes T ∗ as maximum weight base, xT∗ is anoptimal solution (Edmonds ’70)


Matroids Shortest Paths General Arc Lengths ≥ 0

Outline

1 Matroids

2 Shortest PathsGeneral GraphsNonnegative Arc Lengths



Shortest Path Problem

Given

digraph G = (V ,A), integer arc lengths c : A→ Z, s, t ∈ V

want: shortest path & length from s to t (to all nodes v ∈ V )

!"#$%&'%()*%"()$#+,&-

! !"#$%&'()*+,%+('-*.#/'01234564'.*,'7+"-%/8',&'5!"4'84%#3

! 9$%#$%&

! :/;*%+8%'#.%/'<84% =*;>'8'%;'%?

! :/;*%+8%'#.%/8'<84@ ."('()8%.",+8'(2@6'=*;>'8'%;'.77';%/+*'@

! "

#$

%&'(&)

! "

#$

%&'(&)

!

!

)

$

$

##

)

"

In undirected graphs:

=2

2

2



Three main cases

Arbitrary digraphs

includes negative cycle detection

Label correcting (Bellman-Ford) algorithm & Floyd-Warshall

O( nm ), respectively O( n3 ) time

Arbitrary digraphs, nonnegative arc lengths

Label setting (Dijkstra)

O( n2 ) time

Acyclic digraphs

Dynamic programming

O( n + m ) time



Shortest Path Optimality

Let d(v), v ∈ V be arbitrary node labels such thatd(v) ≥ shortest path length from s to v , v ∈ V

Theorem

Labels d(v) are shortest path lengths if and only if the4-inequality holds for all arcs (v ,w) ∈ A

d(w) ≤ d(v) + cvw

!"#$%&'%()*%"'(+,%-.*/-%0(1#23-%-#2

! !"#$%&'(!"#$%&'%(%$)(*+,#-'. )*(&!"%(+#$

! /+-(0123(12#43(5+(.67+(%$5"-$%$8(,60+(*%5+*.%$5"-$%$8(,60+(*%5+*. .9)'(-'%-(0123(".(%,(9&&+$(9&&+$(569,0569,0 :6$(-'+(.'6$-+.-(&%-'(:$67(.(-6(2(12#4;.3<

! ='+(,60+(*%5+*.(0123(%$+(.'6$-+.-(&%-'(*+,#-'.(":(%,0(6,*8(":":(%,0(6,*8(":

01>3(! 01,3 ? ),%- :6$(%**(12@>3#A

! "

#$

%&'(&)

!

!

)

$

$

#

"

)

#

!

*

!

"

"

+ 2

>)2>



Proof

Necessity

trivial, for otherwise there is a shortcut to w via v

Sufficiency

Let `(w) be the (true) shortest path length to w , `(w) ≤ d(w).pick shortest (s,w)-path P = (s, . . . u, v ,w), length `(w)use 4-inequality along this path

d(w) ≤ d(v) + cvw

≤ d(u) + cuv + cvw

...

≤ d(s) + length(P)

= `(w)



Shortest Path Algorithms, Ideas

Use previous theorem

Start with distance labels d(v) ≥ shortest path lengths, thencorrect violated 4-inequalities (in a clever way)

start with d(s) = 0 and d(v) =∞then, check (and correct) 4-inequality along arcs [that is, if

(d(w) > d(v) + cvw ) then d(w) = d(v) + cvw and pred(w) = v ]

say, in rounds where we always check 4-inequality of arcs inorder e1, e2, . . . , em

question: this stops? after how many rounds?Claim: ≤ (n − 1) rounds suffice, if shortest path lengths exist



Proof of Claim

note that shortest paths lengths exist⇔ G has no negative length cycle

consider any shortest path (s, u, v , . . . ,w) to any node w ,consisting of k (≤ n − 1) arcs

after first round, d(u) correctafter second round, d(v) correct. . .after kth (≤ n − 1) round, d(w) is correct



Bellman-Ford Algorithm

Algorithm

one round: label correcting all arcs a ∈ A in some order

if 4-inequality violated in nth round, return “negative cycle”

Time complexity

trivially, O( nm ) [n rounds of O( m )]

Correctness

If 6 ∃ negative length cycle, see previous slide

If (and only if) ∃ negative length cycle, some 4-inequality willstill be violated in nth round (actually, in any round)



All-Pairs Shortest Paths

Use n×Bellman-Ford, all-pairs shortest paths in O( n2m ) ∈ O( n4 )

Definition

For all v ,w ∈ V , define du(v ,w) as the length of a shortest(v ,w)-path that may only pass through nodes 1, 2, . . . , u

(Assume nodes are 1, 2, . . . , n, some order)

have d0(v ,w) = cv ,w if (v ,w) ∈ A, ∞ otherwise

by definition, dn(v ,w) is the shortest (v ,w)-path length

have the recurrence relation

du(v ,w) = min{du−1(v ,w), du−1(v , u) + du−1(u,w)}

So, given du−1, computation of du takes n2O( 1 ) time



Floyd-Warshall Algorithm

Algorithm 3: Floyd-Warshall

input : G = (V ,A, c)output: d(v ,w), pred(v ,w) for all v ,w ∈ V or “negative cycle”d(v ,w) = cvw and pred(v ,w) = v ∀ (v ,w) ∈ A;d(v ,w) =∞ and pred(v ,w) = ∅ ∀ (v ,w) 6∈ A;d(v , v) = 0 ∀ v ∈ V ;for (u = 1, . . . , n) do

for (all v ,w ∈ V ) doif (d(v ,w) > d(v , u) + d(u,w)) then

d(v ,w) = d(v , u) + d(u,w), pred(v ,w) = pred(u,w);if ((v == w) and d(v ,w) < 0) return “negative cycle”;

pred(v ,w) = last node before w on (v ,w)-path

correctness follows by correctness of recurrence relation

computation time, trivially n × n2 × O( 1 ) = O( n3 )



Dijkstra’s Algorithm (1959)

Shortest (s, v) paths for digraphs G = (V ,A, c) with c ≥ 0idea: only one correcting of each 4-inequality

Algorithm 4: Dijkstra

input : G = (V ,A, c) with c ≥ 0, start node s ∈ Voutput: d(v), pred(v) for all v ∈ Vd(s) = 0 and d(v) =∞ ∀v ∈ V \ s;S = ∅;while (S 6= V ) do

pick v ∈ V \ S with smallest label d(v);S = S ∪ v ; // d(v) permanentfor (all w with (v ,w) ∈ A) do

if (d(w) > d(v) + cvw ) thend(w) = d(v) + cvw and pred(w) = v ;



Example: Dijkstra

note: set S already has the correct distance labels at any iteration!



Correctness

We prove by induction on # iterations (ind. start is trivial)

Claim 1: d(u) ≤ d(w) for u ∈ S , w 6∈ S

Iteration S ← S ∪ v : d(u) ≤ d(v) ∀u ∈ S by hypothesisNext, d(v) ≤ d(w) ∀w 6∈ S by choice of v with min. d(v)Finally, d(u) ≤ d(v) ≤ d(v) + cvw = dnew(w) for relabeled w 6∈ S(last “≤”because cvw ≥ 0)

Claim 2: labels within S are correct (check 4-inequality)

Iteration S ← S ∪ v : By hypothesis, need only check arcs with v :By claim 1, d(u) ≤ d(v) ≤ d(v) + cvu for all u ∈ Sso any u ∈ S needs no relabeling via vAlso, d(v) ≤ d(u) + cuv at iteration u, and u wasn’t relabeled since

Upon termination S = V , so Dijkstra is correct



Computation Time

Simple

Initialization O( n )

n iterations of while-loop, in each need to find smallest labelin V \ S , which is doable in O( n ) O( n2 )

in total m relabeling steps of O( 1 ) O( m )

Which gives O( n ) + O( n2 ) + O( m ) ∈ O( n2 ) time

Less simple

An O( m log n ) implementation, which uses priority queue (alsocalled heap) to manage finding minimal d(v) in O( 1 ) time.(log n comes from overhead in organization of the heap)


Discrete Optimization 2010 Lecture 2 Matroids & Shortest Pathsuetzm/do/DO_Lecture2.pdf · 2019. 11....

Documents

Transcript of Discrete Optimization 2010 Lecture 2 Matroids & Shortest Pathsuetzm/do/DO_Lecture2.pdf · 2019. 11....