Post on 22-Aug-2020
Shortest Path Algorithms
Deliverables
Shortest Path Algorithms
Dijkstra Algorithm
Bellman Ford Algorithm
Ford Fulkerson
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 2
Definition
Given a Graph Path P= V1 V2 V3… Vk has weight
W(p) =
Shortest path from u to v is a path of minimum possible weight from u to v. Shortest path weight from u to v is
δ(u, v) = min{ W(p) : P from u to v}
= ∞ if there is no path from u to v
Negative edge weights does not affect the definition of shortest path, however if there is a cycle containing negative edge weight then the shortest path does not exist and δ(u, v) = - ∞
Copyright @ gdeepak.com®
1
1)
1
( ~
k
i i
i
w V V
6/12/2012 6:29 PM 3
Optimal substructure property
• A sub path of a shortest path is a shortest path
• U to V has a shortest path including the sub path X to Y. If there is an alternative shortest path from x to y, then that should have been included in calculating shortest path from u to v.
Copyright @ gdeepak.com®
u x y v
Hypothetical shortest path
6/12/2012 6:29 PM 4
Triangle Inequality
• For all vertices u, v, x є V
• δ (u, v) ≤ δ(u, x) + δ (x, v)
Copyright @ gdeepak.com®
u
x
v δ(u,v)
δ(x,v) δ(u,x)
6/12/2012 6:29 PM 5
Single Source Shortest Path
From a given source vertex s є V find shortest path weight δ (s, v) for all v є V.
The best way to solve from a to b is no easier then solving from a to every one else.
Assuming No negative weights exist and w(u, v) ≥ 0
Means shortest paths exist provided path exist otherwise δ(u, v) = -∞
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 6
Greedy approach
1. Maintain set S of vertices whose shortest path distance from s is known s є S
2. At each step add to S one vertex v є {V-S} whose estimated distance from s is
minimum.
3. Update distance estimates of vertices adjacent to v.
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 7
Dijkstra Example
• Q A B C D E
Copyright @ gdeepak.com®
A
B
C
D
E
10
3
2
1 4
2
9 7
8
6/12/2012 6:29 PM 8
Dijkstra Example
• Q A B C D E
• 0 ∞ ∞ ∞ ∞
Copyright @ gdeepak.com®
A
B
C
D
E
10
3
2
1 4
2
9 7
8
∞
∞
∞
∞
0
s
6/12/2012 6:29 PM 9
Dijkstra Example
• Q A B C D E
• 0 ∞ ∞ ∞ ∞
• 10 3 ∞ ∞
Copyright @ gdeepak.com®
A
B
C
D
E
10
3
2
1 4
2
9 7
8
10
3
∞
∞
0
s
6/12/2012 6:29 PM 10
Dijkstra Example
• Q A B C D E
• 0 ∞ ∞ ∞ ∞
• 10 3 ∞ ∞
• 7 11 5
Copyright @ gdeepak.com®
A
B
C
D
E
10
3
2
1 4
2
9 7
8
7
3
11
5
0
s
6/12/2012 6:29 PM 11
Dijkstra Example
• Q A B C D E
• 0 ∞ ∞ ∞ ∞
• 10 3 ∞ ∞
• 7 11 5
• 7 11
Copyright @ gdeepak.com®
A
B
C
D
E
10
3
2
1 4
2
9 7
8
7
3
11
5
0
s
6/12/2012 6:29 PM 12
Dijkstra Example
• Q A B C D E
• 0 ∞ ∞ ∞ ∞
• 10 3 ∞ ∞
• 7 11 5
• 7 11
• 9
Copyright @ gdeepak.com®
A
B
C
D
E
10
3
2
1 4
2
9 7
8
7
3
9
5
0
s
6/12/2012 6:29 PM 13
Algorithm
• Dijkstra Algorithm(G,V,E) • d[s] 0 • for each v є V –{s} • do d[v] ∞ • S Φ • Q V • while Q ≠ Null • do u Extract-min(Q) • S S Union {u} • for each v є Adj(u) • do if d[v] > d[u] + w(u, v) • then d[v] d[v] + w(u, v) # implicit decrease key or relaxation step
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 14
Time Complexity
• Shortest path tree is union of all shortest path.
• Complexity = |v| extract-min + |E| decrease key
• For Array O(V2)
• For Binary heap O(V+E) lgV
• For Fibonacci Heap O(E+V lg V)
• Dijkstra is equivalent to a BFS if weights of all the edges are taken as one.
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 15
Bellman Ford Algorithm
• Computes δ (s, v) from source vertex s є V to all vertices v є V or reports that a negative weight cycle exists
• Bellman_Ford(G, w, s) • D[s] 0 • For each v є V-{s} • do d[v] ∞ • For i 1 to |v|-1 • do for each edge (u, v) є E • do if d[v] > d[u] + w(u, v) • then d[v] d[u] + w(u, v) • For each edge (u, v) є E • do if d[v] > d[u] + w(u, v) • then report that a negative weight cycle exists • Else d[v] = δ (s, v)
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 16
Time Complexity
Nonnegative edge weights
Dijkstra’s algorithm: O(E + V lg V)
General
Bellman-Ford: O(VE)
DAG
One pass of Bellman-Ford: O(V + E)
Bellman Ford is Slower then Dijkstra but Dijkstra does not handle negative edge weights
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 17
Bellman Ford Example
• A B C D E
Copyright @ gdeepak.com®
A
B
E
C D
-1
3
4
2
5
2
1
-3
6/12/2012 6:29 PM 18
Bellman Ford Example
• A B C D E
• 0 ∞ ∞ ∞ ∞
Copyright @ gdeepak.com®
A
B
E
C D
-1
3
4
2
5
2
1
-3
∞
∞
0
∞
∞
#1
#5 #8
#6
#4
#7
#2
#3
6/12/2012 6:29 PM 19
Bellman Ford Example
We start from #1, ∞ +2 is ∞ so no change
Then edge #2, ∞ +2 is ∞ so no change
Then edge #3, ∞ +1 is ∞ so no change
Then edge #4, 0 + (-1) is -1 which is less than ∞ so change to -1
Then edge #5, 0 +4 Which is less that ∞ so change it to 4
Then edge #6, ∞ +5 is ∞ so no change
Then edge #7, -1 +3 is 2 so change 4 to 2
Then edge #8, ∞ - 3 is ∞ so no change
6/12/2012 6:33 PM Copyright @ gdeepak.com® 20
Bellman Ford Example
• A B C D E
• 0 ∞ ∞ ∞ ∞
• 0 -1 ∞ ∞ ∞
• 0 -1 4 ∞ ∞
Copyright @ gdeepak.com®
A
B
E
C D
-1
3
4
2
5
2
1
-3
∞
-1
0
4
∞
#1
#5 #8
#6
#4
#7
#2
#3
6/12/2012 6:29 PM 21
Bellman Ford Example
• A B C D E
• 0 ∞ ∞ ∞ ∞
• 0 -1 ∞ ∞ ∞
• 0 -1 4 ∞ ∞
• 0 -1 2 ∞ ∞
Copyright @ gdeepak.com®
A
B
E
C D
-1
3
4
2
5
2
1
-3
∞
-1
0
2
∞
#1
#5 #8
#6
#4
#7
#2
#3
6/12/2012 6:29 PM 22
Bellman Ford Example
• A B C D E
• 0 ∞ ∞ ∞ ∞
• 0 -1 ∞ ∞ ∞
• 0 -1 4 ∞ ∞
• 0 -1 2 ∞ ∞
• 0 -1 2 ∞ 1
Copyright @ gdeepak.com®
A
B
E
C D
-1
3
4
2
5
2
1
-3
∞
-1
0
2
1
#1
#5 #8
#6
#4
#7
#2
#3
6/12/2012 6:29 PM 23
Bellman Ford Example
• A B C D E
• 0 ∞ ∞ ∞ ∞
• 0 -1 ∞ ∞ ∞
• 0 -1 4 ∞ ∞
• 0 -1 2 ∞ ∞
• 0 -1 2 ∞ 1
• 0 -1 2 1 1
Copyright @ gdeepak.com®
A
B
E
C D
-1
3
4
2
5
2
1
-3
1
-1
0
2
1
#1
#5 #8
#6
#4
#7
#2
#3
6/12/2012 6:29 PM 24
Bellman Ford Example
• A B C D E
• 0 ∞ ∞ ∞ ∞
• 0 -1 ∞ ∞ ∞
• 0 -1 4 ∞ ∞
• 0 -1 2 ∞ ∞
• 0 -1 2 ∞ 1
• 0 -1 2 1 1
• 0 -1 2 -2 1
Copyright @ gdeepak.com®
A
B
E
C D
-1
3
4
2
5
2
1
-3
-2
-1
0
2
1
#1
#5 #8
#6
#4
#7
#2
#3
6/12/2012 6:29 PM 25
Few points
It is also possible to implement this algorithm in distributed systems because the relaxation step is local and can be independent of other parts.
Bellman ford is used a lot in internet to find the shortest path
If bellman ford fails to converge after |v-1| rounds then there is a negative edge cycle
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 26
All Pairs Shortest Path
• Non Negative Edge Weights
• Dijkstra’s Algorithm |V| times: O(VE + V2lgV)
• Bellman Ford ( once from each vertex)
• Time O(V2E) so for Dense Graph O(v4)
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 27
Another Approach
Consider nXn adjacency matrix A=(aij) of the digraph and define
(dij)(m) =Weight of a shortest path from i to j that uses at
most m edges.
dij(0) = 0 if i=j
dij(0) = ∞ if i≠j
and for m= 1,2,…,n-1
(dij)(m) = mink
{(dik)(m-1) +akj}
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 28
(dij)(m) = mink
{(dik)(m-1) +akj}
for m← 1 to n-1
do for i ← 1 to n
do for j ← 1 to n
do for k ← 1 to n
do if dij > dik + akj
then dij ← dik + akj
Complexity O(n4)
6/12/2012 6:29 PM Copyright @ gdeepak.com® 29
Relaxation Step
No negative weight cycle means
δ (i, j) = dij(n-1)=dij
(n) =dij(n+1)=…
6/12/2012 6:29 PM Copyright @ gdeepak.com® 30
Similarity to Matrix Multiplication
Compute C=A . B, where C, A and B are n x n matrices: cij= 𝑎𝑖𝑘𝑏𝑘𝑗
𝑛𝑘=1
Time = O(n3) using the standard algorithm
There is an identical pattern in the All pair shortest Path if we map + to min and X to +
cij = mink{aik+bkj}
D(m)=D(m-1)XA
where A0= Identity Matrix= 1 ∞ ∞∞ 1 ∞∞ ∞ 1
= I= D0=dij(0)
6/12/2012 6:29 PM Copyright @ gdeepak.com® 31
Similarity to matrix multiplication
• (min,+) multiplication is associative and with real numbers.
• D1=D0.A=A1
• D2=D1.A=A2
• .
• .
• Dn-1=Dn-2.A=An-1 which is equal to δ (i,j)
• Time=O(n.n3) = O(n4) which is no better then the previous ones
6/12/2012 6:29 PM Copyright @ gdeepak.com® 32
Improved Matrix Multiplication
• A2k= AkXAk
• Computing A2, A4,…… in O(lgn) squaring's
• Time = O(n3lgn)
• It can be further improved with Strassen matrix multiplication.
• To detect negative weight cycle check the diagonal for negative values in O(n) additional time
6/12/2012 6:29 PM Copyright @ gdeepak.com® 33
Floyd Warshall Algorithm
cij(k)weight of a shortest path from i to j with intermediate
vertices belonging to the set {1, 2, …, k}
δ(i,j)=cij(n)=aij
6/12/2012 6:29 PM Copyright @ gdeepak.com® 34
Floyd-Warshall Recurrence
6/12/2012 6:29 PM Copyright @ gdeepak.com® 35
Floyd Warshall Recurrence
for k← 1 to n
do for i ← 1 to n
do for j ← 1 to n
do if cij > cik + ckj
then cij ← cik + ckj
• Complexity O(n3)
• It is simple and efficient
6/12/2012 6:29 PM Copyright @ gdeepak.com® 36
Relaxation
Reweighing of edges
Given a label h(v) for each v ∈ V, reweight each edge (u, v) ∈ E by ŵ(u, v) = w(u, v) + h(u) – h(v).
Then, all paths between the same two vertices are reweighted by the same amount
6/12/2012 6:29 PM Copyright @ gdeepak.com® 37
Johnson’s Algorithm
1. Find a vertex labelling h such that ŵ(u, v) ≥ 0 for all (u, v) ∈ E by using Bellman-Ford to solve the difference constraints h(v) – h(u) ≤ w(u, v), or determine that a negative-weight cycle exists. Time = O(VE). 2. Run Dijkstra’s algorithm from each vertex using ŵ. Time = O(VE + V2 lg V). 3. Reweight each shortest-path length ŵ(p) to produce the shortest-path lengths w(p) of the original graph. Time = O(V2). Total time = O(VE + V2 lg V).
6/12/2012 6:29 PM Copyright @ gdeepak.com® 38
Johnson’s algorithm
3 7
4
2
8
-4
6
1 -5
Johnson’s algorithm
3 7
4
2
8
-4
6
1 -5
s
0
0
0
0
0
Add a point s, with a weight 0 edge to each point
Johnson’s algorithm
-1
0 -5
-4 0
3 7
4
2
8
-4
6
1 -5
s
0
0
0
0
0
Implementation of the Bellman-Ford algorithm, a self-s starting shortest distance.
Apply Reweighing
• For edge (0,-4) with w(u, v) as -4 New weight is
w(u, v) + h(u) – h(v) which is [-4+ 0 – (-4) ] = 0
• For edge (0,-1) with w(u, v) as 3 New weight is
w(u, v) + h(u) – h(v) which is [3+ 0 – (-1) ] = 4
and so on
6/12/2012 6:40 PM Copyright @ gdeepak.com® 42
Johnson’s algorithm
-1
0 -5
-4 0
4 10
0
2
13
0
2
0 0
Done
reweighting
Interesting
• Longest path problem is NP-complete because of presence of positive weight cycles.
• DAG’s with negative weights can be done in O(e)
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 44
Questions, Comments and Suggestions
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 45
Question 1
Adding same weight to all the graph edges keeps the shortest path intact?
A) True
B) False
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 46
Question 2
Given a graph G = (V;E) with positive edge weights, the Bellman-Ford algorithm and Dijkstra’s algorithm can produce different shortest-path trees despite always producing the same shortest-path weights. Reason?
A) True
B) False
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 47
Question 3
If all edges in a graph have distinct weights, then the shortest path between two vertices is unique.
• A) True
• B) False
Copyright @ gdeepak.com® 6/12/2012 6:29 PM 48