Data Structures

Haim Kaplan & Uri ZwickDecember 2013

Binary Heaps

1

Required ADT

• Maintain items with keys subject to

• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ)• Delete(x,Q)

2

Required ADT

• Decrease-key(x,Q,Δ): Can simulate by

Delete(x,Q), insert(x-Δ,Q)

We can use Red-Black trees to implement all operations in O(log n) time

We want to implement Decrease-key in O(1) (amortized) time

3

Motivation

• Dijkstra’s algorithm for single source shortest path

• Prim’s algorithm for minimum spanning trees

4

Motivation

• Want to find the shortest route from New York to San Francisco

• Model the road-map with a graph

5

A Graph G=(V,E)

V is a set of vertices

E is a set of edges (pairs of vertices)

6

Model driving distances by weights on the edges

2 19

9

1

5

13

1710

148

21

V is a set of vertices

E is a set of edges (pairs of vertices)

7

Source and destination

V is a set of vertices

E is a set of edges (pairs of vertices)

2 19

9

1

5

13

1710

148

21

8

Dijkstra’s algorithm

• Assume all weights are non-negative

• Finds the shortest path from some fixed vertex s to every other vertex

9

s

s1

5

103

1

4

2s3

s2

s4

15 8

Example

10

Maintain an upper bound d(v) on the shortest path to v

0

∞

∞

∞

∞

s

s1

5

103

1

4

2s3

s2

s4

15 8

Example

11

s

s1

5

103

1

4

2s3

s2

s4

15 8

Maintain an upper bound d(v) on the shortest path to v

0

∞

∞

∞

∞

A node is either scanned (in S) or labeled (in Q)

Initially S = and Q = V

12

s

s1

5

103

1

4

2s3

s2

s4

15 8

Pick a vertex v in Q with minimum d(v) and add it to S

0

∞

∞

∞

∞

Initially S = and Q = V

13

s

s1

5

103

1

4

2s3

s2

s4

15 8

Pick a vertex v in Q with minimum d(v) and add it to S

0

∞

∞

∞

∞

Initially S = and Q = V

v

w15

For every edge (v,w) where w in Q relax(v,w) 14

s

s1

5

103

1

4

2s3

s2

s4

15 8

Relax(v,w)

If d(v) + w(v,w) < d(w) then d(w) ← d(v) + w(v,w) π(w) ← v

0

∞

∞

∞

∞

v

w

For every edge (v,w) where w in Q relax(v,w)

15

15

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

∞

∞

∞

∞

S = {s}

Relax(s,s4)

16

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

∞

∞

15

∞

Relax(s,s4)

S = {s}

17

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

∞

∞

15

∞

Relax(s,s4)Relax(s,s3)

S = {s}

18

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

∞

∞

15

10

Relax(s,s4)Relax(s,s3)

S = {s}

19

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

∞

∞

15

10

Relax(s,s4)Relax(s,s3)Relax(s,s1)

S = {s}

20

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

10

Relax(s,s4)Relax(s,s3)Relax(s,s1)

S = {s}

21

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

10

S = {s}

Pick a vertex v in Q with minimum d(v) and add it to S

22

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

10

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

23

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

10

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s1,s3)

24

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

9

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s1,s3)

25

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

∞

15

9

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s1,s3)Relax(s1,s2)

26

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

9

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s1,s3)Relax(s1,s2)

27

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

9

S = {s,s1}

Pick a vertex v in Q with minimum d(v) and add it to S

28

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

9

S = {s,s1,s2}

Pick a vertex v in Q with minimum d(v) and add it to S

29

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

9

S = {s,s1,s2}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s2,s3)

30

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2}

Pick a vertex v in Q with minimum d(v) and add it to S

Relax(s2,s3)

31

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2}

Pick a vertex v in Q with minimum d(v) and add it to S

32

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2,s3}

Pick a vertex v in Q with minimum d(v) and add it to S

33

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2,s3,s4}

Pick a vertex v in Q with minimum d(v) and add it to S

34

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2,s3,s4}

When Q = then the d() values are the distances from s

The π function gives the shortest path tree

35

s

s1

5

103

1

4

2s3

s2

s4

15 8

0

5

6

15

8

S = {s,s1,s2,s3,s4}

When Q = then the d() values are the distances from s

The π function gives the shortest path tree

36

Implementation of Dijkstra’s algorithm

• We need to find efficiently the vertex with minimum d() in Q

• We need to update d() values of vertices in Q

37

Required ADT

• Maintain items with keys subject to

• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ)

38

Required ADT

• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ): Can simulate

by Delete(x,Q), insert(x-Δ,Q)

39

• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ): Can simulate

by Delete(x,Q), insert(x-Δ,Q)

How many times we do these operations ?

n = |V|

n

n

m = |E|

Total running time: O(m log n) → O(m + n log n)

40

Heap• Heap is

An almost complete binary tree The items at the children of v are

greater than the one at v

Heap-ordered tree

23

7

17

3365

1924 2629

7940

41

• Representing a heap with an array Get the elements from top to

bottom, from left to right

Array Representation

23

7

17

3365

1924 2629

7 23 17 24 29 26 19 65 33 40 79

7940

Q42

Array Representation

23

7

17

3365

1924 2629

7 23 17 24 29 26 19 65 33 40 79

7940

• Left(i): 2i+1• Right(i): 2i+2• Parent(i): (i-1)/2

Q43

0 1 2 3 4 5 6 7 8 9 10

Heap Representation I

info29key

11

Q

arraylen

9n

Store associated information directly in the array

Should only be used only for succinct information

john

Heap Representation II

info

key

11

Q

arraylen

9n

Store pointers to associated information

29

4

29

4infokey

pos

x

Heap Representation III

11

Q

arraylen

9n

Allow decrease-key operations

Find the minimum

23

7

17

3365

1924 2629

7 23 17 24 29 26 19 65 33 40 79

7940

• Return Q[0]

Q 47

Delete the minimum

23

7

17

3365

1924 2629

7 23 17 24 29 26 19 65 33 40 79

7940

Q 48

Delete the minimum

23 17

3365

1924 2629

23 17 24 29 26 19 65 33 40 79

7940

Q 49

Delete the minimum

23

79

17

3365

1924 2629

79 23 17 24 29 26 19 65 33 40

40

Q 50

Delete the minimum

23

17

79

3365

1924 2629

17 23 79 24 29 26 19 65 33 40

40

Q 51

Delete the minimum

23

17

19

3365

7924 2629

17 23 19 24 29 26 79 65 33 40

40

Q 52

Delete the minimum

23

17

19

3365

7924 2629

17 23 19 24 29 26 79 65 33 40

40

Q

Q[0] ← Q[size(Q)-1]

Heapify-down(Q,0)

size(Q) ← size(Q)-1

53

Heapify-down(Q,i)• Heapify-down(Q, i)• l ← left(i)• r ← right(i)• smallest ← i• if l < size(Q) and Q[l] < Q[smallest]

then smallest ← l• if r < size(Q) and Q[r] < Q[smallest]

then smallest ← r• if smallest > i then

Q[i] ↔ Q[smallest]

Heapify-down(Q, smallest)54

Inserting an item

23

17

19

3365

7924 2629

17 23 19 24 29 26 79 65 33 40

40

Q

Insert(15,Q)

55

Inserting an item

23

17

19

3365

7924 2629

17 23 19 24 29 26 79 65 33 40 15

40

Q

Insert(15,Q)Q[size(Q)] ← 15

15

Heapify-up(Q,size(Q))size(Q) ← size(Q) + 1

56

Heapify-up

23

17

19

3365

7924 2615

17 23 19 24 15 26 79 65 33 40 29

40

Q

29

57

15

17

19

3365

7924 2623

17 15 19 24 23 26 79 65 33 40 29

40

Q

29

58

Heapify-up

17

15

19

3365

7924 2623

15 17 19 24 23 26 79 65 33 40 29

40

Q

29

59

Heapify-up

Q

Heapify-up

• Heapify-up(Q, i)• while i > 0 and Q[i] < Q[parent(i)] do• Q[i] ↔ Q[parent(i)]• i ← parent(i)

17

15

19

3365

7924 2623

40 29

15 17 19 24 23 26 79 65 33 40 29 60

Other operations

• Decrease-key(x,Q,Δ)• Delete(x,Q)

• Can implement them easily using heapify

61

Heapsort (Williams, Floyd, 1964)

• Put the elements in an array• Turn the array into a heap• Do a delete-min and put the

deleted element at the last position of the array

• Reverse the array, or use a max-heap

62

65

79

26

3323

2924 1519

79 65 26 24 19 15 29 23 33 40 7

740

Q

Turn an array into a heap

63

0

1 2

3 4 5 6

7 8 9 10

65

79

26

3323

2924 1519

79 65 26 24 19 15 29 23 33 40 7

740

Q

Heapify-down(Q,4)

64

4

Turn an array into a heap

65

79

26

3323

2924 157

79 65 26 24 7 15 29 23 33 40 19

1940

Q

Heapify-down(Q,4)

65

Turn an array into a heap

65

79

26

3323

2924 157

79 65 26 24 7 15 29 23 33 40 19

1940

Q

Heapify-down(Q,3)

66

3

Turn an array into a heap

65

79

26

3324

2923 157

79 65 26 23 7 15 29 24 33 40 19

1940

Q

Heapify-down(Q,3)

67

Turn an array into a heap

65

79

26

3324

2923 157

79 65 26 23 7 15 29 24 33 40 19

1940

Q

Heapify-down(Q,2)

68

2

Turn an array into a heap

65

79

15

3324

2923 267

79 65 15 23 7 26 29 24 33 40 19

1940

Q

Heapify-down(Q,2)

69

Turn an array into a heap

65

79

15

3324

2923 267

79 65 15 23 7 26 29 24 33 40 19

1940

Q

Heapify-down(Q,1)

70

1

Turn an array into a heap

7

79

15

3324

2923 2665

79 7 15 23 65 26 29 24 33 40 19

1940

Q

Heapify-down(Q,1)

71

Turn an array into a heap

7

79

15

3324

2923 2619

79 7 15 23 19 26 29 24 33 40 65

6540

Q

Heapify-down(Q,1)

72

Turn an array into a heap

7

79

15

3324

2923 2619

79 7 15 23 19 26 29 24 33 40 65

6540

Q

Heapify-down(Q,0)

73

0

Turn an array into a heap

79

7

15

3324

2923 2619

7 79 15 23 19 26 29 24 33 40 65

6540

Q

Heapify-down(Q,0)

74

Turn an array into a heap

19

7

15

3324

2923 2679

7 19 15 23 79 26 29 24 33 40 65

6540

Q

Heapify-down(Q,0)

75

Turn an array into a heap

19

7

15

3324

2923 2640

7 19 15 23 40 26 29 24 33 79 65

6579

Q

Heapify-down(Q,0)

76

Turn an array into a heap

23

7

15

3379

2924 2619

6540

How much time does it take to build the heap this way ?

Total time = 77

At most n/2 nodes heapified at height 1

At most n/4 nodes heapified at height 2

At most n/8 nodes heapified at height 3

23

7

15

3379

2924 2619

6540

How much time does it take to build the heap this way ?

Total time = 78

At most n/2 nodes heapified at height 1

At most n/4 nodes heapified at height 2

At most n/8 nodes heapified at height 3

Summary

• We can build the heap in linear time

• We still have to deletemin the elements one by one in order to sort that will take O(nlog(n))

79

An example

• Given an array of length n, find the k smallest numbers.

80