CSCI 3160 Design and Analysis of Algorithms Tutorial 1 Chengyu Lin 1.
-
Upload
elijah-bridges -
Category
Documents
-
view
231 -
download
2
Transcript of CSCI 3160 Design and Analysis of Algorithms Tutorial 1 Chengyu Lin 1.
CSCI 3160 Design and Analysis of
Algorithms
Tutorial 1
Chengyu Lin
1
About me• Name: Chengyu Lin• Email: [email protected]• Office: SHB 117• Office hour: Friday 14:00 – 16:00
• You can always send me emails to make appointments
2
Asymptotic Notations• O(g(n))
o Big Oo Asymptotic upper bound
• Ω(g(n))o Big Omegao Asymptotic lower bound
• Θ(g(n))o Big Thetao Asymptotic tight bound
3
Asymptotic Notations• The time (space) complexity of an algorithm
usually depends on the input size, where the input could beo vertices/edges of a grapho a sequence of integers
• Usually the running time of algorithm grows with the input size
• Usually the running time is written as a function t(n), where n is the input size
4
Asymptotic Notations• Suppose we want to analyze the running time
of a program, e.g.
• This takes t(n) = n(n+1)/2 · n(n+1)/2 = n2(n+1)2/4 steps.
• Calculating the exact running time is tedious.
for (int i = 1; i <= n; ++i) { for (int j = i; j <= n; ++j) { for (int r = 1; r <= n; ++r) { for (int s = r; s <= n; ++s) { puts(“hello”); } } }}
5
Asymptotic Notations• Asymptotic notation simplifies the
calculations
• This takes t(n) = O(n2) · O(n2) = O(n4) steps.• and yet it still captures the behavior of t(n)
very well when the n is “large” enough
for (int i = 1; i <= n; ++i) { for (int j = i; j <= n; ++j) { for (int r = 1; r <= n; ++r) { for (int s = r; s <= n; ++s) { puts(“hello”); } } }}
6
Big O notation• We say that
t(n) = O(g(n)) ifthere exists a constant c > 0 such that t(n) ≤ c · g(n) for every sufficiently large n
• Intuitively, if t(n) is the running time, it tells us– the running time cannot be worse than g(n) by
a constant multiplicative factor
• Usually, g(n) looks simpler than t(n) (e.g. g(n) = n2, n!, log n, …)
7
Big O notation• We say that
t(n) = O(g(n)) ifthere exists a constant c > 0 such that t(n) ≤ c · g(n) for every sufficiently large n
1. The inequality only needs to hold for large n– e.g. n2 = O(2n)– 3n2 ≤ 2n is not true when n = 3– It holds for every n ≥ 4, so it’s okay
8
Big O notation• We say that
t(n) = O(g(n)) ifthere exists a constant c > 0 such that t(n) ≤ c · g(n) for every sufficiently large n
2. We can multiply g(n) by a positive constant– e.g. 4n2 = O(n2)– 4n2 ≤ n2 is not true for any n– But 4n2 ≤ 4n2 holds, so it’s okay
9
Big O notation• We say that
t(n) = O(g(n)) ifthere exists a constant c > 0 such that t(n) ≤ c · g(n) for every sufficiently large n
3. g(n) can be any upper bound– e.g. n2 = O(n2)– It is also true to say n2 = O(n100) or n2 = O(2n)
10
Convention• Actually, O(g(n)) is a set
o O(g(n)) = {t | ∃c > 0 such that t(n) ≤ c · g(n) for every large n}
o it contains all the functions that are upper bounded by g(n)
• When we say t(n) = O(g(n))• Actually we mean t(n) ∈ O(g(n))
• When we say O(f(n)) = O(g(n))• Actually we mean O(f(n)) ⊆ O(g(n))
o meaning if t(n) = O(f(n)) then t(n) = O(g(n)) 11
Comparing functions
• The functions above increases from left to right asymptoticallyo i.e. O(1) < O(loglog n) < O(log n) < log2 n =
(log n)2 < O(n1/3) < O(n) < O(n2) < O(2n) < O(n!) < O(nn)
o Logarithm is a lot smaller than polynomial• e.g. 2k log n = nk
o Polynomial is a lot smaller than exponential• e.g. log 2n = n
|← polynomial →|
… n1/3 n1/2 n n2 n3 … … 2n 2n^2 … 22^n … 2^2^…^2
|←exp→|
double exp
tower
…
O(1) , …, log* n, … loglog n, … log n, log2 n, …
12
Comparing functions
• The functions above increases from left to right asymptoticallyo When we compare two functions asymptotically,
compare the dominating terms on both sides firsto If they are of the same order, compare the next
dominating terms, and so on…o e.g. O(2n n2
log n) < O(2.1n n3 log2 n), since 2n < 2.1n
o the rest contributes very little when n is large
|← polynomial →|
… n1/3 n1/2 n n2 n3 … … 2n 2n^2 … 22^n … 2^2^…^2
|←exp→|
double exp
tower
…
O(1) , …, log* n, … loglog n, … log n, log2 n, …
13
Examples• 1 = O(log n)? • n3 = O(n2)? • log1000n = O(n)? • n3 = O(1.001n)? • 1.002n = O(1.001n n2)? • O((log n)1999) = O(n0.001)?• O(loglog n) = O(log3 n)?• O(2n n4 log n) = O(2n n4 log4 n)?
YesNo
Yes
No
NoYes (see slide 12)
Yes
Yes
14
Properties1. O(f(n)) + O(g(n)) = O(f(n) + g(n))2. O(max(f(n), g(n))) = O(f(n) + g(n))
• t(n) = O(n2) + O(n) = O(n2 + n) (by property 1)
• t(n) = O(n2 + n) = O(n2) (by property 2)
for (int i = 1; i <= n; ++i) { for (int j = i; j <= n; ++j) { puts(“CSCI3160); }}for (int i = 1; i <= n; ++i) { puts(“CSCI3160”);}
15
Properties3. O(f(n)) × O(g(n)) = O(f(n) × g(n))
e.g.
• O(n) × O(n) × O(n) = O(n3) (by property 3)
for (int i = 1; i <= n; ++i) { for (int j = i; j <= n; ++j) { for (int k = j; k <= n; ++k) { puts(“hello”); } }}
16
Big Omega notation• We say that
t(n) = Ω(g(n)) ifthere exists a constant c > 0 such that t(n) ≥ c · g(n) for every sufficiently large n
• Intuitively, if t(n) is the running time, it tells us– the running time cannot be better than g(n) by a
constant multiplicative factor• e.g. All comparison sort algorithms run in Ω(n log n) time
– Again, g(n) usually looks simple
17
Big Omega notation• We say that
t(n) = Ω(g(n)) ifthere exists a constant c > 0 such that t(n) ≥ c · g(n) for every sufficiently large n
1. The inequality only needs to hold for large n– e.g. 2n = Ω(n2)– 2n ≥ n2 is not true when n = 3– It holds for every n ≥ 4, so it’s okay
18
Big Omega notation• We say that
t(n) = Ω(g(n)) ifthere exists a constant c > 0 such that t(n) ≥ c · g(n) for every sufficiently large n
2. We can multiply g(n) by a positive constant– e.g. 0.1 n2 = Ω(n2)– 0.1n2 ≥ n2 is not true for any n– But 0.1n2 ≥ 0.1n2 holds, so it’s okay
19
Big Omega notation• We say that
t(n) = Ω(g(n)) ifthere exists a constant c > 0 such that t(n) ≥ c · g(n) for every sufficiently large n
3. g(n) can be any lower bound– e.g. n2 = Ω(n2)– It is also true to say n2 = Ω(n) or n2 = Ω(log n)
20
Properties1. Ω(f(n)) + Ω(g(n)) = Ω(f(n) + g(n))2. Ω(max(f(n), g(n))) = Ω(f(n) + g(n))3. Ω(f(n)) × Ω(g(n)) = Ω(f(n) × g(n))
21
Examples• 1 = Ω(log n)? No• n3 = Ω(n2)? Yes• log1000n = Ω(n)? No• n3 = Ω(1.001n)? Yes• 1.002n = Ω(1.001n n2)?
Yes
• 1 = O(log n)? Yes• n3 = O(n2)? No• log1000n = O(n)? Yes• n3 = O(1.001n)? No• 1.002n = O(1.001n n2)? No
• If t(n) = O(g(n)) then t(n) ≠ Ω(g(n))?• If t(n) = Ω(g(n)) then t(n) ≠ O(g(n))?
No!
• e.g. n3 = O(n3) = Ω(n3)22
Big Theta notation• Big O gives an asymptotical upper bound to t(n)
• Big Omega gives an asymptotical lower bound to t(n)
• There could be a gap between both bounds– The upper and lower bounds could be very loose– e.g. n2 = O(2n), n2 = Ω(1)
• When the upper bound matches the lower bound, we say this bound is tight (asymptotically)
23
Big Theta notation• We say that
t(n) = Θ(g(n)) ift(n) = O(g(n)) and t(n) = Ω(g(n))
• Combining the definitions of O(g(n)) and Ω(g(n)),t(n) = Θ(g(n)) ifthere exists a constant c1, c2 > 0 such that
c1 · g(n) ≤ t(n) ≤ c2 · g(n) for every sufficiently large n
• Intuitively, if t(n) is the running time, it tells us– the running time grows at about the same rate as g(n)
24
Examples1. t(n) = n3 - 4n2 + log n + 1o t(n) = O(n3)o t(n) = Ω(n3)o t(n) = Θ(n3)
25
Examples• 1 = Ω(log n)? No
• n3 = Ω(n2)? Yes
• log1000n = Ω(n)? No
• n3 = Ω(1.001n)? Yes
• 1.002n = Ω(1.001n n2)? Yes
• 1 = O(log n)? Yes
• n3 = O(n2)? No
• log1000n = O(n)? Yes
• n3 = O(1.001n)? No
• 1.002n = O(1.001n n2)? No
• If t(n) = O(g(n)) then t(n) ≠ Ω(g(n))?• If t(n) = Ω(g(n)) then t(n) ≠ O(g(n))?
No!
• e.g. n3 = O(n3) = Ω(n3)26
Properties• But we have
t(n) = O(g(n)) if and only if g(n) = Ω(t(n))
For Big Theta, we havet(n) = Θ(g(n)) if and only if g(n) = Θ(t(n))
1. Θ(f(n)) + Θ(g(n)) = Θ(f(n) + g(n))2. Θ(max(f(n), g(n))) = Θ(f(n) + g(n))3. Θ(f(n)) × Θ(g(n)) = Θ(f(n) × g(n))
27
Examples• Θ((log n)1999) = Θ(n0.001)?• Θ(loglog n) = Θ(log3 n)?• Θ(2n + n4 + log n) = Θ(2n + n4 + log4 n)?
NoNo
Yes
• Θ(2n + n4 + log n) = Θ(max{2n, n4, log n}) = Θ(2n)• Similarly, Θ(2n + n4 + log4 n) = Θ(2n)• t(n) = Θ(g(n)) if and only if g(n) = Θ(t(n))• So Θ(2n + n4 + log n) = Θ(2n) = Θ(2n + n4 + log4 n)
28
Asymptotic Notations• O(g(n)), big O• Ω(g(n)), big Omega• Θ(g(n)), big theta• In this course, we will use Big O notation a
lot• It is important to get familiar with it
• There are two other asymptotic notationso o(g(n)) (small o)o ω(g(n)) (small omega)
29
End• Questions
30