Algorithm Complexity & Big-O Notation: From the Basics

14
ϵ O ( Algorithm Complexity & Big-O Notation: From the Basics CompSci Club 29 May 2014

description

ϵ O (. Algorithm Complexity & Big-O Notation: From the Basics. CompSci Club 29 May 2014. History -- Number Theory. ←Edmund Landau (1877-1938). Intelligent mathematician of Germany Supervisor: Frobenius Dirichlet Series - PowerPoint PPT Presentation

Transcript of Algorithm Complexity & Big-O Notation: From the Basics

Page 1: Algorithm Complexity & Big-O Notation:  From the Basics

ϵO (

Algorithm Complexity & Big-O Notation: From the Basics

CompSci Club

29 May 2014

Page 2: Algorithm Complexity & Big-O Notation:  From the Basics

History -- Number Theory

• ←Edmund Landau (1877-1938)

Image source:http://www.ma.huji.ac.il/~landau/landau.jpg

• Intelligent mathematician of Germany• Supervisor: Frobenius• Dirichlet Series• Number theory – over 250 papers, simple proof of the Prime Number Theorem, development on algebraic number fields• *Asymptotic behavior of functions*;

O is for Order

Page 3: Algorithm Complexity & Big-O Notation:  From the Basics

History -- Application to CS• Big-O Notation -- used to study performance,

complexity of algorithms in Comp Sci– Execution Time T(n)–Memory Usage (hard drive, network use, etc)

• Performance – what are these variables?• Complexity – how does execution time change with

greater amnt of data?• Amortized analysis – studying the worst case

scenario of algorithms, using big-O notation, determining complexity

Page 4: Algorithm Complexity & Big-O Notation:  From the Basics

Definition & Notations, I

1. If there is number N and number c such that: f(x) ≤ c*g(x) for all x > N

Then we can write: f(N) ϵ O (g(N))N – problem size, input size, list sizeWe see various examples, as different functions:

O (N3)

O (aN)

O (log (N))

Page 5: Algorithm Complexity & Big-O Notation:  From the Basics

Definitions & Notations, II

• f(n) ϵ O (g(x)), f(x) ≤ c*g(x)• f(n) ϵ Θ (g(x)), f(x) = c*g(x)• f(n) ϵ Ω (g(x)), f(x) ≥ c*g(x)

*Note: in many texts these will be ‘equals’ signs, though many mathematicians such as myself find this to be inadequate notation(the ‘equals’ operator implies true converses, which is not true in all cases)

Page 6: Algorithm Complexity & Big-O Notation:  From the Basics

For Those of You in Calc Class…

2. If we know that:

lim _f(x)_

x→∞ g(x)

…then f(x) = o (g(x)).o However! This is actually little-oh notation (a stricter

quality of Big-Oh Notation)o There exists a number N such that f(x) < c*g(x) for

x > N and for all values of c.

= 0 ,

Page 7: Algorithm Complexity & Big-O Notation:  From the Basics

Example Problems• You may note that, in the coming examples, constant

values don’t end up mattering very much.• Dept. CS at Univ. Wisconsin-Madison describes /

proves the evaluation of complexity

1. Summing up the times of each statement

public void testComplexity () {

statement1;

statement2;

} = O (1)Solely a function of number of statements (N) …

Page 8: Algorithm Complexity & Big-O Notation:  From the Basics

Some More Complicated Examples

2. for-loop complexity• Proportional to the upper index of the loop

3. Nested for-loops, each starting at int ** = 0:• Proportional to O (N*M) or O (N2) if N=M

= O (N*M)

= O (N)

Page 9: Algorithm Complexity & Big-O Notation:  From the Basics

Some More Complicated Examples

for (int k = 0; k < N; k++){ for (int j = k; j < N; j++) { statements; }}

= O (N2)n(n+1)/2 = (1/2)(n2 ± n)

1 + 2 + 3 + 4 + … + N =

Page 10: Algorithm Complexity & Big-O Notation:  From the Basics

Some Practice Problems

What is the worst-case complexity of the each of the following code fragments?

Two loops in a row:

for (i = 0; i < N; i++)

{

sequence of statements

}

for (j = 0; j < M; j++)

{

sequence of statements

}

How would the complexity change if the second loop went to N instead of M?

http://pages.cs.wisc.edu/~vernon/cs367/notes/3.COMPLEXITY.html

Page 11: Algorithm Complexity & Big-O Notation:  From the Basics

Some Practice Problems

A nested loop followed by a non-nested loop:

for (i = 0; i < N; i++)

{

for (j = 0; j < N; j++)

{

sequence of statements

}

}

for (k = 0; k < N; k++)

{

sequence of statements

}

A nested loop in which the number of times the inner loop executes depends on the value of the outer loop index: for (i = 0; i < N; i++) { for (j = N; j > i; j--) { sequence of statements } }

http://pages.cs.wisc.edu/~vernon/cs367/notes/3.COMPLEXITY.html

Page 12: Algorithm Complexity & Big-O Notation:  From the Basics

When Does a Constant Matter?

• One can study time functions with greater specificity, for smaller differences in complexity

• T1 (N) = kN; T2 (N) = aNb, b > 1

T1 can become more efficient than T2 after a certain number of trials

Page 13: Algorithm Complexity & Big-O Notation:  From the Basics

Some Well-known Algorithms& Their Complexities

• From Wikipedia:Constant time: Size of arrayLogarithmic: BinarySearch AlgorithmQuadratic: Bubble Sort & Insertion Sort

• …And these can be verified by hand (Binary Search, List size, insertion sort, etc)

= O (1)

= O (log(N))

= O (N2)

Page 14: Algorithm Complexity & Big-O Notation:  From the Basics

Sources• Citedhttp://web.mit.edu/16.070/www/lecture/big_o.pdf

http://pages.cs.wisc.edu/~vernon/cs367/notes/3.COMPLEXITY.html

http://www-history.mcs.st-andrews.ac.uk/Biographies/Landau.html

http://en.wikipedia.org/wiki/Time_complexity#Table_of_common_time_complexities

http://mthcompsci.wordpress.com/