CS2012 Programming Techniques II

Mon 29 Sep 2014Lecture 41

Running Time Performance analysisTechniques until now:ExperimentalCost modelscounting execution of operations or lines of code.under some assumptionsonly some operations countcost of each operation = 1 time unitTilde notation: T(n) ~ (5/3)n2

Today:-notationExamples: insertionSort & binarySearchMon 29 Sep 2014Lecture 42InsertionSort PseudocodeMon 29 Sep 2014Lecture 43

Algorithm (in pseudocode)for (j = 1; j=0 and A[i]>A[i+1] { swap A[i], A[i+1] i=i-1}}return AWorst CaseMon 29 Sep 2014Lecture 44costno of timesfor (j = 1; j=0 and A[i]>A[i+1] {12++n swap A[i], A[i+1]11++(n-1) i=i-111++(n-1)}}return A11In the worst case the array is in reverse sorted order.

T(n) = n + n-1 + Sumx=2..n(x) + 2Sumx=1..n-1(x-1) + 1 =n + n-1 + (n(n+1)/2 - 1) + 2n(n-1)/2 + 1 = (3/2)n2 + (3/2)n - 1We also saw best- and average-caseMon 29 Sep 2014Lecture 45How fast is T(n) = (3/2)n2 + (3/2)n - 1 ?Fast computer vs. Slow computer

Mon 29 Sep 2014Lecture 46T1(n) = (3/2)n2 + (3/2)n - 1T2(n) = (3/2)n - 1Fast Computer vs. Smart Programmer

Mon 29 Sep 2014Lecture 47T1(n) = (3/2)n2 + (3/2)n - 1T2(n) = (3/2)n - 1Fast Computer vs Smart Programmer (rematch!)

A smart programmer with a better algorithm always beats a fast computer with a worst algorithm for sufficiently large inputs.Mon 29 Sep 2014Lecture 48At large enough input sizes only the rate of growth of an algorithms running time matters.Thats why we dropped the lower-order terms with the tilde notation:When T(n) = (3/2)n2 + (3/2)n -1 we write: T(n) ~ (3/2)n2However: to calculate (3/2)n2 we need to first calculate (3/2)n2 + (3/2)n -1It is not possible to calculate the coefficient 3/2 without the complete polynomials.

Mon 29 Sep 2014Lecture 49Simpler approachIt turns out that even the coefficient of the highest order term of polynomials is not all that important for large enough inputs.

This leads us to Asymptotic running time:T(n) = (3/2)n2 + (3/2)n - 1 = (n2)

We ignore everything except for the most significant growth functionEven with such a simplification, we can compare algorithms to discover the best onesSometimes constants matter in the real-world performance of algorithms, but this is rare.

Mon 29 Sep 2014Lecture 410Important Growth FunctionsFrom better to worse: Function fName1 constantlog n logarithmicn linearn.log nn2quadraticn3cubic2n exponential...Mon 29 Sep 2014Lecture 411Important Growth FunctionsFrom better to worse: Function fName1 constantlog n logarithmicn linearn.log nn2quadraticn3cubic2n exponential...Mon 29 Sep 2014Lecture 412The first 4 are practically fast(most commercial programs run in such -time) Anything less than exponentialis theoretically fast (P vs NP)Important Growth FunctionsFrom better to worse: Function fName Problem size solved in mins (today) 1 constantanylog n logarithmicanyn linearbillionsn.log nhundreds of millionsn2quadratictens of thousandsn3cubicthousands2n exponential100...Mon 29 Sep 2014Lecture 413Growth FunctionsFrom better to worse: Function fNameExample code of (f):1 constantswap A[i], A[j]log n logarithmicj=n; while(j>0){ ; j=j/2}n linearfor(j=1; j=0 and A[i]>A[i+1] {(n2) swap A[i], A[i+1](n2) i=i-1(n2)}}return A(1)

T(n) = (n) + (n) + (n2) + (n2) + (n2) + (1) = (n2)

More Asymptotic Notation O, When we are giving exact bounds we write: T(n) = (f(n))When we are giving upper bounds we write: (n) (f(n)) or alternatively T(n) = (f(n))

When we are giving lower bounds we write: (n) (f(n)) or alternatively T(n) = (f(n))Mon 29 Sep 2014Lecture 418Examples , O, 3n2 . log n + n2 + 4n - 2 = ?3n2 . log n + n2 + 4n - 2 = O(n2 . log n)3n2 . log n + n2 + 4n - 2 = O(n3)3n2 . log n + n2 + 4n - 2 = O(2n)3n2 . log n + n2 + 4n - 2 O(n2)

Mon 29 Sep 2014Lecture 419Examples , O, 3n2 . log n + n2 + 4n - 2 = (n2 . log n)3n2 . log n + n2 + 4n - 2 = (n2 . log n)3n2 . log n + n2 + 4n - 2 = (n2)3n2 . log n + n2 + 4n - 2 = (1)3n2 . log n + n2 + 4n - 2 (n3 . log n)

Mon 29 Sep 2014Lecture 420Examples (comparisons)(n logn) =?= (n)Mon 29 Sep 2014Lecture 421Examples (comparisons)(n logn) > (n)(n2 + 3n 1) =?= (n2)Mon 29 Sep 2014Lecture 422Examples (comparisons)(n logn) > (n)(n2 + 3n 1) = (n2)Mon 29 Sep 2014Lecture 423Examples (comparisons)(n log(n)) > (n)(n2 + 3n 1) = (n2)(1) =?= (10)(5n) =?= (n2)(n3 + log(n)) =?= (100n3 + log(n))Write all of the above in order, writing = or < between them

Mon 29 Sep 2014Lecture 424Principle bounds are the most precise asymptotic performance bounds we can giveO/ bounds may be impreciseMon 29 Sep 2014Lecture 425One more example: BinarySeachSpecification:Input: array a[0..n-1], integer keyInput property: a is sortedOutput: integer posOutput property: if key==a[i] then pos==i

Trivial?First binary search published in 1946First bug-free binary search published in 1962Bug in Javas Arrays.binarySearch() found in 2006Mon 29 Sep 2014Lecture 426BinarySearch pseudocodeMon 29 Sep 2014Lecture 427lo = 0, hi = a.length-1while (lo a[mid]) then lo = mid + 1 else return mid}return -1Note: here array indices start from 0 and go up to length-1.

BinarySearch Loop InvariantMon 29 Sep 2014Lecture 428lo = 0, hi = a.length-1while (lo a[mid]) then lo = mid + 1 else return mid}return -1Note: here array indices start from 0 and go up to length-1.Invariant: if key in a[0..n-1] then key in a[lo..hi]

BinarySearch Asymptotic Running TimeMon 29 Sep 2014Lecture 429Asymptotic costlo = 0, hi = a.length-1while (lo a[mid]) then lo = mid + 1 else return mid}return -1Note: array indices start from 0 and go up to length-1.

BinarySearch Asymptotic Running TimeMon 29 Sep 2014Lecture 430Asymptotic costlo = 0, hi = a.length-1(1)while (lo a[mid]) then lo = mid + 1(log n) else return mid(log n)}return -1(1)Note: array indices start from 0 and go up to length-1.T(n) = (log n)

When a loop throws away half the input array at each iteration: it will perform (log n) iterations!We will use the Asymptotic -notation from now on because its easier to calculateThe book uses the ~ notation (more accurate but similar to )We will mostly look at the worst case (sometimes the average)Sometimes we can sacrifice some memory space to improve running timeWe will discuss space performance and space/time tradeoff in next lectureDont forget the labs tomorrow, Tuesday and Thursdaysee websiteMon 29 Sep 2014Lecture 431