Design and Analysis of Algorithms - Chapter 11 Σχεδίαση Αλγορίθμων Introduction...

Design and Analysis of Algorithms - Chapter 1 1

Σχεδίαση ΑλγορίθμωνΣχεδίαση Αλγορίθμων

Introduction to the Design and Analysis of Algorithms by Anany Levitin www.aw-bc.com/info/levitin/www.aw-bc.com/info/levitin/


Σχεδίαση ΑλγορίθμωνΣχεδίαση Αλγορίθμων

http://http://delab.csd.auth.gr/courses.htmldelab.csd.auth.gr/courses.html

Ημερολόγιο, βιβλιογραφία, ανακοινώσειςΗμερολόγιο, βιβλιογραφία, ανακοινώσεις

Θα διανεμηθεί το βιβλίο «Αλγόριθμοι» Θα διανεμηθεί το βιβλίο «Αλγόριθμοι» του Παναγιώτη Μποζάνη, Εκδόσεις του Παναγιώτη Μποζάνη, Εκδόσεις Τζιόλα, 2005.Τζιόλα, 2005.


Wirth 1976: Wirth 1976: Algorithms + Data Structures = ProgramsAlgorithms + Data Structures = Programs

Theoretical importanceTheoretical importance• the core of computer sciencethe core of computer science

Practical importancePractical importance• A practitioner’s toolkit of known algorithmsA practitioner’s toolkit of known algorithms

• Framework for designing and analyzing Framework for designing and analyzing algorithms for new problemsalgorithms for new problems

Why study algorithms?Why study algorithms?


AlgorithmAlgorithm

An An algorithmalgorithm is a sequence of is a sequence of unambiguous instructions for unambiguous instructions for solving a problem, i.e., for solving a problem, i.e., for obtaining a required output for obtaining a required output for any legitimate input in a finite any legitimate input in a finite amount of time.amount of time.


AlgorithmAlgorithm

““Besides merely being a finite set of rules that Besides merely being a finite set of rules that gives a sequence of operations for solving a gives a sequence of operations for solving a specific type of problem, an algorithm has the specific type of problem, an algorithm has the five important features” [Knuth]five important features” [Knuth]

– finiteness (otherwise: computational method)finiteness (otherwise: computational method)» terminationtermination

– definitenessdefiniteness» precise definition of each stepprecise definition of each step

– input (zero or more)input (zero or more)– output (one or more)output (one or more)– effectivenesseffectiveness

» Its operations must all be sufficiently basic that they Its operations must all be sufficiently basic that they can in principle be done exactly and in a finite length can in principle be done exactly and in a finite length of time by someone using pencil and paperof time by someone using pencil and paper


Historical PerspectiveHistorical Perspective Euclid’s algorithm for finding the greatest common Euclid’s algorithm for finding the greatest common

divisordivisor http://en.wikipedia.org/wiki/Al-Khwarizmihttp://en.wikipedia.org/wiki/Al-Khwarizmi Abū ‘Abd Allāh Muḥammad ibn Mūsā al-Khwārizmī Abū ‘Abd Allāh Muḥammad ibn Mūsā al-Khwārizmī

(( الخوارزمي موسى بن محمد الله عبد الخوارزمي أبو موسى بن محمد الله عبد was a ) was a (أبوMuslim mathematician, astronomer, astrologer, Muslim mathematician, astronomer, astrologer, geographer and author. He was born in Persia around geographer and author. He was born in Persia around 780, and died around 850.780, and died around 850.

Because of his book on the systematic solution of linear Because of his book on the systematic solution of linear and quadratic equations, and quadratic equations, al-Kitāb al-mukhtaṣar fī hīsāb al-Kitāb al-mukhtaṣar fī hīsāb al-ğabr wa’l-muqābalaal-ğabr wa’l-muqābala, together with Diophantus, he is , together with Diophantus, he is considered to be the father of algebra. The word considered to be the father of algebra. The word algebra is derived from algebra is derived from al-ğabral-ğabr, one of the two , one of the two operations used to solve quadratic equations, as operations used to solve quadratic equations, as described in his book. described in his book. Algoritmi de numero IndorumAlgoritmi de numero Indorum, the , the Latin translation of his other major work, on the Indian Latin translation of his other major work, on the Indian numerals, introduced the positional number system and numerals, introduced the positional number system and the number zero to the Western world in the 12th the number zero to the Western world in the 12th century. The words algorism and algorithm stem from century. The words algorism and algorithm stem from AlgoritmiAlgoritmi, the Latinization of his name, the Latinization of his name


Notion of algorithmNotion of algorithm

“computer”

Algorithmic solution : a way to teach the computer

problem

algorithm

input output


Greatest Common Divisor - Greatest Common Divisor - Algorithm 1Algorithm 1

Step 1Step 1 Assign the value of min{ Assign the value of min{m,nm,n} to } to tt

Step 2Step 2 Divide Divide mm by by tt. If the remainder of this . If the remainder of this division is 0, go to Step 3; otherwise, to Step 4division is 0, go to Step 3; otherwise, to Step 4

Step 3Step 3 Divide Divide nn by by tt. If the remainder of this . If the remainder of this division is 0, return the value of division is 0, return the value of tt as the answer as the answer and stop; otherwise, proceed to Step 4and stop; otherwise, proceed to Step 4

Step 4Step 4 Decrease the value of Decrease the value of tt by 1. Go to Step 2 by 1. Go to Step 2

NoteNote: : mm and and nn are positive integers are positive integers



The algorithm The algorithm terminatesterminates, because , because tt is decreased is decreased by 1 each time we go through step 4, 1 divides by 1 each time we go through step 4, 1 divides any integer, and any integer, and tt eventually will reach 1 unless eventually will reach 1 unless the algorithm stops earlier.the algorithm stops earlier.

The algorithm is The algorithm is partially correctpartially correct, because when , because when it returns it returns tt as the answer, as the answer, tt is the minimum is the minimum value that divides both value that divides both mm and and nn

CorrectnessCorrectness



Step 1Step 1 Find the prime factors of Find the prime factors of mmStep 2Step 2 Find the prime factors of Find the prime factors of nnStep 3Step 3 Identify all the common factors in the two prime Identify all the common factors in the two prime

expansions found in Steps 1 and 2. If expansions found in Steps 1 and 2. If pp is a common is a common factor repeated factor repeated ii times in times in mm and and jj times in times in nn, assign , assign to to pp the multiplicity min{ the multiplicity min{i,ji,j}}

Step 4Step 4 Compute the product of all the common factors Compute the product of all the common factors with their multiplicities and return it as the GCD of with their multiplicities and return it as the GCD of mm and and nn

NoteNote: as written, the algorithm requires than : as written, the algorithm requires than mm and and nn be integers greater than 1, since 1 is not a primebe integers greater than 1, since 1 is not a prime



In fact “Algorithm 2” is not an algorithm In fact “Algorithm 2” is not an algorithm unless we can provide an unless we can provide an effectiveeffective way to find way to find prime factors of a numberprime factors of a number

The sieve of Eratosthenes is an algorithm that The sieve of Eratosthenes is an algorithm that provides such an effective procedureprovides such an effective procedure

Πως δουλεύει το κόσκινο του Ερατοσθένη ??Πως δουλεύει το κόσκινο του Ερατοσθένη ??



Euclid’s AlgorithmEuclid’s Algorithm

E0E0 [Ensure n [Ensure n≤≤ mm]] If If mm << nn, exchange , exchange mm with with nn

E1E1 [Find Remainder] [Find Remainder] Divide Divide mm by by nn and let and let rr be be the remainder (We will have 0 the remainder (We will have 0 ≤≤ rr < < nn))

E2E2 [Is it zero?] [Is it zero?] If If rr=0, the algorithm terminates; =0, the algorithm terminates; nn is the answer is the answer

E3E3 [Reduce] [Reduce] Set Set mm to to nn, , nn to to rr, and go back to E1, and go back to E1



Termination of Euclid’s AlgorithmTermination of Euclid’s Algorithm

The second number of the pair gets smaller The second number of the pair gets smaller with each iteration and cannot become with each iteration and cannot become negative: indeed, the new value of negative: indeed, the new value of nn is is rr = = mm mod mod nn, which is always smaller than , which is always smaller than nn. Eventually, . Eventually, rr becomes zero, and the becomes zero, and the algorithms stops.algorithms stops.



Correctness of Euclid’s AlgorithmCorrectness of Euclid’s Algorithm After step E1, we have After step E1, we have m=qn+rm=qn+r, for some integer , for some integer qq. .

If If r=0r=0, then , then mm is a multiple of is a multiple of nn, and clearly in such , and clearly in such a case a case nn is the GCD of is the GCD of mm and and nn. If . If rr≠≠00, note that , note that any number that divides both any number that divides both mm and and nn must divide must divide m–qn=rm–qn=r, and any number that divides both , and any number that divides both nn and and rr must divide must divide qn+r=mqn+r=m; so the set of divisors of {; so the set of divisors of {m,nm,n} } is the same as the set of divisors of {is the same as the set of divisors of {n,rn,r}. In }. In particular, the GCD of {particular, the GCD of {m,nm,n} is the same as the } is the same as the GCD of {GCD of {n,rn,r}. Therefore, E3 does not change the }. Therefore, E3 does not change the answer to the original problem. answer to the original problem.



Book Variation (p.4)Book Variation (p.4)

Step 1Step 1 If n=0, return the value of m as the answer If n=0, return the value of m as the answer and stop; otherwise, proceed to Step 2. and stop; otherwise, proceed to Step 2.

Step 2Step 2 Divide Divide mm by by nn and assign the value of the and assign the value of the remainder to r. remainder to r.

Step 3Step 3 Assign the value of n to m and the value of r Assign the value of n to m and the value of r to n. Go to Step 1.to n. Go to Step 1.

NoteNote: : mm and and nn are nonnegative, not-both-zero are nonnegative, not-both-zero integersintegers



Pseudocode for book versionPseudocode for book version

while n <> 0 dowhile n <> 0 do

r r m mod n m mod n

m m n n

n n r r

return mreturn m


Example of computational problem: Example of computational problem: sortingsorting

Statement of problem:Statement of problem:• Input:Input: A sequence of A sequence of nn numbers <a numbers <a11,a,a22,…,a,…,ann>>

• Output:Output: A reordering of the input sequence <a A reordering of the input sequence <a11,a,a´́22,,

…,a…,ann> so that a> so that aii≤≤aajj whenever whenever ii<<jj

Instance: The sequence <5, 3, 2, 8, 3>Instance: The sequence <5, 3, 2, 8, 3> Algorithms:Algorithms:

• Selection sortSelection sort• Insertion sortInsertion sort• Merge sortMerge sort• (many others)(many others)


Selection SortSelection Sort

Input: array Input: array a[1],...,a[n]a[1],...,a[n] Output: array Output: array aa sorted in non-decreasing sorted in non-decreasing

orderorder Algorithm:Algorithm:

for i=1 to n swap a[i] with smallest of a[i],…a[n]

(In place sorting, stable sorting)

• see also pseudocode, section 3.1


Some Well-known Computational Some Well-known Computational ProblemsProblems

SortingSorting SearchingSearching String searchingString searching Graph problems (shortest paths, minimum Graph problems (shortest paths, minimum

spanning tree, graph-coloring etc)spanning tree, graph-coloring etc) Combinatorial problems (tsp)Combinatorial problems (tsp) Geometric problems (closest-pair, convex hull)Geometric problems (closest-pair, convex hull) Numerical problems (equations, integrals)Numerical problems (equations, integrals) Other (primality testing, knapsack problem, chess, Other (primality testing, knapsack problem, chess,

towers of Hanoi)towers of Hanoi)


Basic Issues Related to Basic Issues Related to AlgorithmsAlgorithms

Understanding the problem (instances, ranges)Understanding the problem (instances, ranges) Ascertaining the capabilities of a computational device (RAM Ascertaining the capabilities of a computational device (RAM

and parallel machines, speed and amount of memory)and parallel machines, speed and amount of memory) Choosing between exact and approximate algorithmsChoosing between exact and approximate algorithms Deciding the correct data structureDeciding the correct data structure Algorithm design techniquesAlgorithm design techniques Methods of specifying an algorithm (free language, Methods of specifying an algorithm (free language,

pseudocode, flowchart)pseudocode, flowchart) Proving an algorithm’s correctnessProving an algorithm’s correctness Analyzing an algorithm (efficiency, theoretical and empirical Analyzing an algorithm (efficiency, theoretical and empirical

analysis, optimality)analysis, optimality) Coding an algorithmCoding an algorithm


Algorithm design strategiesAlgorithm design strategies

Brute forceBrute force

Divide and conquerDivide and conquer

Decrease and conquerDecrease and conquer

Transform and conquerTransform and conquer

Greedy approachGreedy approach

Dynamic programmingDynamic programming

Backtracking and Backtracking and Branch and boundBranch and bound

Space and time tradeoffsSpace and time tradeoffs


Analysis of AlgorithmsAnalysis of Algorithms

How good is the algorithm?How good is the algorithm?• CorrectnessCorrectness

• Time efficiencyTime efficiency

• Space efficiencySpace efficiency

• Simple (recursive, iterative)Simple (recursive, iterative)

• GeneralGeneral

Does there exist a better algorithm?Does there exist a better algorithm?• Lower boundsLower bounds

• OptimalityOptimality

Design and Analysis of Algorithms - Chapter 11 Σχεδίαση Αλγορίθμων Introduction...

Documents

Transcript of Design and Analysis of Algorithms - Chapter 11 Σχεδίαση Αλγορίθμων Introduction...