ICC Module Computation –Theory of Computation Feedback ICC Module Computation –Theory...

download ICC Module Computation –Theory of Computation Feedback ICC Module Computation –Theory of Computation

of 35

  • date post

    21-Mar-2021
  • Category

    Documents

  • view

    2
  • download

    0

Embed Size (px)

Transcript of ICC Module Computation –Theory of Computation Feedback ICC Module Computation –Theory...

  • ICC Module Computation – Theory of Computation

    1

    Feedback 75-100% : 41% 50- 75%: 44%

  • ICC Module Computation – Theory of Computation

    2

    Complexity Example Behavior Example Algorithm

    Θ(log n) Look at only log n elements of the input Binary search (Recherche dichotomique)

    Θ(n) Look at each input element once Linear search

    Θ(n・log n) Split input into half (which can be done at most log n times) At each split look at each element once

    Merge sort (tri fusion)

    Θ(n2) Look at each pair of elments (i,j) Insertion sort

    Θ(n3) Look at each triple of element (i,j,k) Floyd’s shortest path algorithm

    Θ(2n) Look at all subset of a list or all permutation of a list or all paths in a graph

    Create a list of binary numbers of length n, e.g., input: 3 output:{{0,0,0},{0,0,1},{0,1,0},{0,1,1},

    {1,0,0},{1,0,1},{1,1,0},{1,1,1}}

    Representative Algorithms

    In this table “look at element x” means to do a fix number instructions for element x, e.g., compare x with 3, add x to another variable,…

  • ICC Module Computation – Theory of Computation

    3

    Information, Computation, and Communication

    Theory of Computation

  • ICC Module Computation – Theory of Computation

    4

    Topics Study the two big questions of the theory of computation: § Which problems can we solve with an algorithm? § Which problems can we solve efficiently with an

    algorithm?

    Important notions: § Countability § Decidability (video) § Complexity classes: P and NP (video)

  • ICC Module Computation – Theory of Computation

    5

    § Previously, we have studied algorithms that work regardless of the input data.

    § Each algorithm solves the same question for different input data each time.

    § Running the algorithm on one input gives us, the answer to one instance of the problem

    § Running an algorithm is not the solution of the problem, the solution is the algorithm itself

    Algorithms & Problems

  • ICC Module Computation – Theory of Computation

    6

    § sort({4, 6, 2,10}) … one instance § max({4, 6, 2,10}) … one instance § add(3, 5) … one instance § add(x, y) … infinitely many instances

    (if x and y are not bounded) § add(x, y), where x, y from {1, 2, 3, 4, 5}

    …finitely many instances

    Problem Instances

  • ICC Module Computation – Theory of Computation

    7

    § If the set of problem instances is finite, it is always possible to resolve the problem by building a Look-Up Table (LUT): we store the answer to every instance. The algorithm consults this table and prints the solution found inside the table – no computation involved!

    § Only the problems involving infinite sets of instances require real work. They are the ones that interest us in today's class.

    Infinity of Instances

  • ICC Module Computation – Theory of Computation

    8

    § add(x,y) where x, y from {1, 2, 3, 4, 5} Example: Look-Up Table

    1 2 3 4 5

    1 2 3 4 5 6

    2 3 4 5 6 7

    3 4 5 6 7 8

    4 5 6 7 8 9

    5 6 7 8 9 10

    x y

    Another example: multiplication table until 10

  • ICC Module Computation – Theory of Computation

    9

    § add(x,y) where x, y from {1, 2, 3, 4, 5} Example: Look-Up Table

    x y

    1 2 3 4 5

    1 2 3 4 5 6

    2 3 4 5 6 7

    3 4 5 6 7 8

    4 5 6 7 8 9

    5 6 7 8 9 10

    x y

  • ICC Module Computation – Theory of Computation

    10

    § What is the complexity? Example: Look-Up Table

    x y x

    y

  • ICC Module Computation – Theory of Computation

    11

    § Notation: Countability § Observation:

    the natural numbers, N = {1, 2, 3,...}, define counting.

    § Definition: an infinite set S is countable if and only if there exists a bijection f : N → S (a 1-to-1 mapping)

    § Function f defines the enumeration of S.

    Comparing Infinite Sets

  • ICC Module Computation – Theory of Computation

    12

    § The integers, ℤ = {… ,−3,−2,−1, 0, 1, 2, 3… }, are a countable set.

    § We define a function 𝑓:ℕ → ℤ as follows: • if i is even, then 𝑓 𝑖 = !

    " • if i is odd, then 𝑓 𝑖 = − !#$

    "

    Example: Integers

    𝑖 ∈ ℕ 1 2 3 4 5 6 … f(𝑖) ∈ ℤ 0 1 -1 2 -2 3 …

  • ICC Module Computation – Theory of Computation

    13

    § Is the set of even numbers countable? § Yes, we define a bijection as follows:

    𝑓 𝑖 = 2 % 𝑖

    Countability: Example

    𝑖 ∈ ℕ 1 2 3 4 5 6 … Even numbers f(𝑖) 2 4 6 8 10 12 …

  • ICC Module Computation – Theory of Computation

    14

    § Is the set of rational numbers countable? ℚ =

    𝑝 𝑞 | 𝑝, 𝑞 ∈ ℕ 𝑎𝑛𝑑 𝑞 ≠ 0

    Countability: Another Example

    1 2 3 4 …

    1 1/1 1/2 1/3 1/4 ..

    2 2/1 2/2 2/3 2/4 …

    3 3/1 3/2 3/3 3/4 …

    4 4/1 4/2 4/3 4/4 …

    … … … … … …

    p q

  • ICC Module Computation – Theory of Computation

    15

    § Is the set of rational numbers countable? ℚ =

    𝑝 𝑞 | 𝑝, 𝑞 ∈ ℕ 𝑎𝑛𝑑 𝑞 ≠ 0

    Countability: Another Example

    1 2 3 4 …

    1 1/1 1/2 1/3 1/4 ..

    2 2/1 2/2 2/3 2/4 …

    3 3/1 3/2 3/3 3/4 …

    4 4/1 4/2 4/3 4/4 …

    … … … … … …

    p q

    You can use the same order to counter pairs of integers (i,j)

  • ICC Module Computation – Theory of Computation

    16

    § Is the set of rational numbers countable? ℚ =

    𝑝 𝑞 | 𝑝, 𝑞 ∈ ℕ 𝑎𝑛𝑑 𝑞 ≠ 0

    Countability: Another Example

    1 2 3 4 …

    1 1/1 1/2 1/3 1/4 ..

    2 2/1 2/2 2/3 2/4 …

    3 3/1 3/2 3/3 3/4 …

    4 4/1 4/2 4/3 4/4 …

    … … … … … …

    p q

    You can use the same order to counter pairs of integers (i,j)

  • ICC Module Computation – Theory of Computation

    17

    § Every algorithm has a finite description, which is a text written in a chosen alphabet. It is easy to enumerate all the possible texts.

    § Let’s start by enumerating the texts of one character (26):

    § Let’s continue with the texts of two characters (26・26 = 676)

    § Now, let’s move to the texts of 3 characters:

    § The set of algorithms is smaller than the set of all possible texts because some texts are not algorithms. Therefore, the set of algorithms is countable.

    The Set of Algorithms is Countable

    𝑖 ∈ ℕ 1 2 3 4 5 6 … Text a b c d e f …

    𝑖 ∈ ℕ 27 28 29 30 31 32 … Text aa ab ac ad ae af …

    𝑖 ∈ ℕ 703 704 705 706 707 708 … Text aaa aab aac aad aae aaf …

  • ICC Module Computation – Theory of Computation

    18

    § Let’s look at a subset of problems, namely all problems that take an integer as input and return true or false.

    § This is the set of all Boolean functions 𝑓 𝑖 ∶ ℕ → 0, 1

    The Set of Problems is not Countable

  • ICC Module Computation – Theory of Computation

    19

    Consider the set of Boolean functions of an integer variable B = 𝑓 | 𝑓 𝑖 ∶ ℕ → 0, 1

    Each function is represented by a list of binary values:

    If the set B were countable, we could enumerate the functions: f1, f2, f3, f4, f5, f6, f7, . . . Let’s put them in a table format:

    n 1 2 3 4 5 6 7 … f1 0 1 1 0 0 0 1 . . . f2 0 0 0 0 1 0 1 . . . f3 1 1 0 0 1 1 1 . . . f4 0 1 0 1 0 1 1 . . . f5 1 0 1 0 0 0 0 . . . f6 1 1 0 1 1 0 1 . . . f7 0 0 0 1 1 1 1 . . .

    The Set of Boolean Functions is Not Countable

    n 1 2 3 4 5 6 7 . . . f (n) 0 0 0 1 0 0 1 . . .

  • ICC Module Computation – Theory of Computation

    20

    Consider the set of Boolean functions of an integer variable B = 𝑓 | 𝑓 𝑖 ∶ ℕ → 0, 1

    Each function is represented by a list of binary values:

    If the set B were countable, we could enumerate the functions. f1, f2, f3, f4, f5, f6, f7, . . .

    The Set of Boolean Functions is Not Countable

    Observe the diagonal: a list of binary values

    n 1 2 3 4 5 6 7 . . . f (n) 0 0 0 1 0 0 1 . . .

    n 1 2 3 4 5 6 7 … f1 0 1 1 0 0 0 1 . . . f2 0 0 0 0 1 0 1 . . . f3 1 1 0 0 1 1 1 . . . f4 0 1 0 1 0 1 1 . . . f5 1 0 1 0 0 0 0 . . . f6 1 1 0 1 1 0 1 . . . f7 0 0 0 1 1 1 1 . . .

  • ICC Module Computation – Theory of Computation

    21

    § If B is countable, we can enumerate the functions: f1, f2, f3, f4, f5, f6, f7, . . .

    § The diagonal is a list of binary values 0 0 0 1 0 0 1 …

    § It can be seen as another Boolean function

    6𝑓 𝑖 ∶ ℕ → 0, 1 that can be written as

    6𝑓 𝑖 = 𝑓!(𝑖)

    The Set of Boolean Functions is Not Countable

    n 1 2 3 4 5 6 7 … f1 0 1 1 0 0 0 1 . . . f2 0 0 0 0 1 0 1 . . . f3 1 1 0 0 1 1 1 . . . f4 0 1 0 1 0 1 1 . . . f5 1 0 1 0 0 0 0 . . . f6 1 1 0 1 1 0 1 . . . f7 0 0 0 1 1 1 1 . . .

  • ICC Module Computation – Theory of Computation

    22

    § Let’s define a new Boolean function 𝑓∗ 𝑖 = 1 − 6