CSCE 551: Theory of ComputationTheory of Computation

CSCE 551:Theory of ComputationTheory of Computation

Chin-Tser [email protected]

University of South Carolina

Church-Turing ThesisChurch Turing ThesisThe definition of the algorithm came in the 1936 papers of Al Ch h d Al T i Ch h’ d fi iti dAlonzo Church and Alan Turing. Church’s definition used λ-calculus and Turing’s definition used “machines”.Their definitions turned out to be equivalent. Since thenTheir definitions turned out to be equivalent. Since then the connection between the informal notion of an algorithm and the precise definition is called the Church-Turing thesisTuring thesis.Hilbert's Tenth Problem basically asks to construct a decider for the language:g g

D = {p | p is a polynomial with an integer root}That is, for any given polynomial p, we need to decide whether p ∈ D (in a finite number of steps).

03/03/2010 2

Example: Finding an Integer RootExample: Finding an Integer RootA Turing machine for finding a integer root of a given polynomial (of a single variable x for simplicity) can bepolynomial (of a single variable x for simplicity) can be informally described as:

Evaluate a given polynomial p(x) at x set successively to the l 0 1 1 2 2 3 3values 0, 1, -1, 2, -2, 3, -3, …

If at any such value of x the value p(x) is zero, then accept.

Q: Is this Turing machine recognizer or decider?Q g gFor a univariate polynomial, it is possible to compute an upper bound for an absolute value of its integer root. If no root found below this bound the polynomial has no integer roots and itsbelow this bound, the polynomial has no integer roots, and its TM can reject (making it a decider solving the problem).However, as shown by Matijasevic, for multivariate polynomials no such bound can be computed making it impossible to haveno such bound can be computed, making it impossible to have a decider for this problem in general case.

03/03/2010 3

Description of TM AlgorithmsDescription of TM AlgorithmsFrom now on, we focus on algorithms, and use TM as a

i d l f th d fi iti f l ithprecise model for the definition of an algorithm.Algorithms can be described in three ways:

Formal : Explicit description of a Turing machine (its statesFormal : Explicit description of a Turing machine (its states, transition function etc.);Implementation : Description of TM using English prose, i.e., how TM moves head read/writes symbols etc (without explicitlyhow TM moves head, read/writes symbols etc. (without explicitly specifying its states and transition function);High-level : Description of an algorithm using English prose, ignoring implementation details.ignoring implementation details.

We start with the first two levels and later will switch to the third one, when you feel confident on how to go from

l l f l ith d i ti t thone level of algorithm description to another.

03/03/2010 4

Example of High-Level DescriptionExample of High Level Description

Let A be the language consisting of all strings ti t d di t d h W itrepresenting connected undirected graphs. We write

A = {⟨G⟩ | G is a connected undirected graph}A high-level description of a TM M that decides A is asA high-level description of a TM M that decides A is as follows:M = “On input ⟨G⟩, the encoding of a graph G:1. Select the first node of G and mark it.2. Repeat the following stage until no new nodes are marked:

For each node in G mark it if it is attached by an edge to aFor each node in G, mark it if it is attached by an edge to a node that is already marked.

3. Scan all nodes of G to determine whether they all are marked. If they are accept; otherwise reject ”they are, accept; otherwise, reject.

03/03/2010 5

Chapter 4: DecidabilityChapter 4: Decidability

Turing Machine: LanguagesTuring Machine: LanguagesRecall that a collection of strings that a TM M accepts is called the language of M or language recognized by M denoted L(M)the language of M or language recognized by M, denoted L(M).DefinitionA language is Turing-recognizable (or recursively enumerable) g g g g ( y )if it is recognized by some Turing machine.A TM M on an input that is not in L(M) can either reject or loop. We distinguish TMs that never loop and call them deciders AWe distinguish TMs that never loop and call them deciders. A decider M decides the language that it recognizes.DefinitionA language is decidable (or recursive) if it is decided by some Turing machine.It is easy to see that the complement of a decidable language y p g gis also decidable.

03/03/2010 7

Decidable LanguagesDecidable LanguagesWe will study some examples of languages concerning automata and grammars and show that they are decidable by algorithms

S bl f thi ki d l t d t i t tSome problems of this kind are related to important applications, such as compilersSome other problems concerning automata andSome other problems concerning automata and grammars are not decidable; learning decidable languages first will help us understand undecidable languageslanguages

03/03/2010 8

Decidable Languages concerning RLDecidable Languages concerning RL

Acceptance problems:ADFA = {⟨B, w⟩ | B is a DFA that accepts string w}ANFA = {⟨B, w⟩ | B is a NFA that accepts string w}A {⟨R ⟩ | R i th t t t i }AREX = {⟨R, w⟩ | R is a regexp that generates string w}

Emptiness testing:Emptiness testing:EDFA = {⟨A⟩ | A is a DFA and L(A) = ∅}

Equality testing:EQDFA = {⟨A, B⟩ | A, B are DFAs and L(A) = L(B)}

03/03/2010 9

Deciding Languages concerning RLDeciding Languages concerning RL

ADFA = {⟨B, w⟩ | B is a DFA that accepts string w}We need to present a TM that decides ADFA. Idea is to “execute” given B on the given input w within our TM.Q: How to implement such a TM?Q: How to implement such a TM?A few implementation details:

Check that input represent a valid DFA B (a list of five components Q, Σ, δ, q0, and F) and a valid string w over the alphabet of B (i.e., Σ).Make that TM store and maintain the configuration of Bsomewhere on its tape.Transitions from state to state in B can be done in a finite number of steps in TM.Check that “execution” of B on w will terminate at some point.

03/03/2010 10


ANFA = {⟨B, w⟩ | B is an NFA that accepts string w}AREX = {⟨R, w⟩ | R is a regexp that generates string w}

Idea is to construct a TM that first converts a given NFA into an equivalent DFA (or first converts a given regexpinto an equivalent DFA (or first converts a given regexp into an equivalent NFA), and then executes this DFA as before.

03/03/2010 11


EDFA = {⟨A⟩ | A is a DFA and L(A) = ∅}Idea is to construct a TM that view given DFA A as a directed graph with its vertices representing states and its edges representing transitions, and explores all pathsedges representing transitions, and explores all paths from the start state in this graph.

First, mark the start state of A.Th t th f ll i til t t t k d kThen repeat the following until no new states get marked: mark any state that has a transition coming into it from any state that is already marked.If t t t f A i k d it t th i it j tIf no accept state of A is marked, it accepts; otherwise it rejects.

03/03/2010 12


EQDFA = {⟨A, B⟩ | A, B are DFAs and L(A) = L(B)}First notice that L(A) = L(B) is equivalent to emptiness of both set differences L(A) \ L(B) = L(A)∩L(B) and L(B) \L(A) = L(B) ∩ L(A), and thus their union:L(A) L(B) ∩ L(A), and thus their union:

(L(A)∩L(B)) ∪ (L(B) ∩ L(A))called the symmetric difference of L(A) and L(B).The symmetric difference is a result of regular operations over L(A) and L(B), thus it represents a regular language recognized of some DFA Crecognized of some DFA C.Therefore, we can design a TM that first constructs a DFA C from the given DFAs A and B, and then tests emptiness g , pfor L(C).

03/03/2010 13

Decidable Languages concerning CFLDecidable Languages concerning CFL

ACFG = {⟨G, w⟩ | G is a CFG that generates string w}

Emptiness testing:E {⟨G⟩ | G i CFG d L(G) ∅}ECFG = {⟨G⟩ | G is a CFG and L(G) = ∅}

But equality testing for CFG is NOT decidable:But equality testing for CFG is NOT decidable:EQCFG = {⟨G, H⟩ | G, H are CFGs and L(G) = L(H)}

03/03/2010 14


ACFG = {⟨G, w⟩ | G is a CFG that generates string w}Q: Can we try to go through all derivation of G to determine whether any is a derivation of w?To show that A is decidable we need to ensure thatTo show that ACFG is decidable, we need to ensure that the TM only tries finitely many derivations.It has been shown that if G is in Chomsky normal form, any derivation of w has 2n -1 steps, where n is the length of w.

03/03/2010 15


ECFG = {⟨G⟩ | G is a CFG and L(G) = ∅}Similar to the proof that ACFG is decidable, we cannot try all possible strings.Instead we test whether the start variable can generate aInstead, we test whether the start variable can generate a string of terminals.

First, mark all terminal symbols in G.Then repeat the following until no new variables get marked: mark any variable A where G has a rule A → U1U2…Uk and each symbol U1, …, Uk has already been marked.If no accept state of A is marked, it accepts; otherwise it rejects.

03/03/2010 16

CFL is DecidableCFL is DecidableTheoremEvery context-free language is decidable.ProofLet A be a CFL. Let G be a CFG for A. We can make use of the TM we just constructed for G to decide whether any input w can be generated by G.

03/03/2010 17

Relationship among LanguagesRelationship among Languages

RL ⊂ CFL ⊂ DECIDABLE ⊂ RECOGNIZABLE

03/03/2010 18

Decidable vs. Undecidable LanguagesDecidable vs. Undecidable Languages

EQDFA = {⟨A, B⟩ | A, B are DFAs and L(A) = L(B)}EQCFG = {⟨G, H⟩ | G, H are CFGs and L(G) = L(H)}

We proved that EQDFA is decidable. In contrast to EQDFA, it is not clear how to prove that EQ is decidableis not clear how to prove that EQCFG is decidable.In fact, it is NOT decidable. But it is still unclear how to prove that.Our goal is to prove undecidability of another somewhat “simpler” language

A {⟨M ⟩ | M i TM d M t }ATM = {⟨M, w⟩ | M is a TM and M accepts w}

03/03/2010 19

Decidable vs. Undecidable LanguagesDecidable vs. Undecidable Languages

ACFG = {⟨G, w⟩ | G is a CFG that generates string w}ATM = {⟨M, w⟩ | M is a TM and M accepts w}

We proved that ACFG is decidable and that was used to prove that every context-free language is decidable (byprove that every context-free language is decidable (by simulating a TM for ACFG ).Q: What would happen if we prove that ATM is decidable?TM

If ATM were decidable, the language of any TM would be decidable. In particular, recognizable languages would coincide with decidable languages (which is verycoincide with decidable languages (which is very unrealistic!). So, it is natural to expect that ATM is NOT decidable.

03/03/2010 20

Halting ProblemHalting ProblemIt is easy to establish that ATM is recognizable: for a given pair ⟨M, w⟩we need simply to simulate M on the input w and accept or rejectwe need simply to simulate M on the input w, and accept or reject depending on whether M accepts or rejects. Note that M may loop in which case our simulation will loop as well, making it just a recognizer not a decider.not a decider.If we were able to determine whether M would ever stop on a given input w, a TM for ATM could reject given ⟨M, w⟩ as soon as it determined that M would loop on w. In this case it would be a deciderdetermined that M would loop on w. In this case it would be a decider for ATM (which accepts if M accepts; and rejects if M rejects or loops).Therefore, the crucial point is determination of whether M would ever stop on a given input w, giving the name halting problem to ATM. Instop on a given input w, giving the name halting problem to ATM. In these new terms, our goal is to prove that the halting problem is undecidable.

03/03/2010 21

Sets and Their CardinalitiesSets and Their CardinalitiesIt is easy to tell whether two given finite sets (such as {5, 7, 23} and {a, b, c}) have the same size -- just count up their elements and compare the countsthe counts.But is there a way to compare sizes of two infinite sets? Obviously counting elements won't help assets? Obviously, counting elements won t help as it would never end. But can we determine that two sets have equal sizes without counting theirtwo sets have equal sizes without counting their elements?

03/03/2010 22

Sets and Their CardinalitiesSets and Their CardinalitiesCantor proposed a technique called diagonalization to

l thi bl Th id i t t isolve this problem. The idea is to set up a mapping between the elements of two sets such that each element of one set is mapped to an element of the other, and vice pp ,versa. This idea works equally well for finite and infinite sets. For example we can map elements of the two sets aboveFor example, we can map elements of the two sets above as follows: 5 ↔ a, 7 ↔ b, and 23 ↔ c; implying that these sets are the same size.

03/03/2010 23

One-to-one, Onto, CorrespondenceOne to one, Onto, Correspondence

Let A, B be two sets and f : A → B be a function from A to B.Definitionf is called one-to-one (injective) if it maps different elements of A to different elements of B.Definitionf is called onto (surjective) if for every elements b ∈ B, there exists a A such that f (a) bexists a ∈ A such that f (a) = b.Definitionf is called correspondence (bijective) if it is one-to-one and p ( j )onto.Two set are of the same size if there exists a correspondence between thembetween them.

03/03/2010 24

An ExampleAn ExampleWhich of the following two sets, E = {2, 4, 6, … } of even

iti i t d N {1 2 3 } f t lpositive integers and N = {1, 2, 3, … } of natural numbers, have a larger size?It turns out they are of the same size! Why?It turns out they are of the same size! Why?Define f : N → E such that f (n) = 2n for every n ∈ N. It is one-to-one since for n ≠ m we have 2n ≠ 2m. It is onto i f k E k /2 i t l b d f (k/2) ksince for k ∈ E, k /2 is a natural number and f (k/2) = k.

Hence, f is a correspondence between E and N, implying that they are of the same size.yDefinitionA set is called countable if either it is finite or has the

i Nsame size as N.

03/03/2010 25

AnnouncementAnnouncement

Homework 2 is due TODAY in class; please ; psubmit it before you leaveReading assignment: Ch. 4.2Reading assignment: Ch. 4.2Happy spring break!

03/03/2010 26

CSCE 551: Theory of ComputationTheory of Computation

Documents

Transcript of CSCE 551: Theory of ComputationTheory of Computation