Regular languages: The Myhill-Nerode Theorem Context ...

82
Computational Models - Lecture 4 Regular languages: The Myhill-Nerode Theorem Context-free Grammars Chomsky Normal Form Pumping Lemma for context free languages Non context-free languages: Examples Push Down Automata (PDA) Hopcroft and Ullman, 3.4 Sipser, 2.1, 2.2 & 2.3 Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 1

Transcript of Regular languages: The Myhill-Nerode Theorem Context ...

Page 1: Regular languages: The Myhill-Nerode Theorem Context ...

Computational Models - Lecture 4

Regular languages: The Myhill-Nerode Theorem

Context-free Grammars

Chomsky Normal Form

Pumping Lemma for context free languages

Non context-free languages: Examples

Push Down Automata (PDA)

Hopcroft and Ullman, 3.4

Sipser, 2.1, 2.2 & 2.3Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 1

Page 2: Regular languages: The Myhill-Nerode Theorem Context ...

The Equivalence Relation ∼L

Let L ⊆ Σ∗ be a language.

Define an equivalence relation ∼L on pairs of strings:

Let x, y ∈ Σ∗. We say that x∼L y if for every string z ∈ Σ∗,xz ∈ L if and only if yz ∈ L.

It is easy to see that ∼L is indeed an equivalence relation(reflexive, symmetric, transitive) on Σ∗.

In addition, if x∼L y then for every string z ∈ Σ∗, xz∼L yz aswell (this is called right invariance).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 2

Page 3: Regular languages: The Myhill-Nerode Theorem Context ...

The Equivalence Relation ∼L

Like every equivalence relation, ∼L partitions Σ∗ to (disjoint)equivalence classes. For every string x, let [x] ⊆ Σ∗ denoteits equivalence class w.r.t. ∼L (if x ∼L y then [x] = [y] –equality of sets).

Question is, how many equivalence classes does ∼L

induce?

In particular, is the number of equivalence classes of ∼L

finite or infinite?

Well, it could be either finite or infinite. This depends on thelanguage L.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 3

Page 4: Regular languages: The Myhill-Nerode Theorem Context ...

Example: L = (ab ∪ ba)∗

∼L has the following four equivalence classes:

[ǫ] = L

[a] = La

[b] = Lb

[aa] = L(aa ∪ bb)Σ∗

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 4

Page 5: Regular languages: The Myhill-Nerode Theorem Context ...

Classes of ∼L: More Examples

Let L1 ⊂ {0, 1}∗ contain all strings where the number of1s is divisible by 4. Then ∼L1

has finitely manyequivalence classes.

Let L2 = L(0∗10∗). Then ∼L2has finitely many

equivalence classes.

Let L3 ⊂ {0, 1}∗ contain all strings of the form 0n1n.Then ∼L3

has infinitely many equivalence classes.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 5

Page 6: Regular languages: The Myhill-Nerode Theorem Context ...

Myhill-Nerode Theorem

Theorem: Let L ⊆ Σ∗ be a language. Then

L is regular ⇐⇒ ∼L has finitely many equivalenceclasses.

Three specific consequences:

L1 ⊂ {0, 1}∗ contains all strings where the number of 1sis divisible by 4. Then L1 is regular.

L2 = L(0∗10∗) is regular.

L3 ⊂ {0, 1}∗ contains all strings of the form 0n1n. ThenL2 is not regular.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 6

Page 7: Regular languages: The Myhill-Nerode Theorem Context ...

Myhill-Nerode Theorem: Proof

=⇒Suppose L is regular. Let M = (Q,Σ, δ, q0, F ) be a DFAaccepting it. For every x ∈ Σ∗, let δ(q0, x) ∈ Q be the statewhere the computation of M on input x ends.

The relation ∼M on pairs of strings is defined as follows:x ∼M y if δ(q0, x) = δ(q0, y).Clearly, ∼M is an equivalence relation.

Furthermore, if x ∼M y, then for every z ∈ Σ∗, alsoxz ∼M yz. Therefore, xz ∈ L if and only if yz ∈ L.

This means that x ∼M y =⇒ x ∼L y.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 7

Page 8: Regular languages: The Myhill-Nerode Theorem Context ...

Myhill-Nerode Theorem: Proof (cont.)

=⇒

The equivalence relation ∼M has finitely manyequivalence classes (at most the number of states inM).

We saw that x ∼M y =⇒ x ∼L y.

This means that the equivalence classes of ∼M refinethose of ∼L.

Thus, the number of equivalence classes of ∼M isgreater or equal than the number of equivalenceclasses of ∼L.

Therefore, ∼L has finitely many equivalence classes, asdesired. ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 8

Page 9: Regular languages: The Myhill-Nerode Theorem Context ...

Myhill-Nerode Theorem: Proof (cont.)

⇐=

Suppose ∼L has finitely many equivalence classes.We’ll construct a DFA M that accepts L.

Let x1, . . . , xn ∈ Σ∗ be representatives for the finitelymany equivalence classes of ∼L.

The states of M are the equivalence classes[x1], . . . , [xn].

The transition function δ is defined as follows: For alla ∈ Σ, δ([xi], a) = [xia] (the equivalence class of xia).

Why is δ well-defined?(Hint: right invariance: if x ∼L y, then xz ∼L yz).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 9

Page 10: Regular languages: The Myhill-Nerode Theorem Context ...

Myhill-Nerode Theorem: Proof (conc.)

⇐=

The initial state is [ε].

The accept states are F = {[xi] |xi ∈ L}.

Easy: On input x ∈ Σ∗, M ends at state [x] (why?).

Therefore M accepts x iff x ∈ L (why?).

So L is accepted by DFA, hence L is regular. ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 10

Page 11: Regular languages: The Myhill-Nerode Theorem Context ...

Context Switch

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 11

Page 12: Regular languages: The Myhill-Nerode Theorem Context ...

Algorithmic Questions for NFAs

Q.: Given an NFA, N , and a string s, is s ∈ L(N)?

Answer: Construct the DFA equivalent to N and run it on w.

Q.: Is L(N) = ∅?Answer: This is a reachability question in graphs: Is there apath in the states’ graph of N from the start state to someaccepting state. There are simple, efficient algorithms forthis task.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 12

Page 13: Regular languages: The Myhill-Nerode Theorem Context ...

More Algorithmic Questions for NFAs

Q.: Is L(N) = Σ∗?

Answer: Check if L(N) = ∅.

Q.: Given N1 and N2, is L(N1) ⊆ L(N2)?

Answer: Check if L(N2) ∩ L(N1) = ∅.

Q.: Given N1 and N2, is L(N1) = L(N2)?

Answer: Check if L(N1) ⊆ L(N2) and L(N2) ⊆ L(N1).

In the future, we will see that for stronger models ofcomputations, many of these problems cannot be solved byany algorithm.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 13

Page 14: Regular languages: The Myhill-Nerode Theorem Context ...

Another, More Radical Context Switch

So far we sawfinite automata,regular languages,regular expressions,Myhill-Nerode theorem and pumping lemma forregular languages.

We now introduce stronger machines andlanguages with more expressive power:

pushdown automata,context-free languages,context-free grammars,pumping lemma for context-free languages.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 14

Page 15: Regular languages: The Myhill-Nerode Theorem Context ...

Context-Free Grammars

An example of a context free grammar, G1:

A → 0A1

A → B

B → #

Terminology:

Each line is a substitution rule or production.

Each rule has the form: symbol → string.The left-hand symbol is a variable(usually upper-case).

A string consists of variables and terminals.

One variable is the start variable.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 15

Page 16: Regular languages: The Myhill-Nerode Theorem Context ...

Rules for Generating Strings

Write down the start variable (lhs of top rule).

Pick a variable written down in current string and aderivation that starts with that variable.

Replace that variable with right-hand side of thatderivation.

Repeat until no variables remain.

Return final string (concatenation of terminals).

Process is inherently non deterministic.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 16

Page 17: Regular languages: The Myhill-Nerode Theorem Context ...

Example

Grammar G1:

A → 0A1

A → B

B → #

Derivation with G1:

A ⇒ 0A1

⇒ 00A11

⇒ 000A111

⇒ 000B111

⇒ 000#111

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 17

Page 18: Regular languages: The Myhill-Nerode Theorem Context ...

A Parse Tree

A

A

A

B

#0 10 0 1 1

Question: What strings can be generated in this way fromthe grammar G1?Answer: Exactly those of the form 0n#1n (n ≥ 0).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 18

Page 19: Regular languages: The Myhill-Nerode Theorem Context ...

Context-Free Languages

The language generated in this way is called the languageof the grammar.

For example, L(G1) is {0n#1n|n ≥ 0}.

Any language generated by a context-free grammar iscalled a context-free language.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 19

Page 20: Regular languages: The Myhill-Nerode Theorem Context ...

A Useful Abbreviation

Rules with same variable on left hand side

A → 0A1

A → B

are written as:

A → 0A1 | B

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 20

Page 21: Regular languages: The Myhill-Nerode Theorem Context ...

English-like Sentences

A grammar G2 to describe a few English sentences:

< SENTENCE > → < NOUN-PHRASE >< VERB >

< NOUN-PHRASE > → < ARTICLE >< NOUN >

< NOUN > → boy | girl | flower

< ARTICLE > → a | the

< VERB > → touches | likes | sees

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 21

Page 22: Regular languages: The Myhill-Nerode Theorem Context ...

Deriving English-like Sentences

A specific derivation in G2:

< SENTENCE > ⇒ < NOUN-PHRASE >< VERB >

⇒ < ARTICLE >< NOUN >< VERB >

⇒ a < NOUN >< VERB >

⇒ a boy < VERB >

⇒ a boy sees

More strings generated by G2:

a flower seesthe girl touches

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 22

Page 23: Regular languages: The Myhill-Nerode Theorem Context ...

Derivation and Parse Tree

< SENTENCE > ⇒ < NOUN-PHRASE >< VERB >

⇒ < ARTICLE >< NOUN >< VERB >

⇒ a < NOUN >< VERB >

⇒ a boy < VERB >

⇒ a boy sees

SENTENCE

NOUN-PHRASE VERB

ARTICLE NOUN

a boy seesBased on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 23

Page 24: Regular languages: The Myhill-Nerode Theorem Context ...

Formal Definitions

A context-free grammar is a 4-tuple (V,Σ, R, S) where

V is a finite set of variables,

Σ is a finite set of terminals,

R is a finite set of rules: each rule is a variable and afinite string of variables and terminals.

S is the start symbol.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 24

Page 25: Regular languages: The Myhill-Nerode Theorem Context ...

Formal Definitions

If u and v are strings of variables and terminals,

and A → w is a rule of the grammar, then

we say uAv yields uwv, written uAv ⇒ uwv.

We write u∗⇒ v if u = v or

u ⇒ u1 ⇒ . . . ⇒ uk ⇒ v.

for some sequence u1, u2, . . . , uk.

Definition: The language of the grammar is{

w ∈ Σ∗ | S∗⇒ w

}

.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 25

Page 26: Regular languages: The Myhill-Nerode Theorem Context ...

Example

Consider G3 = (V, {(, )} , R, S).

R (Rules): S → (S) | SS | ε .

Some words in the language: (()), (()()).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 26

Page 27: Regular languages: The Myhill-Nerode Theorem Context ...

Arithmetic Example

Consider G4 = (V,Σ, R,E) where

V = {E, T, F}Σ = {a,+,×, (, )}

Rules:E → E + T | T

T → T × F | F

F → (E) | a

Strings generated by the grammar:a + a × a and (a + a) × a.What is the language of this grammar?Hint: arithmetic expressions.E = expression, T = term, F = factor.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 27

Page 28: Regular languages: The Myhill-Nerode Theorem Context ...

Parse Tree for a + a × a

E → E + T | T

T → T × F | F

F → (E) | a

aXa+a

FFF

T T

TE

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 28

Page 29: Regular languages: The Myhill-Nerode Theorem Context ...

Parse Tree for (a + a) × a

E → E + T | T

T → T × F | F

F → (E) | a

( a + aX)a

F F F

T T

E

E

F

T

T

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 29

Page 30: Regular languages: The Myhill-Nerode Theorem Context ...

Designing Context-Free Grammars - 1

If CFG is the union of several CFGs, rename variables(not terminals) so they are disjoint, and add new ruleS → S1 | S2 | . . . | Si.

For example, to get a grammar for the languageL = {0n1n| n ≥ 0} ∪ {1n0n| n ≥ 0} we construct thegrammars:

S1 → 0 S1 1 | ǫ

S2 → 1 S2 0 | ǫ

and then add the rule:

S → S1 | S2

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 30

Page 31: Regular languages: The Myhill-Nerode Theorem Context ...

Designing Context-Free Grammars - 2

To construct CFG for a regular language, “follow” a DFA forthe language. For initial state q0, make R0 the start variable.For state transition δ(qi, a) = qj add rule Ri → aRj togrammar. For each final state qf , add rule Rf → ε togrammar.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 31

Page 32: Regular languages: The Myhill-Nerode Theorem Context ...

Designing Context-Free Grammars - 3

For languages with linked substrings, a rule of formR → uRv may be helpful, forcing desired relationbetween substrings.

For L = {0n1n| n ≥ 0}:

S → 0 S 1 | ǫ

For L = {1n##0n| n ≥ 0}:

S → 1 S 0 | ##

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 32

Page 33: Regular languages: The Myhill-Nerode Theorem Context ...

Closure Properties

Regular languages are closed underunionconcatenationstar

Context-Free Languages are closed underunion : S → S1 | S2

concatenation S → S1S2

star S → ε | SS

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 33

Page 34: Regular languages: The Myhill-Nerode Theorem Context ...

More Closure Properties

Regular languages are also closed undercomplement (reverse accept/non-accept states ofDFA)

intersection(

L1 ∩ L2 = L1 ∪ L2

)

.

What about complement and intersection ofcontext-free languages?

Not clear . . .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 34

Page 35: Regular languages: The Myhill-Nerode Theorem Context ...

Ambiguity

Grammar: E → E+E | E×E | (E) | a

aXa+a

EEE

E

E

aXa+a

EEE

E

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 35

Page 36: Regular languages: The Myhill-Nerode Theorem Context ...

Ambiguity

We say that a string w is derived ambiguously fromgrammar G if w has two or more parse trees that generate itfrom G.

Ambiguity is usually not only a syntactic notion but also asemantic one, implying multiple meanings for the samestring.

It is sometime possible to eliminate ambiguity by finding adifferent context free grammar generating the samelanguage. This is true for the grammar above, which can bereplaced by unambiguous grammar from slide 50.

Some languages (e.g. {1i2j3k | i = j or j = k} areinherentrly ambigous.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 36

Page 37: Regular languages: The Myhill-Nerode Theorem Context ...

Chomsky Normal Form

A simplified, canonical form of context free grammars.Every rule has the form

A → BC

A → a

S → ε

where S is the start symbol, A, B and C are any variable,except B and C not the start symbol, and A can be the startsymbol.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 37

Page 38: Regular languages: The Myhill-Nerode Theorem Context ...

Theorem

Theorem: Any context-free language is generated by acontext-free grammar in Chomsky normal form.Basic idea:

Add new start symbol S0.

Eliminate all ε rules of the form A → ε.

Eliminate all “unit” rules of the form A → B.

Patch up rules so that grammar generates the samelanguage.

Convert remaining long rules to proper form.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 38

Page 39: Regular languages: The Myhill-Nerode Theorem Context ...

Proof Idea

Add new start symbol S0 and rule S0 → S.Guarantees that new start symbol does not appear onright-hand-side of a rule.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 39

Page 40: Regular languages: The Myhill-Nerode Theorem Context ...

Proof

Eliminating ε rules.

Repeat:

remove some A → ε.

for each R → uAv, add rule R → uv.

and so on: for R → uAvAw add R → uvAw, R → uAvw,and R → uvw.

for R → A add R → ε, except if R → ε has already beenremoved.

until all ε-rules not involving the original start variable havebeen removed.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 40

Page 41: Regular languages: The Myhill-Nerode Theorem Context ...

Proof

Eliminate unit rules.

Repeat:

remove some A → B.

for each B → u, add rule A → u, unless this ispreviously removed unit rule. (u is a string of variablesand terminals.)

until all unit rules have been removed.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 41

Page 42: Regular languages: The Myhill-Nerode Theorem Context ...

Proof

Finally, convert long rules.To replace each A → u1u2 . . . uk (for k ≥ 3), introduce newnon-terminals

N1, N2, . . . , Nk−1

and rules

A → u1N1

N1 → u2N2

...Nk−3 → uk−2Nk−2

Nk−2 → uk−1uk ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 42

Page 43: Regular languages: The Myhill-Nerode Theorem Context ...

Conversion Example

Initial Grammar:

S → ASA | aB

A → B | S

B → b | ε

(1) Add new start state:

S0 → S

S → ASA | aB

A → B | S

B → b | ε

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 43

Page 44: Regular languages: The Myhill-Nerode Theorem Context ...

Conversion Example (2)

S0 → S

S → ASA | aB

A → B | S

B → b | ε

(2) Remove ε-rule B → ε:

S0 → S

S → ASA | aB | a

A → B | S | ε

B → b | ε

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 44

Page 45: Regular languages: The Myhill-Nerode Theorem Context ...

Conversion Example (3)

S0 → S

S → ASA | aB | a

A → B | S | ε

B → b

(3) Remove ε-rule A → ε:

S0 → S

S → ASA | aB | a | AS | SA | S

A → B | S | ε

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 45

Page 46: Regular languages: The Myhill-Nerode Theorem Context ...

Conversion Example (4)

S0 → S

S → ASA | aB | a | AS | SA | S

A → B | S

B → b

(4) Remove unit rule S → S

S0 → S

S → ASA | aB | a | AS | SA | S

A → B | S

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 46

Page 47: Regular languages: The Myhill-Nerode Theorem Context ...

Conversion Example (5)

S0 → S

S → ASA | aB | a | AS | SA

A → B | S

B → b

(5) Remove unit rule S0 → S:

S0 → S | ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → B | S

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 47

Page 48: Regular languages: The Myhill-Nerode Theorem Context ...

Conversion Example (6)

S0 → ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → B | S

B → b

(6) Remove unit rule A → B:

S0 → ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → B | S | b

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 48

Page 49: Regular languages: The Myhill-Nerode Theorem Context ...

Conversion Example (7)

S0 → ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → S | b

B → b

Remove unit rule A → S:

S0 → ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → S | b | ASA | aB | a | AS | SA

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 49

Page 50: Regular languages: The Myhill-Nerode Theorem Context ...

Conversion Example (8)

S0 → ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → b | ASA | aB | a | AS | SA

B → b

(8) Final simplification – treat long rules:

S0 → AA1 | UB | a | SA | AS

S → AA1 | UB | a | SA | AS

A → b | AA1 | UB | a | SA | AS

A1 → SA

U → a

B → b√

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 50

Page 51: Regular languages: The Myhill-Nerode Theorem Context ...

Non-Context-Free Languages

The pumping lemma for finite automata andMyhill-Nerode theorem are our tools for showing thatlanguages are not regular.

We do not have a simple Myhill-Nerode type theorem.

However, will now show a similar pumping lemma forcontext-free languages.

It is slightly more complicated . . .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 51

Page 52: Regular languages: The Myhill-Nerode Theorem Context ...

Pumping Lemma for CFL

Also known as the uvxyz Theorem.

Theorem: If A is a CFL , there is an ℓ (critical length), suchthat if s ∈ A and |s| ≥ ℓ, then s = uvxyz where

for every i ≥ 0, uvixyiz ∈ A

|vy| > 0, (non-triviality)

|vxy| ≤ ℓ.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 52

Page 53: Regular languages: The Myhill-Nerode Theorem Context ...

Basic Intuition

Let A be a CFL and G its CFG.

Let s be a “very long” string in A.

Then s must have a “tall” parse tree.

And some root-to-leaf path must repeat a symbol.ahmmm,. . . why is that so?

T

R

u v x y z

R

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 53

Page 54: Regular languages: The Myhill-Nerode Theorem Context ...

Proof

let G be a CFG for CFL A.

let b be the max number of symbols in right-hand-sideof any rule.

no node in parse tree has > b children.

at depth d, can have at most bd leaves.

let |V | be the number of variables in G.

set ℓ = b|V |+2.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 54

Page 55: Regular languages: The Myhill-Nerode Theorem Context ...

Proof (2)

let s be a string where |s| ≥ ℓ

let T be parse tree for s with fewest nodes

T has height ≥ |V | + 2

some path in T has length ≥ |V | + 2

that path repeats a variable R

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 55

Page 56: Regular languages: The Myhill-Nerode Theorem Context ...

Proof (3)

Split s = uvxyz

T

R

u v x y z

R

each occurrence of R produces a string

upper produces string vxy

lower produces string xBased on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 56

Page 57: Regular languages: The Myhill-Nerode Theorem Context ...

Proof(4)

Replacing smaller by larger yields uvixyiz, for i > 0.

T

R

u v x y z

R

R

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 57

Page 58: Regular languages: The Myhill-Nerode Theorem Context ...

Proof (5)

Replacing larger by smaller yields uxz.

T

R

u

x

zTogether, they establish:

for all i ≥ 0, uvixyiz ∈ A

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 58

Page 59: Regular languages: The Myhill-Nerode Theorem Context ...

Proof (6)

Next condition is:

|vy| > 0 (out of uvxyz)

If v and y are both ε, then

T

R

u

x

z

is a parse tree for s with fewer nodes than T , contradiction.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 59

Page 60: Regular languages: The Myhill-Nerode Theorem Context ...

Proof (7)

Final condition is |vxy| ≤ ℓ:T

R

u v x y z

R |V|+2

b |V|+2

the upper occurrence of R generates vxy.

can choose symbols such that both occurrences of R liein bottom |V | + 1 variables on path.

subtree where R generates vxy is ≤ |V | + 2 high.

strings by subtree at most b|V |+2 = ℓ long. ♠Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 60

Page 61: Regular languages: The Myhill-Nerode Theorem Context ...

Non CFL Example

Theorem: B = {anbncn} is not a CFL.Proof: By contradiction.

Let ℓ be the critical length.

Consider s = aℓbℓcℓ.If s = uvxyz, neither v nor y can contain

both a’s and b’s, or

both b’s and c’s,

because otherwise uv2xy2z would have out-of-ordersymbols.But if v and y contain only one letter, then uv2xy2z has animbalance! ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 61

Page 62: Regular languages: The Myhill-Nerode Theorem Context ...

Non CFL Example (2)

The language C ={

aibjck|0 ≤ i ≤ j ≤ k}

is not context-free.Let ℓ be the critical length, and s = aℓbℓcℓ.

Let s = uvxyz

Neither v nor y contains two distinct symbols, becauseuv2xy2z would then have out-of-order symbols.

Then one of the symbols a,b,c does not appear neither inv nor in y.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 62

Page 63: Regular languages: The Myhill-Nerode Theorem Context ...

Non CFL Example (2) - cont.

The language C ={

aibjck|0 ≤ i ≤ j ≤ k}

is not context-free.Let ℓ be the critical length, and s = aℓbℓcℓ.Let s = uvxyz One of the symbols a,b,c does not appearneither in v nor in y:

If a’s do not appear, then the string uv0xy0z 6∈C, as it hasthe same number of a’s as s, but has fewer b’s or fewerc’s.

If b’s do not appear, then either a’s or c’s must appear inv or y (as they cannot be both empty). If a’s appear,then the string uv2xy2z 6∈C, as it contains more a’s thanb’s. If c’s appear, the string uv0xy0z 6∈C, as it containsmore b’s

If c’s do not appear, then the string uv2xy2z 6∈C, as itcontains more a’s or more b’s than c’s.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 63

Page 64: Regular languages: The Myhill-Nerode Theorem Context ...

Non CFL Example (3)

The language D ={

ww|w ∈ {0, 1}∗}

is not context-free.Let s = 0ℓ1ℓ0ℓ1ℓ As before, suppose s = uvxyz .

Recall that |vxy| ≤ ℓ.

if vxy is in the first half of s, uv2xy2z moves a 1 into thefirst position in second half.

if vxy is in the second half, uv2xy2z moves a 0 into thelast position in first half.

if vxy straddles the midpoint, then pumping down to uxz

yields 0ℓ1i0j1ℓ where i and j cannot both be ℓ. ♠Interestingly, we will shortly see that D =

{

wwR|w ∈ {0, 1}∗}

is a CFL.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 64

Page 65: Regular languages: The Myhill-Nerode Theorem Context ...

String Generators and String Acceptors

Regular expressions are string generators – they tell ushow to generate all strings in a language L

Finite Automata (DFA, NFA) are string acceptors – theytell us if a specific string w is in L

CFGs are string generators

Are there string acceptors for Context-Free languages?

YES! Push-down automata

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 65

Page 66: Regular languages: The Myhill-Nerode Theorem Context ...

A Finite Automaton

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 66

Page 67: Regular languages: The Myhill-Nerode Theorem Context ...

A PushDown Automaton

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 67

Page 68: Regular languages: The Myhill-Nerode Theorem Context ...

A PushDown Automaton

We add a memory device with restricted access: A stack(last in, first out; push/pop).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 68

Page 69: Regular languages: The Myhill-Nerode Theorem Context ...

An Example

Recall that the language {0n1n | n ≥ 0} is not regular.Consider the following PDA:

Read input symbols, one by one

For each 0, push it on the stack

As soon as a 1 is seen, pop a 0 for each 1 read

Accept if stack is empty when last symbol read

Reject ifStack is non-empty when end of input symbol readStack is empty but input symbol(s) still exist,0 is read after 1.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 69

Page 70: Regular languages: The Myhill-Nerode Theorem Context ...

PDA Configuration

A Configuration of a Push-Down Automata is a triplet

(<state>, <remaining input string>, <stack>).

A configuration is like a snapshot of PDA progress.

A PDA computation is a sequence of successiveconfigurations, starting from start configuration.

In the string describing the stack, we put the top of thestack on the left (technical item).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 70

Page 71: Regular languages: The Myhill-Nerode Theorem Context ...

Comparing PDA and Finite Automata

Our PDAs have a special end of input symbol, $. Thissymbol is not part of input alphabet Σ. (This technicalpoint differs from book. It is for convenience, and canbe handled differently too.)

PDA may be deterministic or non-deterministic.

Unlike finite automata, non-determinism adds power:There are some languages accepted only bynon-deterministic PDAs.

Transition function δ looks different than DFA or NFA cases,reflecting stack functionality.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 71

Page 72: Regular languages: The Myhill-Nerode Theorem Context ...

The Transition Function

Denote input alphabet by Σ and stack alphabet by Γ.

the domain of the transition function δ iscurrent state: Q

next input, if any: Σε,$ (=Σ ∪ {ε} ∪ {$})

stack symbol popped, if any: Γε(=Γ ∪ {ε})

and its range isnew state: Q

stack symbol pushed, if any: Γε

non-determinism: P(of two components above)

δ : Q × Σε,$ × Γε → P(Q × Γε)

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 72

Page 73: Regular languages: The Myhill-Nerode Theorem Context ...

Formal Definitions

A pushdown automaton (PDA) is a 6-tuple (Q,Σ,Γ, δ, q0, F ),where

Q is a finite set called the states,

Σ is a finite set called the input alphabet,

Γ is a finite set called the stack alphabet,

δ : Q × Σε,$ × Γε → P(Q × Γε) is the transition function,

q0 ∈ Q is the start state, and

F ⊆ Q is the set of accept states.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 73

Page 74: Regular languages: The Myhill-Nerode Theorem Context ...

Conventions

It will be convenient to be able to know when the stackis empty, but there is no built-in mechanism to do that.

Solution:Start by pushing $ onto stack.When you see it again, stack is empty.

Question: When is input string exhausted?Answer: When $ is read.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 74

Page 75: Regular languages: The Myhill-Nerode Theorem Context ...

Semi Formal Definitions

A pushdown automaton (PDA) M accepts a string x ifthere is a computation of M on x (a sequence of stateand stack transitions according to M ’s transitionfunction and corresponding to x) that leads to anaccepting state and empty stack.

The language accepted by M is the set of all strings x

accepted by M .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 75

Page 76: Regular languages: The Myhill-Nerode Theorem Context ...

Notation

Transition a, b → c means

if read a from inputand pop b from stackthen push c onto stack

Meaning of ε transitions:

if a = ε, don’t read inputsif b = ε, don’t pop any symbolsif c = ε, don’t push any symbols

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 76

Page 77: Regular languages: The Myhill-Nerode Theorem Context ...

Example

The PDA

accepts {0n1n|n ≥ 1}.Does it also accept the empty string?

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 77

Page 78: Regular languages: The Myhill-Nerode Theorem Context ...

Another Example

A PDA that accepts{

aibjck|i, j, k > 0 and i = j or i = k}

Informally:

read and push a’s

either pop and match with b’s

or else ignore b’s. Pop and match with c’s

a non-deterministic choice!

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 78

Page 79: Regular languages: The Myhill-Nerode Theorem Context ...

Another Example (cont.)

This PDA accepts{

aibjck|i, j, k > 0 and i = j or i = k}

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 79

Page 80: Regular languages: The Myhill-Nerode Theorem Context ...

Another Example (cont.)

A PDA that accepts{

aibjck|i, j, k > 0 and i = j or i = k}

Note: non-determinism is essential here!

Unlike finite automata, non-determinismdoes add power. Let us try to think how a proof thatnondeterminism is indeed essential could go,

...and realize it does not seem trivial or immediate.

We will later give a proof that the languageL = {xnyn|n ≥ 0} ∪

{

xny2n | n ≥ 0}

is accepted by anon-deterministic PDA but not by a deterministic one.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 80

Page 81: Regular languages: The Myhill-Nerode Theorem Context ...

PDA Languages

The Push-Down Automata Languages, LPDA, is the set ofall languages that can be described by some PDA:

LPDA = {L : ∃ PDA M ∧ L[M ] = L}

We already know LPDA ) LDFA, since every DFA is just aPDA that ignores the stack.

LCFG ⊆ LPDA ?

LPDA ⊆ LCFG ?

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 81

Page 82: Regular languages: The Myhill-Nerode Theorem Context ...

Equivalence Theorem

Theorem: A language is context free if and only if somepushdown automaton accepts it.

This time, both the “if” part and the “only if” part arenon-trivial.We will present a high level view of the proof (not all details)later.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 82