# Regular languages: The Myhill-Nerode Theorem Context

date post

23-Feb-2022Category

## Documents

view

0download

0

Embed Size (px)

### Transcript of Regular languages: The Myhill-Nerode Theorem Context

()Context-free Grammars

Non context-free languages: Examples

Push Down Automata (PDA)

Hopcroft and Ullman, 3.4

Sipser, 2.1, 2.2 & 2.3 Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 1

The Equivalence Relation ∼L

Define an equivalence relation ∼L on pairs of strings:

Let x, y ∈ Σ∗. We say that x∼L y if for every string z ∈ Σ∗, xz ∈ L if and only if yz ∈ L.

It is easy to see that ∼L is indeed an equivalence relation (reflexive, symmetric, transitive) on Σ∗.

In addition, if x∼L y then for every string z ∈ Σ∗, xz∼L yz as well (this is called right invariance).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 2

The Equivalence Relation ∼L

Like every equivalence relation, ∼L partitions Σ∗ to (disjoint) equivalence classes. For every string x, let [x] ⊆ Σ∗ denote its equivalence class w.r.t. ∼L (if x ∼L y then [x] = [y] – equality of sets).

Question is, how many equivalence classes does ∼L

induce?

In particular, is the number of equivalence classes of ∼L

finite or infinite?

Well, it could be either finite or infinite. This depends on the language L.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 3

Example: L = (ab ∪ ba)∗

[] = L

[a] = La

[b] = Lb

[aa] = L(aa ∪ bb)Σ∗

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 4

Classes of ∼L: More Examples

Let L1 ⊂ {0, 1}∗ contain all strings where the number of 1s is divisible by 4. Then ∼L1

has finitely many equivalence classes.

Let L2 = L(0∗10∗). Then ∼L2 has finitely many

equivalence classes.

Let L3 ⊂ {0, 1}∗ contain all strings of the form 0n1n. Then ∼L3

has infinitely many equivalence classes.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 5

Myhill-Nerode Theorem

L is regular ⇐⇒ ∼L has finitely many equivalence classes.

Three specific consequences:

L1 ⊂ {0, 1}∗ contains all strings where the number of 1s is divisible by 4. Then L1 is regular.

L2 = L(0∗10∗) is regular.

L3 ⊂ {0, 1}∗ contains all strings of the form 0n1n. Then L2 is not regular.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 6

Myhill-Nerode Theorem: Proof

=⇒ Suppose L is regular. Let M = (Q,Σ, δ, q0, F ) be a DFA accepting it. For every x ∈ Σ∗, let δ(q0, x) ∈ Q be the state where the computation of M on input x ends.

The relation ∼M on pairs of strings is defined as follows: x ∼M y if δ(q0, x) = δ(q0, y). Clearly, ∼M is an equivalence relation.

Furthermore, if x ∼M y, then for every z ∈ Σ∗, also xz ∼M yz. Therefore, xz ∈ L if and only if yz ∈ L.

This means that x ∼M y =⇒ x ∼L y.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 7

Myhill-Nerode Theorem: Proof (cont.)

=⇒

The equivalence relation ∼M has finitely many equivalence classes (at most the number of states in M).

We saw that x ∼M y =⇒ x ∼L y.

This means that the equivalence classes of ∼M refine those of ∼L.

Thus, the number of equivalence classes of ∼M is greater or equal than the number of equivalence classes of ∼L.

Therefore, ∼L has finitely many equivalence classes, as desired. ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 8

Myhill-Nerode Theorem: Proof (cont.)

⇐=

Suppose ∼L has finitely many equivalence classes. We’ll construct a DFA M that accepts L.

Let x1, . . . , xn ∈ Σ∗ be representatives for the finitely many equivalence classes of ∼L.

The states of M are the equivalence classes [x1], . . . , [xn].

The transition function δ is defined as follows: For all a ∈ Σ, δ([xi], a) = [xia] (the equivalence class of xia).

Why is δ well-defined? (Hint: right invariance: if x ∼L y, then xz ∼L yz).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 9

Myhill-Nerode Theorem: Proof (conc.)

The accept states are F = {[xi] |xi ∈ L}.

Easy: On input x ∈ Σ∗, M ends at state [x] (why?).

Therefore M accepts x iff x ∈ L (why?).

So L is accepted by DFA, hence L is regular. ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 10

Context Switch

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 11

Algorithmic Questions for NFAs

Q.: Given an NFA, N , and a string s, is s ∈ L(N)?

Answer: Construct the DFA equivalent to N and run it on w.

Q.: Is L(N) = ∅? Answer: This is a reachability question in graphs: Is there a path in the states’ graph of N from the start state to some accepting state. There are simple, efficient algorithms for this task.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 12

More Algorithmic Questions for NFAs

Q.: Is L(N) = Σ∗?

Answer: Check if L(N) = ∅.

Answer: Check if L(N2) ∩ L(N1) = ∅.

Q.: Given N1 and N2, is L(N1) = L(N2)?

Answer: Check if L(N1) ⊆ L(N2) and L(N2) ⊆ L(N1).

In the future, we will see that for stronger models of computations, many of these problems cannot be solved by any algorithm.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 13

Another, More Radical Context Switch

So far we saw finite automata, regular languages, regular expressions, Myhill-Nerode theorem and pumping lemma for regular languages.

We now introduce stronger machines and languages with more expressive power:

pushdown automata, context-free languages, context-free grammars, pumping lemma for context-free languages.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 14

Context-Free Grammars

A → 0A1

A → B

Each line is a substitution rule or production.

Each rule has the form: symbol → string. The left-hand symbol is a variable (usually upper-case).

A string consists of variables and terminals.

One variable is the start variable.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 15

Rules for Generating Strings

Write down the start variable (lhs of top rule).

Pick a variable written down in current string and a derivation that starts with that variable.

Replace that variable with right-hand side of that derivation.

Repeat until no variables remain.

Return final string (concatenation of terminals).

Process is inherently non deterministic.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 16

Example

⇒ 000#111

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 17

A Parse Tree

#0 10 0 1 1

Question: What strings can be generated in this way from the grammar G1? Answer: Exactly those of the form 0n#1n (n ≥ 0).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 18

Context-Free Languages

The language generated in this way is called the language of the grammar.

For example, L(G1) is {0n#1n|n ≥ 0}.

Any language generated by a context-free grammar is called a context-free language.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 19

A Useful Abbreviation

A → 0A1

A → B

are written as:

A → 0A1 | B

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 20

English-like Sentences

< SENTENCE > → < NOUN-PHRASE >< VERB >

< NOUN-PHRASE > → < ARTICLE >< NOUN >

< VERB > → touches | likes | sees

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 21

Deriving English-like Sentences

< SENTENCE > ⇒ < NOUN-PHRASE >< VERB >

⇒ < ARTICLE >< NOUN >< VERB >

⇒ a < NOUN >< VERB >

⇒ a boy < VERB >

⇒ a boy sees

a flower sees the girl touches

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 22

Derivation and Parse Tree

NOUN-PHRASE VERB

ARTICLE NOUN

a boy sees Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 23

Formal Definitions

A context-free grammar is a 4-tuple (V,Σ, R, S) where

V is a finite set of variables,

Σ is a finite set of terminals,

R is a finite set of rules: each rule is a variable and a finite string of variables and terminals.

S is the start symbol.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 24

Formal Definitions

If u and v are strings of variables and terminals,

and A → w is a rule of the grammar, then

we say uAv yields uwv, written uAv ⇒ uwv.

We write u ∗⇒ v if u = v or

u ⇒ u1 ⇒ . . . ⇒ uk ⇒ v.

Definition: The language of the grammar is {

w ∈ Σ∗ | S ∗⇒ w

}

.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 25

Example

R (Rules): S → (S) | SS | ε .

Some words in the language: (()), (()()).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 26

Arithmetic Example

V = {E, T, F} Σ = {a,+,×, (, )}

Rules: E → E + T | T

T → T × F | F

F → (E) | a

Strings generated by the grammar: a + a × a and (a + a) × a. What is the language of this grammar? Hint: arithmetic expressions. E = expression, T = term, F = factor.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 27

Parse Tree for a + a × a

E → E + T | T

T → T × F | F

TE

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 28

Parse Tree for (a + a) × a

E → E + T | T

T → T × F | F

E

E

F

T

T

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 29

Designing Context-Free Grammars - 1

If CFG is the union of several CFGs, rename variables (not terminals) so they are disjoint, and add new rule S → S1 | S2 | . . . | Si.

For example, to get a grammar for the language L = {0n1n| n ≥ 0} ∪ {1n0n| n ≥ 0} we construct the grammars:

S1 → 0 S1 1 |

S2 → 1 S2 0 |

and then add the rule:

S → S1 | S2

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 30

Designing Context-Free Grammars - 2

To construct CFG for a regular language, “follow” a DFA for the language. For initial state q0, make R0 the start variable. For state transition δ(qi, a) = qj add rule Ri → aRj to grammar. For each final state qf , add rule Rf → ε to grammar.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 31

Designing Context-Free Grammars - 3

For languages with linked substrings, a rule of form R → uRv may be helpful, forcing desired relation between substrings.

For L = {0n1n| n ≥ 0}:

S → 0 S 1 |

For L = {1n##0n| n ≥ 0}:

S → 1 S 0 | ##

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 32

Closure Properties

Context-Free Languages are closed under union : S → S1 | S2

concatenation S → S1S2

star S → ε | SS

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 33

More Closure Properties

intersection (

Not clear . . .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 34

Ambiguity

aXa+a

EEE

E

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 35

Ambiguity

We say that a string w is derived ambiguously from grammar G if w has two or more parse trees that generate it from G.

Ambiguity is usually not only a syntactic notion but also a semantic one, implying multiple meanings for the same string.

It is sometime possible to eliminate ambiguity by finding a different context free grammar generating the same language. This is true for the grammar above, which can be replaced by unambiguous grammar from slide 50.

Some languages (e.g. {1i2j3k | i = j or j = k} are inherentrly ambigous.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 36

Chomsky Normal Form

A simplified, canonical form of context free grammars. Every rule has the form

A → BC

A → a

S → ε

where S is the start symbol, A, B and C are any variable, except B and C not the start symbol, and A can be the start symbol.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 37

Theorem

Theorem: Any context-free language is generated by a context-free grammar in Chomsky normal form. Basic idea:

Add new start symbol S0.

Eliminate all ε rules of the form A → ε.

Eliminate all “unit” rules of the form A → B.

Patch up rules so that grammar generates the same language.

Convert remaining long rules to proper form.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 38

Proof Idea

Add new start symbol S0 and rule S0 → S. Guarantees that new start symbol does not appear on right-hand-side of a rule.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 39

Proof

for each R → uAv, add rule R → uv.

and so on: for R → uAvAw add R → uvAw, R → uAvw, and R → uvw.

for R → A add R → ε, except if R → ε has already been removed.

until all ε-rules not involving the original start variable have been removed.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 40

Proof

remove some A → B.

for each B → u, add rule A → u, unless this is previously removed unit rule. (u is a string of variables and terminals.)

until all unit rules have been removed.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 41

Proof

Finally, convert long rules. To replace each A → u1u2 . . . uk (for k ≥ 3), introduce new non-terminals

N1, N2, . . . , Nk−1

Nk−2 → uk−1uk ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 42

Conversion Example

Initial Grammar:

S0 → S

S → ASA | aB

A → B | S

B → b | ε

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 43

Conversion Example (2)

S0 → S

B → b | ε

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 44

Conversion Example (3)

S0 → S

A → B | S | ε

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 45

Conversion Example (4)

A → B | S

S0 → S

A → B | S

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 46

Conversion Example (5)

A → B | S

S0 → S | ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → B | S

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 47

Conversion Example (6)

A → B | S

A → B | S | b

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 48

Conversion Example (7)

A → S | b

S0 → ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → S | b | ASA | aB | a | AS | SA

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 49

Conversion Example (8)

A → b | ASA | aB | a | AS | SA

B → b

A → b | AA1 | UB | a | SA | AS

A1 → SA

U → a

B → b √

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 50

Non-Context-Free Languages

The pumping lemma for finite automata and Myhill-Nerode theorem are our tools for showing that languages are not regular.

We do not have a simple Myhill-Nerode type theorem.

However, will now show a similar pumping lemma for context-free languages.

It is slightly more complicated . . .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 51

Pumping Lemma for CFL

Also known as the uvxyz Theorem.

Theorem: If A is a CFL , there is an (critical length), such that if s ∈ A and |s| ≥ , then s = uvxyz where

for every i ≥ 0, uvixyiz ∈ A

|vy| > 0, (non-triviality)

|vxy| ≤ .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 52

Basic Intuition

Then s must have a “tall” parse tree.

And some root-to-leaf path must repeat a symbol. ahmmm,. . . why is that so?

T

R

R

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 53

Proof

let G be a…

Non context-free languages: Examples

Push Down Automata (PDA)

Hopcroft and Ullman, 3.4

Sipser, 2.1, 2.2 & 2.3 Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 1

The Equivalence Relation ∼L

Define an equivalence relation ∼L on pairs of strings:

Let x, y ∈ Σ∗. We say that x∼L y if for every string z ∈ Σ∗, xz ∈ L if and only if yz ∈ L.

It is easy to see that ∼L is indeed an equivalence relation (reflexive, symmetric, transitive) on Σ∗.

In addition, if x∼L y then for every string z ∈ Σ∗, xz∼L yz as well (this is called right invariance).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 2

The Equivalence Relation ∼L

Like every equivalence relation, ∼L partitions Σ∗ to (disjoint) equivalence classes. For every string x, let [x] ⊆ Σ∗ denote its equivalence class w.r.t. ∼L (if x ∼L y then [x] = [y] – equality of sets).

Question is, how many equivalence classes does ∼L

induce?

In particular, is the number of equivalence classes of ∼L

finite or infinite?

Well, it could be either finite or infinite. This depends on the language L.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 3

Example: L = (ab ∪ ba)∗

[] = L

[a] = La

[b] = Lb

[aa] = L(aa ∪ bb)Σ∗

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 4

Classes of ∼L: More Examples

Let L1 ⊂ {0, 1}∗ contain all strings where the number of 1s is divisible by 4. Then ∼L1

has finitely many equivalence classes.

Let L2 = L(0∗10∗). Then ∼L2 has finitely many

equivalence classes.

Let L3 ⊂ {0, 1}∗ contain all strings of the form 0n1n. Then ∼L3

has infinitely many equivalence classes.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 5

Myhill-Nerode Theorem

L is regular ⇐⇒ ∼L has finitely many equivalence classes.

Three specific consequences:

L1 ⊂ {0, 1}∗ contains all strings where the number of 1s is divisible by 4. Then L1 is regular.

L2 = L(0∗10∗) is regular.

L3 ⊂ {0, 1}∗ contains all strings of the form 0n1n. Then L2 is not regular.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 6

Myhill-Nerode Theorem: Proof

=⇒ Suppose L is regular. Let M = (Q,Σ, δ, q0, F ) be a DFA accepting it. For every x ∈ Σ∗, let δ(q0, x) ∈ Q be the state where the computation of M on input x ends.

The relation ∼M on pairs of strings is defined as follows: x ∼M y if δ(q0, x) = δ(q0, y). Clearly, ∼M is an equivalence relation.

Furthermore, if x ∼M y, then for every z ∈ Σ∗, also xz ∼M yz. Therefore, xz ∈ L if and only if yz ∈ L.

This means that x ∼M y =⇒ x ∼L y.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 7

Myhill-Nerode Theorem: Proof (cont.)

=⇒

The equivalence relation ∼M has finitely many equivalence classes (at most the number of states in M).

We saw that x ∼M y =⇒ x ∼L y.

This means that the equivalence classes of ∼M refine those of ∼L.

Thus, the number of equivalence classes of ∼M is greater or equal than the number of equivalence classes of ∼L.

Therefore, ∼L has finitely many equivalence classes, as desired. ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 8

Myhill-Nerode Theorem: Proof (cont.)

⇐=

Suppose ∼L has finitely many equivalence classes. We’ll construct a DFA M that accepts L.

Let x1, . . . , xn ∈ Σ∗ be representatives for the finitely many equivalence classes of ∼L.

The states of M are the equivalence classes [x1], . . . , [xn].

The transition function δ is defined as follows: For all a ∈ Σ, δ([xi], a) = [xia] (the equivalence class of xia).

Why is δ well-defined? (Hint: right invariance: if x ∼L y, then xz ∼L yz).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 9

Myhill-Nerode Theorem: Proof (conc.)

The accept states are F = {[xi] |xi ∈ L}.

Easy: On input x ∈ Σ∗, M ends at state [x] (why?).

Therefore M accepts x iff x ∈ L (why?).

So L is accepted by DFA, hence L is regular. ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 10

Context Switch

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 11

Algorithmic Questions for NFAs

Q.: Given an NFA, N , and a string s, is s ∈ L(N)?

Answer: Construct the DFA equivalent to N and run it on w.

Q.: Is L(N) = ∅? Answer: This is a reachability question in graphs: Is there a path in the states’ graph of N from the start state to some accepting state. There are simple, efficient algorithms for this task.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 12

More Algorithmic Questions for NFAs

Q.: Is L(N) = Σ∗?

Answer: Check if L(N) = ∅.

Answer: Check if L(N2) ∩ L(N1) = ∅.

Q.: Given N1 and N2, is L(N1) = L(N2)?

Answer: Check if L(N1) ⊆ L(N2) and L(N2) ⊆ L(N1).

In the future, we will see that for stronger models of computations, many of these problems cannot be solved by any algorithm.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 13

Another, More Radical Context Switch

So far we saw finite automata, regular languages, regular expressions, Myhill-Nerode theorem and pumping lemma for regular languages.

We now introduce stronger machines and languages with more expressive power:

pushdown automata, context-free languages, context-free grammars, pumping lemma for context-free languages.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 14

Context-Free Grammars

A → 0A1

A → B

Each line is a substitution rule or production.

Each rule has the form: symbol → string. The left-hand symbol is a variable (usually upper-case).

A string consists of variables and terminals.

One variable is the start variable.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 15

Rules for Generating Strings

Write down the start variable (lhs of top rule).

Pick a variable written down in current string and a derivation that starts with that variable.

Replace that variable with right-hand side of that derivation.

Repeat until no variables remain.

Return final string (concatenation of terminals).

Process is inherently non deterministic.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 16

Example

⇒ 000#111

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 17

A Parse Tree

#0 10 0 1 1

Question: What strings can be generated in this way from the grammar G1? Answer: Exactly those of the form 0n#1n (n ≥ 0).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 18

Context-Free Languages

The language generated in this way is called the language of the grammar.

For example, L(G1) is {0n#1n|n ≥ 0}.

Any language generated by a context-free grammar is called a context-free language.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 19

A Useful Abbreviation

A → 0A1

A → B

are written as:

A → 0A1 | B

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 20

English-like Sentences

< SENTENCE > → < NOUN-PHRASE >< VERB >

< NOUN-PHRASE > → < ARTICLE >< NOUN >

< VERB > → touches | likes | sees

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 21

Deriving English-like Sentences

< SENTENCE > ⇒ < NOUN-PHRASE >< VERB >

⇒ < ARTICLE >< NOUN >< VERB >

⇒ a < NOUN >< VERB >

⇒ a boy < VERB >

⇒ a boy sees

a flower sees the girl touches

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 22

Derivation and Parse Tree

NOUN-PHRASE VERB

ARTICLE NOUN

a boy sees Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 23

Formal Definitions

A context-free grammar is a 4-tuple (V,Σ, R, S) where

V is a finite set of variables,

Σ is a finite set of terminals,

R is a finite set of rules: each rule is a variable and a finite string of variables and terminals.

S is the start symbol.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 24

Formal Definitions

If u and v are strings of variables and terminals,

and A → w is a rule of the grammar, then

we say uAv yields uwv, written uAv ⇒ uwv.

We write u ∗⇒ v if u = v or

u ⇒ u1 ⇒ . . . ⇒ uk ⇒ v.

Definition: The language of the grammar is {

w ∈ Σ∗ | S ∗⇒ w

}

.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 25

Example

R (Rules): S → (S) | SS | ε .

Some words in the language: (()), (()()).

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 26

Arithmetic Example

V = {E, T, F} Σ = {a,+,×, (, )}

Rules: E → E + T | T

T → T × F | F

F → (E) | a

Strings generated by the grammar: a + a × a and (a + a) × a. What is the language of this grammar? Hint: arithmetic expressions. E = expression, T = term, F = factor.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 27

Parse Tree for a + a × a

E → E + T | T

T → T × F | F

TE

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 28

Parse Tree for (a + a) × a

E → E + T | T

T → T × F | F

E

E

F

T

T

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 29

Designing Context-Free Grammars - 1

If CFG is the union of several CFGs, rename variables (not terminals) so they are disjoint, and add new rule S → S1 | S2 | . . . | Si.

For example, to get a grammar for the language L = {0n1n| n ≥ 0} ∪ {1n0n| n ≥ 0} we construct the grammars:

S1 → 0 S1 1 |

S2 → 1 S2 0 |

and then add the rule:

S → S1 | S2

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 30

Designing Context-Free Grammars - 2

To construct CFG for a regular language, “follow” a DFA for the language. For initial state q0, make R0 the start variable. For state transition δ(qi, a) = qj add rule Ri → aRj to grammar. For each final state qf , add rule Rf → ε to grammar.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 31

Designing Context-Free Grammars - 3

For languages with linked substrings, a rule of form R → uRv may be helpful, forcing desired relation between substrings.

For L = {0n1n| n ≥ 0}:

S → 0 S 1 |

For L = {1n##0n| n ≥ 0}:

S → 1 S 0 | ##

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 32

Closure Properties

Context-Free Languages are closed under union : S → S1 | S2

concatenation S → S1S2

star S → ε | SS

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 33

More Closure Properties

intersection (

Not clear . . .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 34

Ambiguity

aXa+a

EEE

E

E

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 35

Ambiguity

We say that a string w is derived ambiguously from grammar G if w has two or more parse trees that generate it from G.

Ambiguity is usually not only a syntactic notion but also a semantic one, implying multiple meanings for the same string.

It is sometime possible to eliminate ambiguity by finding a different context free grammar generating the same language. This is true for the grammar above, which can be replaced by unambiguous grammar from slide 50.

Some languages (e.g. {1i2j3k | i = j or j = k} are inherentrly ambigous.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 36

Chomsky Normal Form

A simplified, canonical form of context free grammars. Every rule has the form

A → BC

A → a

S → ε

where S is the start symbol, A, B and C are any variable, except B and C not the start symbol, and A can be the start symbol.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 37

Theorem

Theorem: Any context-free language is generated by a context-free grammar in Chomsky normal form. Basic idea:

Add new start symbol S0.

Eliminate all ε rules of the form A → ε.

Eliminate all “unit” rules of the form A → B.

Patch up rules so that grammar generates the same language.

Convert remaining long rules to proper form.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 38

Proof Idea

Add new start symbol S0 and rule S0 → S. Guarantees that new start symbol does not appear on right-hand-side of a rule.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 39

Proof

for each R → uAv, add rule R → uv.

and so on: for R → uAvAw add R → uvAw, R → uAvw, and R → uvw.

for R → A add R → ε, except if R → ε has already been removed.

until all ε-rules not involving the original start variable have been removed.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 40

Proof

remove some A → B.

for each B → u, add rule A → u, unless this is previously removed unit rule. (u is a string of variables and terminals.)

until all unit rules have been removed.

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 41

Proof

Finally, convert long rules. To replace each A → u1u2 . . . uk (for k ≥ 3), introduce new non-terminals

N1, N2, . . . , Nk−1

Nk−2 → uk−1uk ♠

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 42

Conversion Example

Initial Grammar:

S0 → S

S → ASA | aB

A → B | S

B → b | ε

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 43

Conversion Example (2)

S0 → S

B → b | ε

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 44

Conversion Example (3)

S0 → S

A → B | S | ε

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 45

Conversion Example (4)

A → B | S

S0 → S

A → B | S

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 46

Conversion Example (5)

A → B | S

S0 → S | ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → B | S

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 47

Conversion Example (6)

A → B | S

A → B | S | b

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 48

Conversion Example (7)

A → S | b

S0 → ASA | aB | a | AS | SA

S → ASA | aB | a | AS | SA

A → S | b | ASA | aB | a | AS | SA

B → b

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 49

Conversion Example (8)

A → b | ASA | aB | a | AS | SA

B → b

A → b | AA1 | UB | a | SA | AS

A1 → SA

U → a

B → b √

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 50

Non-Context-Free Languages

The pumping lemma for finite automata and Myhill-Nerode theorem are our tools for showing that languages are not regular.

We do not have a simple Myhill-Nerode type theorem.

However, will now show a similar pumping lemma for context-free languages.

It is slightly more complicated . . .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 51

Pumping Lemma for CFL

Also known as the uvxyz Theorem.

Theorem: If A is a CFL , there is an (critical length), such that if s ∈ A and |s| ≥ , then s = uvxyz where

for every i ≥ 0, uvixyiz ∈ A

|vy| > 0, (non-triviality)

|vxy| ≤ .

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 52

Basic Intuition

Then s must have a “tall” parse tree.

And some root-to-leaf path must repeat a symbol. ahmmm,. . . why is that so?

T

R

R

Based on slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 53

Proof

let G be a…