Theory of Languages and Automata

Chapter 1- Regular Languages

& Finite State Automaton

Sharif University of Technology

Finite State Automaton

O We begin with the simplest model of Computation, called finite state machine or finite automaton.

O are good models for computers with an extremely limited amount of memory.

➜ Embedded Systems

O Markov Chains are the probabilistic counterpart of Finite Automata

Prof. MovagharTheory of Languages and Automata 2

Simple Example

O Automatic door

Prof. MovagharTheory of Languages and Automata 3

Door

Simple Example (cont.)

O State Diagram

O State Transition Table

Prof. MovagharTheory of Languages and Automata 4

Neither Front Rear Both

Closed Closed Open Closed Closed

Open Closed Open Open Open

Formal Definition

O A finite automaton is a 5-tuple (Q ,Σ ,δ ,q0, F), where

1. Q is a finite set called states,

2. Σ is a finite set called the alphabet,

3. δ : Q ×Σ ⟶ Q is the transition function,

4. q0 ∈ Q is the start state, and

5. F ⊆ Q is the set of accept states.

Prof. MovagharTheory of Languages and Automata 5

Example

O M1 = (Q, Σ, δ, q0, F) , where

1. Q = {q1, q2, q3},

2. Σ = {0,1},

3. δ is described as

1. q1 is the start state, and

2. F = {q2}.

Prof. MovagharTheory of Languages and Automata 6

0 1

q1 q1 q2

q2 q3 q2

q3 q2 q2

Language of a Finite machine

O If A is the set of all strings that machine M accepts, we say that A is the language of machine M and write: L(M) = A.

✓ We say that M recognizes A

or that M accepts A.

Prof. MovagharTheory of Languages and Automata 7

Example

O L(M1) = {w | w contains at least one 1 and

even number of 0s follow the last 1}.

Prof. MovagharTheory of Languages and Automata 8

Example

O M4 accepts all strings that start and end with a or with b.

Prof. MovagharTheory of Languages and Automata 9

Formal Definition

O M = (Q, Σ, δ, q0, F)

O w = w1w2 … wn ∀i, wi ∈ Σ

O M accepts w ⇔ ∃ r0, r1, …, rn ∀i, ri ∈ Q

1. r0 = q0 ,

2. δ(ri, wi+1) = ri+1, for i = 0, … , n-1,

3. rn ∈ F.

Prof. MovagharTheory of Languages and Automata 10

Regular Language

O A language is called a regular language if some finite automaton recognizes it.

Prof. MovagharTheory of Languages and Automata 11

Example

O L (M5) = {w | the sum of the symbols in w is 0 modulo 3, except that <RESET> resets the count to 0}.

✓ As M5 recognizes this language, it is a regular language.

Prof. MovagharTheory of Languages and Automata 12

Designing Finite Automata

O Put yourself in the place of the machine and then see how you would go about performing the machine’s task.

O Design a finite automaton to recognize the regular language of all strings that contain the string 001 as a substring.

Prof. MovagharTheory of Languages and Automata 13

Designing Finite Automata (cont.)

O There are four possibilities: You

1. haven’t just seen any symbols of the pattern,

2. have just seen a 0,

3. have just seen 00, or

4. have seen the entire pattern 001.

Prof. MovagharTheory of Languages and Automata 14

The Regular Operations

O Let A and B be languages. We define the regular operations union, concatenation, and star as follows.

O Union: A∪B = {x | x∈A or x∈B}.

O Concatenation: A∘B = {xy | x∈A and y∈B }.

O Star: A* = {x1x2…xk | k ≥ 0 and each xi∈A }.

Prof. MovagharTheory of Languages and Automata 15

Closure Under Union

O THEOREM

The class of regular languages is closed under the union operation.

Prof. MovagharTheory of Languages and Automata 16

Proof

O Let M1 = (Q1, Σ1, δ1, q1, F1) recognize A1, and

M2 = (Q2, Σ2, δ2, q2, F2) recognize A2.

O Construct M = (Q, Σ, δ, q0, F) to recognize A1∪A2.

1. Q = Q1 × Q2

2. Σ = Σ1 ∪ Σ2

3. δ((r1,r2),a) = (δ1(r1,a), δ2 (r2,a)).

4. q0 is the pair (q1, q2).

5. F is the set of pair in which either members in an accept state of M1 or M2.

F = (F1× Q2 ) ∪ (Q1 × F2) F ≠ F1× F2

Prof. MovagharTheory of Languages and Automata 17

Closure under Concatenation

O THEOREM

The class of regular languages is closed under the concatenation operation.

O To prove this theorem we introduce a new technique called nondeterminism.

Prof. MovagharTheory of Languages and Automata 18

Nondeterminism

O In a nondeterministic machine, several choices may exit for the next state at any point.

O Nondeterminism is a generalization of determinism, so every deterministic finite automaton is automatically a nondeterministic finite automaton.

Prof. MovagharTheory of Languages and Automata 19

Differences between DFA & NFA

O First, very state of a DFA always has exactly one exiting transition arrow for each symbol in the alphabet.

In an NFA a state may have zero, one, or more exiting arrows for each alphabet symbol.

O Second, in a DFA, labels on the transition arrows are symbols from the alphabet.

An NFA may have arrows labeled with members of the alphabet or ε. Zero, one, or many arrows may exit from each state with the label ε.

Prof. MovagharTheory of Languages and Automata 20

Deterministic vs. Nondeterministic Computation

Prof. MovagharTheory of Languages and Automata 21

Example

O Consider the computation of N1 on input 010110.

Prof. MovagharTheory of Languages and Automata 22

Example (cont.)

Prof. MovagharTheory of Languages and Automata 23

Formal Definition

O A nondeterministic finite automaton is a 5-tuple (Q ,Σ ,δ ,q0, F), where

1. Q is a finite set of states,

2. Σ is a finite alphabet,

3. δ : Q ×Σε ⟶ P(Q) is the transition function,

4. q0 ∈ Q is the start state, and

5. F ⊆ Q is the set of accept states.

Prof. MovagharTheory of Languages and Automata 24

ExampleO N1 = (Q, Σ, δ, q0, F) , where

1. Q = {q1, q2, q3, q4},

2. Σ = {0,1},

3. δ is given as

1. q1 is the start state, and

2. F = {q4}.

Prof. MovagharTheory of Languages and Automata 25

0 1 ε

q1 {q1} {q1,q2} ∅q2 {q3} ∅ {q4}q3 ∅ {q4} ∅q4 {q4} {q4} ∅

Equivalence of NFAs & DFAs

O THEOREM

Every nondeterministic finite automaton has an equivalent deterministic finite automaton.

O PROOF IDEA convert the NFA into an equivalent DFA that simulates the NFA.

If k is the number of states of the NFA, so the DFA simulating the NFA will have 2k states.

Prof. MovagharTheory of Languages and Automata 26

Proof

O Let N = (Q ,Σ ,δ ,q0, F) be the NFA recognizing A. We construct a DFA M =(Q' ,Σ' ,δ' ,q0', F’) recognizing A.

O let's first consider the easier case wherein N has no ε arrows.

1. Q' = P(Q).

2.

3. q0’ = {q0 }.

4. F' = {R∈ Q’ | R contains an accept state of N}.

Prof. MovagharTheory of Languages and Automata 27

Proof (cont.)

O Now we need to consider the ε arrows.

O for R ⊆ Q let

O E(R) = {q | q can be reached from R by traveling along 0 or more ε arrows}.

1. Q' = P(Q).

2. δ' (R,a) ={q∈ Q | q∈ E(δ(r,a)) for some r∈ R}.

3. q0’ = E({q0}) .

4. F' = {R∈ Q’ | R contains an accept state of N}.

Prof. MovagharTheory of Languages and Automata 28

Corollary

O A language is regular if and only if some nondeterministic finite automaton recognizes it.

Prof. MovagharTheory of Languages and Automata 29

Example

O D’s state set is

{∅,{1},{2},{3},{1,2},{1,3},{2,3},{1,2,3}}.

O The start state is E({1}) = {1,3}.

O The accept states are {{1},{1,2},{1,3},{1,2,3}}.

Prof. MovagharTheory of Languages and Automata 30

Example (cont.)

Prof. MovagharTheory of Languages and Automata 31

Afterremoving unnecessary states

CLOSURE UNDER THE REGULAR OPERATIONS

[Using NFA]

Theory of Languages and Automata Prof. Movaghar 32

Closure Under UnionO The class of regular languages is closed

under the Union operation.

Let NFA1 recognize A1 and NFA2 recognize A2. Construct NFA3 to recognize A1 U A2.

Theory of Languages and Automata Prof. Movaghar 33

Proof (cont.)

Theory of Languages and Automata Prof. Movaghar 34

Closure Under Concatenation Operation

O The class of regular languages is closed under the concatenation operation.

Theory of Languages and Automata Prof. Movaghar 35

Proof (cont.)

Theory of Languages and Automata Prof. Movaghar 36

Closure Under Star operation

O The class of regular languages is closed under the star operation.

O We represent another NFA to recognize A*.

Theory of Languages and Automata Prof. Movaghar 37

1. � = �� � �� The states of N are the states of N1 plus a new start state.

2. The state q0 is the new start state.3. � = �� � ��

4. The accept states are the old accept states plus the new start state.

5. Define � so that for any � ∈ � ��� � ∈ ∑� :

Proof (cont.)

Theory of Languages and Automata Prof. Movaghar 38

Recursive Definition?

Regular Expression

O Say that R is a regular expression if R is:

1. a for some a in the alphabet Σ

2. �

3. ∅

4. ����� , where �� and �� are regular exp.

5. ��� �� , where �� and �� are regular exp.

6. ��∗ , where �� and �� are regular exp.

Theory of Languages and Automata Prof. Movaghar 39

Regular Expression Language

O Let R be a regular expression. L( R ) is the language that is defined by R:

1. if � = � for � ∈ Σ then L( R )={a}

2. if � = � then � � = {�}

3. if � = ∅ then � � = ∅

4. if � = ����� then � � = � �� � � ��

5. i� � = �� � �� then � � = � �� � � ��

6. if � = ��∗ then � � = � ��

∗

Theory of Languages and Automata Prof. Movaghar 40

1. 0 ∪ � 1∗ = 01∗ ∪ 1∗

2. Σ∗1Σ∗ = {�|� contains at least one 1}3. 0∗10∗ = {�|� contains a single 1}4. Σ∗001Σ∗ = {�|� contains 001 as a substring}5. 01�10 = {01,10}6. (ΣΣ)∗= {�|� is a string of even length}7. (ΣΣΣ)∗= {�|the lentgh of w is a multiple of 3}8(ΣΣΣΣ)∗= {�|the lentgh of w is a multiple �� 4}9. 0 ∪ � 1∗ = 01∗ ∪ 1∗

10. 0 ∪ � (1 ∪ �) = {�, 0,1,01}11. 1∗∅ = ∅12. ∅∗ = {�}

Examples(cont.)

Theory of Languages and Automata Prof. Movaghar 41

O A language is regular if and only if some regular expression describes it.

Lemma:

O If a language is described by a regular expression, then it is regular.

O If a language is regular, then it is described by a regular expression.

Equivalence of DFA and Regular Expression

Theory of Languages and Automata Prof. Movaghar 42

Theory of Languages and Automata Prof. Movaghar 43

O We consider the six cases in the formal definition of regular expressions

Building an NFA from the Regular Expression

Theory of Languages and Automata Prof. Movaghar 44

Examples

O We need to show that, if a language A is regular, a regular expression describes it!

O First we show how to convert DFAs into GNFAs, and then GNFAs into regular expressions.

O We can easily convert a DFA into a GNFA in the special form.

Prof. MovagharTheory of Languages and Automata 45

Other direction of the proof

A generalized nondeterministic finite automaton is a 5-tuple, a 5-tuple (Q ,Σ ,δ ,qstart , qaccept), where

1. Q is a finite set called states,

2. Σ is a the input alphabet,

3. δ : (� − {�������}) × (Q − {������} ⟶ � is

the transition function,

4. qstart is the start state, and

5. qaccept is the accept state.

Formal Definition

Theory of Languages and Automata Prof. Movaghar 46

For convenience we require that GNFAs always have a special form that meets the following conditions:

1. The start state has transition arrows going to every other state but no arrows coming in from any other state.

2. There is only a single accept state, and it has arrows coming in from every other state but no arrows going to any other state. Furthermore, the accept state is not the same as the start state.

3. Except for the start and accept states, one arrow goes from every state to every other state and also from each state to itself.

Assumptions

Theory of Languages and Automata Prof. Movaghar 47

Acceptance of Languages for GNFA

O A GNFA accepts a string w in Σ* if w = w1 w2 … wk, where each wi is in Σ* is in Σ* and a sequence of q0, q1, …, qk exists such that

1. q0 = qstart is the start state,

2. qk = qaccept is the accept state, and

3. For each i, we have wi L(Ri) where Ri = δ(qi-1, qi); in other words Ri is the expression on the arrow from qi-1 to qi.

How to Eliminate a State?

Theory of Languages and Automata Prof. Movaghar 49

Example

Theory of Languages and Automata Prof. Movaghar 50

Example

Theory of Languages and Automata Prof. Movaghar 51

O A grammar G is a 4-tuple

G = (V, Σ, R, S)

where:

1. V is a finite set of variables,

2. Σ is a finite, disjoint from V, of terminals,

3. R is a finite set of rules,

4. S is the start variable.

Grammar

Theory of Languages and Automata Prof. Movaghar 52

A rule is of the form� → �

where � ∈ (� ∪ Σ)

+ and � ∈ (� ∪ Σ)

∗

The rules are applied in the following manner: given a string w of the form

w = uxv,We say that the rule x → y is applicable to this string, and we may use it to replace x with y, thereby obtaining a new string

z = uyv,This is written as

� ⇒ �.

Rule

Theory of Languages and Automata Prof. Movaghar 53

O If�� ⇒ �� ⇒ … ⇒ ��

we say that W1 derives Wn and write

�� ⇒∗ ��

Thus, we always have� ⇒∗ �

Derivation

Theory of Languages and Automata Prof. Movaghar 54

Let G = (V, Σ, R, S) be a grammar. Then, the set

� � = {� ∈ Σ∗: � ⇒∗ �}

is the language generated by G.

Language of a Grammar

Theory of Languages and Automata Prof. Movaghar 55

Consider the grammar

G = ({S}, {a,b}, P, S}

with P given by

S → aSb

S → ε

Then � → ��� → ����� → ����

So we can write� ⇒∗ ����

Then,

L(G) = {an bn: n ≥ 0}

Example

Theory of Languages and Automata Prof. Movaghar 56

Consider the grammar

G = ({S}, {a,b}, P, S}

with P given by

S → aSb

S → ε

The above grammar is usually written as:

G: S → aSb | ε

A Notation for Grammars

Theory of Languages and Automata Prof. Movaghar 57

A grammar G = (V, Σ, R, S) is said to be right-linear if all rules are of the form

A → xB

A → x

Where A, B V, and X Σ*. A grammar is said to be left-linear if all rules are of the form

A → Bx

A → x

A regular grammar is one that is either right-linear orleft-linear.

Regular Grammar

Theory of Languages and Automata Prof. Movaghar 58

Let G = (V, Σ, R, S) be a right-linear grammar. Then:

L(G) is a regular language.

Theory of Languages and Automata Prof. Movaghar 59

Theorem

Construct a NFA that accepts the language generated by the grammar

V0 → aV1

V1 → abV0 | b

V0VfV1

V2

b

a

a

b

Theory of Languages and Automata Prof. Movaghar 60

Example

Let L be a regular language on the alphabet Σ.

Then:

There exists a right-linear grammar G = (V, Σ, R, S)

Such that L = L(G).

Theory of Languages and Automata Prof. Movaghar 61

Theorem

Theorem A language is regular if and only if there exists a left-linear grammar G such that L = L(G).

Outline of the proof:

Given any left-linear grammar with rules of the form

A → Bx

A → x

We can construct a right-linear Ĝ by replacing every such rule of G with

A → xRB

A → xR

We have L(G) = L(Ĝ)R .

Theory of Languages and Automata Prof. Movaghar 62

Theorem

A language L is regular if and only if there exists a regular grammar G such that L = L(G).

� �� ������� ↔ ∃ �: � = � � ; � �� �������

Prof. MovagharTheory of Languages and Automata 63

Theorem