cs302 slides 2006 v22 - University of Virginia School of ...robins/cs302/cs302_slides... · 114...

107

Formal languages

• Alphabet: finite set of symbols Ex: Σ = a,b,c

• String: finite symbol sequence Ex: w = abcaabbcc

• Length: # of symbols Ex: |bca| = 3

• String concatenation: w1w2 or w1•w2 Ex: w1 = aaa w2 = bb w1w2= aaabb

109

• Language Concatenation: L1L2=w1w2 | w1∈L1, w2∈L2

LL = L2

Lk=LLk-1 L0=ε

Ex: a, b1, 2, 3 = a1, a2, a3, b1, b2, b3

• Kleene closure:

L* = L0 ∪ L1 ∪ L2 ∪ L3 ∪...

L+ = L1 ∪ L2 ∪ L3 ∪...

L+ = LL*

110

• Trivial language: ε

• Empty language: Ø

• All finite strings: Σ*

(L*)*=L*

L⊆Σ* ∀L

• Σ* is countable: |Σ*| = |Z|

a, b, ..., aa, ab, ..., aaa, aab, ...

• 2Σ* is uncountable (why?)

111

• An algorithm is a “string”

Ex: “main() int k; k=1;”

⇒ |algorithms| is countable • A problem is a “language”

Ex: 2$4, 3$9, 4$16, 5$25, ...

⇒ |problems| is uncountable ⇒ ∃ more problems than algorithms!

[Alan Turing, 1936]

• A description is a “string”

⇒ ∃ undescribable languages!

112

Q: Prove that 2Σ* is uncountable.

113

Machine Models

“The Chomsky hierarchy”

Language type Automata type • regular • finite • context-free • pushdown • context-sensitive • linear-bounded • unrestricted • Turing

114

Finite Automata

Idea: “machine” changes states while processing symbols, one at a time. 1) Finite set of states Q Q = q0, q1, q3, ..., qk 2) Transition function δ δ: Q×Σ → Q

3) Initial state: q0 ∈ Q

4) Final states: F ⊆ Q

Def: FA M=(Q, Σ, δ, q0, F)

115

Pictorial Representation

• State: qi

• Initial state:

• Final State:

• Transition: a

Ex: FA accepting all odd-length strings over Σ = a:

aq0

aq1

116

String Recognition Automaton consume string w∈Σ*, one symbol at a time, & change states. • Acceptance: end in a final state

• Rejection: anything else (including hang-up) Ex:

aq0 q1

a,b

117

Language Recognition

Extend δ to strings:

δ: Q×Σ* → Q

δ(q0,wx) = δ(δ(q0,w),x)

δ(q,ε) = q L(M) = set of strings accepted by M

= w∈Σ*| δ(q0,w) ∈ F

“regular” ≡ accepted by some FA

118

Ex: All strings over Σ = 0, 1 with odd # of 0’s and odd # of 1’s:

1q0

q2 q3

1

1

1

0

0

0

0

q1

Q: All strings over Σ = 0, 1 with

even # of 0’s and 1’s.

Q: All strings in 0,1,2* with even #’s of 0’s,1’s, and 2’s.

119

Σ = a, b Ex: L1=w|w has 3 consecutive b’s

a a,b

b

a

q2ba

q1b q3q0

Ex: L2=w | w has no 3 consec. b’s

a a,b

b

a

q2ba

q1b q3q0

Note: L2 = Σ* - L1

120

Thm: If L is regular, so is L’. Proof:Complement the final state set: i.e.,given M=(Q,Σ,δ,q0,F), construct

FA (Q,Σ,δ,q0,Q-F) accepting Σ*-L(M).

Ex:

aq0 q1

a,b

⇒

aq0 q1

a,b

bq0 q1

a,b

a

q2

a,b

⇒

bq0 q1

a,b

a

q2

a,b

121

Nondeterminism • Generalization of determinism • Many “next-moves”: δ: Q×Σ → 2Q • Computation is a “tree” • Acceptance: ∃ path to accepting leaf Ex: 6th symbol from end is an “a”:

aq0 q2a,bq1 q3

a,b

a,b

a,b

q5a,bq6 q4

a,b

122

Thm: DFAs can simulate NFAs. Proof: (powerset construction) Every subset of Q becomes a single state in our new machine! i.e., given NFA M=(Q,Σ,δ,q0,F), consruct DFA M’=(Q,Σ,δ’,q0,F) with δ’([q1,q2,...,qi],a) = [p1,p2,...,pj] iff δ(q1,q2,...,qi,a) = p1,p2,...,pj. Note: size of new DFA can be exponential in size of old NFA!

123

Thm: ∩ preserves regularity. Proof: (cross product construction) Given M1=(Q1, Σ, δ1, q’, F1) and M2=(Q2, Σ, δ2, q”, F2),

construct M=(Q, Σ, δ, q, F) Q = Q1×Q2

q=(q’,q”) F = F1×F2

δ:Q×Σ → Q δ((qi,qj),x) = (δ1(qi,x),δ2(qj,x))

i.e., “parallel” simulation!

124

Thm: ∪ preserves regularity. Proof: (by DeMorgan’s law)

L1 ∪ L2 = L1 ∩ L2 Thm: - preserves regularity. Proof: L1 - L2 = L1 ∩ L2

L1 L2

125

Thm: ε-transitions do not increase recognition power of FAs.

a

b

a

q2ba

q1q0

a,b

b

q3aq5 q6

a,b

q4

ε

ε

a

b

a

q2ba

q1q0

b

aq5 q3

a,b

q4

ε

ε

126

Regular Expressions

• Ø • ε • x ∀ x ∈ Σ • If R,S regular expressions, then so are: • (R+S) • RS • R* Note: “+” denotes union Ex: a*b* a*+b* a(a+b)*

(a+b)*a(a+b)* aaaaa*

127

Thm: Any regular expression is accepted by some FA. Proof: (by construction)

q0

⇒ Ø

q0 ⇒ ε

x q1q0 ⇒ x ∀x∈ Σ

Assume M1 accepts L(R), M2 accepts L(S)

ε

ε

M2

M1

⇒ (R+S)

128

εM1 M2

⇒ RS

εM1 qFε

ε

εq0

⇒ R*

Thm: Each FA language can be denoted by a regular expression. Proof idea: construct regular expression for union of all paths from q0 to F.

⇒ FAs ≡ regular expressions

129

Regular Expression Identities • RS ≠ SR • R+S = S+R • R(ST) = (RS)T • R(S+T) = RS+RT • (R+S)T = RT+ST

• Ø* = ε

• (R*)* = R*

• (ε + R)* = R*

• (R*S*)* = (R+S)*

130

2-Way Finite Automata

• Can move backwards on input

i.e., δ:Q×Σ → Q×left,right

I compute, therefore I am.

M

Thm: 2-Way capability does not increase the power of FAs.

Proof idea: “crossing sequences”

131

Pumping

Thm: Almost all strings in a regular language contain a pumpable substring.

∃ N ∋ ∀z∈L, |z|≥N ∃ u,v,w∈Σ* ∋ z=uvw, |uv|≤N, |v|≥1, uviw∈L ∀ i≥0.

Note: not every long string in L is of the form uviw !

Ex: Show L = aibi | i≥1 not regular. Assume L regular. ∃ k≥N ∋ akbk = uvw ∈ L, |v≥1.

⇒ uviw∈L ∀ i≥0.

But if v∈a+ or v∈b+ then uw∉L, else if

v∈a+b+ then uv2w∉L, a contradiction.

132

Decidable FA Questions Def: A problem is decidable iff there exists an algorithm which can determine (within finite time) the correct answer for any instance.

Q1: Is L(M) = Ø ? Hint: look at w ∋ |w|≤N Q2: Is L(M) infinite ? Hint: look at w ∋ N≤|w|≤2N Q3: Is L(M1) = L(M2) ?

Hint: look at (L1-L2)∪(L2-L1)

133

Context-Free Grammars

• Finite set of variables V

• Finite set of terminals Τ

• Finite set of productions P

• Start symbol S

productions: A → α Α ∈ V α∈ (V∪T)*

Applying A → α to vAw yields vαw

i.e., does not depend on “context”

G = (V,T,P,S)

134

Grammars as Generators L(G) = w∈T*| S ⎯→⎯* w

L is CF ⇔ ∃ CFG G ∋ L = L(G) Ex: G: S→ 0S1 S→ ε

A derivation in G:

S→ 0S1→ 00S11→ 000S111 → . . .→ 0kS1k→ 0kε1k = 0k1k

⇒ L(G) = aibi | i≥1 is CF

Q: If L is CF, is L_ necessarily CF?

135

Pushdown Automata PDA ≡ FA with a stack:

I'm a PDA (and I like it!) ¢

M

$ZYX

Stack

L(M) = language accepted by PDA M

136

Pushdown Automata (PDA)

1) Finite set of states Q

2) Transition function δ

3) A stack

• Q = q0, q1, q3, ..., qk

• Input alphabet Σ

• Stack alphabet Γ

• δ:Q×(Σ+ε+¢)×(Γ+ε+$)→Q×Γ∗

• Initial state: q0 ∈ Q

• Initial stack: $

• Final states: F ⊆ Q

PDA M=(Q, Σ, Γ, δ, q0, $, F)

137

Ex: Σ = 0,1 Γ = x

PDA M: q2¢/$/ε

0/ε/x

q1q01/x/ε

1/x/ε

L(M) = aibi | i≥1

M Fully Specified:

q3 (S+¢+ε)/(C+$+ε)/ε

1/$/ε¢/x/ε1/$/ε

q2¢/$/ε

0/ε/x

q1q01/x/ε

1/x/ε

138

Nondeterministic PDAs

Many “next-moves”: δ: Q×(Σ+ε+¢)×(Γ+ε+$) → 2Q×Γ∗ M is deterministic if:

• ∀ q∈Q, z∈Γ δ(q,ε,z)≠Ø ⇒ δ(q,a,z)=Ø ∀ a∈Σ

i.e., ε-moves are not allowed along with input-dependent moves

• ∀ q∈Q, z∈Γ, a∈Σ∪ε |δ(q,a,z)|≤1

i.e., no choices allowed

Otherwise, M is nondeterministic.

139

Ex: L1 = wwR | w ∈ a,b*

q2¢/$/ε

b/ε/b

q1q0ε/ε/ε

a/ε/ab/b/εa/a/ε

vs. L2 = w#wR | w ∈ a,b,#*

q2¢/$/ε

b/ε/b

q1q0#/ε/ε

a/ε/ab/b/εa/a/ε

Thm: wwR|w∈a,b* can not be accepted by any deterministic PDA.

140

Ambiguous CFGs

CFG G is ambiguous if some word in L(G) has >1 non-isomorphic derivations. EX: S → SS | ε | a | b S → SS → aa S → SS → SSS → aa CFL L is inherently ambiguous if every CFG for L is ambiguous. Ex: the following is an ambiguous CFL: anbncmdm | n≥1,m≥1 ∪ anbmcmdn | n≥1,m≥1

141

Normal Forms

Chomsky normal form(CNF): All rules of form: A → BC A → a Greibach normal form (GNF): All rules of form: A → aα where α ∈ V*, a ∈ T Thm: Every CFL may be represented in CNF or GNF.

142

2-Way PDAs

• Can move backwards on input i.e., δ:Q×(Σ+ε+¢)×(Γ+ε+$) →Q×Γ∗×L,R

I'm a PDA (and I like it!) ¢

M

$ZYX

Stack

Thm: a 2-Way "input-tape" can increase the power of PDAs.

143

Pumping for CFLs

Thm: Almost all strings in a context-free language contain a pumpable substring.

L is CF ⇒ ∃ N ∋ ∀z∈L, |z|≥N ∃ u,v,w,x,y∈Σ* ∋ z=uvwxy, |vwx|≤N, |vx|≥1, uviwxiy∈L ∀ i≥0.

Proof idea: put grammar for L into CNF and find a repeated variable along some path in a derivation of a long string.

Ex: Can show L = aibici | i≥1 not CF.

144

Thm: CFLs are closed under ∪.

Thm: CFLs are closed under Kleene.

Thm: CFLs are closed under ∩ with regular sets.

145

Thm: CFLs are not closed under ∩.

Thm: CFLs are not closed under complementation.

146

Decidable PDA Questions

Q1: Is L(M) = Ø ? Q2: Is L(M) finite ? Q3: Is L(M) infinite ? Q4: Is w ∈ L(M) ?

147

Turing Machines TM ≡ FA with a tape:

TM

Tape0 0 0 1 1 1 2 2 2 $

L(M) = language accepted by TM M

148

Turing Machine Model

1) Finite set of states Q

2) Transition function δ

3) A semi-infinite tape

• Q = q0, q1, q3, ..., qk • Tape alphabet Γ • Input alphabet Σ ⊆ Γ

• δ:Q×(Γ+β)→Q×Γ×L,R

• Initial state: q0 ∈ Q

• Blank symbol: β

• Final states: F ⊆ Q

TM M=(Q, Σ, Γ, δ, q0, β, F)

149

Nondeterministic TMs

Many “next-moves”: δ:Q×(Γ+β)→2Q×Γ×L,R M is deterministic if: ∀ q∈Q, a∈Γ |δ(q,a)|≤1 i.e., no multiple choices allowed Otherwise, M is nondeterministic. Q: Does non-determinism increase the power of TMs?

150

Problem 1: design a TM to accept L1 = 0n1n2n | n≥1 Problem 2: design a TM to accept L2 = n$12n | n≥1 Problem 3: give generating grammars for L1 and L2.

151

TM “Enhancements”

• Larger alphabet

• Doubly-infinite tape

• Multiple tapes

• Multiple heads

• Multi-dimensional tape

• Non-determinism

• Combinations

Thm: These "enhancements" do not increase the power of TMs.

• "Oracles"?

152

Church's Thesis

Computable ≡ TM computable

• Recognition vs. enumeration

• Recursively Enumerable: ≡ TM-generatable ≡ recongizable by TM that stops on all "yes" instances

• Recursive: ≡ TM-generatable in order ≡ recongizable by TM that stops on all instances

153

Decidability

Def: L is decidable if ∃ an always-halting TM M that accepts L.

Ex: Finite languages are decidable. Ex: Regular languages are decidable. Ex: CF languages are decidable. Ex: Halting problem is not decidable.

Def: L is (Turing-)recognizable if ∃ an TM M that accepts L.

Ex: Halting problem is recognizable. Ex: Its complement is not recognizable. Theorem: L is decidable iff L and its complement are Turing-recognizable.

154

Reducibilities

Def: language A is reducible to a lang. B if ∃ computable function ƒ:∑*→∑* where ∀w, w∈A ⇔ ƒ(w)∈B

Aƒ

ƒ(w)w

B•

•

• ƒ is called a “reduction” of A to B • Denotation: A ≤ B

Theorem: If A ≤ B and B is decidable, then A is decidable.

Corrolary: If A ≤ B and A is undecidable, then B is undecidable.

155

Examples of Reductions:

Hε: Given TM M, does M halt on ε?

Reduction from the Halting Problem H:

Given an arbitrary TM M & string w, construct a new TM M’ that for input x:

1) Completely erases the input tape; 2) Writes w onto the input tape; 3) Simulates M on the input w; 4) Accepts ⇔ M accepts w. ⇒ M’ halts on ε ⇔ M halts on w

An algorithm for Hε can thus be used to solve the general Halting Problem H!

⇒ Hε is not decidable

156

LØ: Given TM M, is L(M)=Ø?


Given an arbitrary TM M & string w, construct a new TM M’ that for input x:

1) Completely erases input tape; 2) Writes w onto input tape; 3) Simulates M on the input w; 4) Accepts ⇔ M accepts w.

⇒ L(M’)=∑* if M halts on w; L(M’)=Ø if M does not halt on w

An algorithm for LØ can thus be used to solve the general Halting Problem H!

⇒ LØ is not decidable

157

Lreg: Given TM M, is L(M) regular?


Given an arbitrary TM M & string w, construct a new TM M’ that on input x:

1) Accepts if x∈0n1n; else: 2) Completely erases input tape; 3) Writes w onto input tape; 4) Simulate M on the input w; 5) Accepts ⇔ M accepts w.

⇒ L(M’)=∑* if M halts on w L(M’)= 0n1n otherwise

An algorithm for Lreg can thus be used to solve the general Halting Problem H!

⇒ Lreg is not decidable

158

Rice's Theorem

Def: property P ≡ a set of languages Ex: P1=L | L is a regular language

P2=L | L is a finite language P3=L | L contains the string 011

L is said to have the property P if L∈P

Ex: a*b* has property P1 a*b* does not have property P2

Def: A property P is trivial if P=Ø or

P contains all R.E. languages

159

The trivial properties are decidable: Ex: Pall=L | L is an R.E. language

Pnone= Ø TM that determines property Pall:

Mallx yes

TM that determines property Pnone:

Mnonex no

Q: What other properties are decidable?

A: None!

160

Thm [Rice]: All non-trivial properties of the recursive languages are undecidable.

Proof: Let P be a non-trivial property.

Select L∈P (WLOG assume L≠Ø, otherwise consider complement of P).

Assume ML decides L:

MLyes

x no

(note: input x is a TM description)

Reduction strategy: “solve” the halting problem using ML as an “oracle”.

161

Given TM M & string w, construct M’:

Mw yesstart

MLyes yes

x

M'

Analyze language of M’:

Note: L(M') is either L or Ø, and L(M') = L ⇔ w ∈ L(M) L(M') = Ø ⇔ w ∉ L(M)

Assume ∃ ΤΜ MP that decides P. Now use MP to determine if M' has property P, i.e., if L(M') = L∈P

But this solves the halting problem for an arbitrary M(w)! ⇒ Property P is not decidable!

162

Ex: Undecidable Properties:

• L is empty

• L is finite

• L is infinite

• L is regular

• L is context-free

• L is decidable

• L= ∅

• L= ∑*

• L contains an odd string

• L contains a palindrome

163

Algorithms

• Existance • Efficiency

Analysis

• Correctness • Time • Space • Other resources

Worst case analysis (as function of input size |w|)

Asymptotic growth: Ο Ω Θ ο

164

Upper Bounds

f(n) = Ο(g(n)) ⇔ ∃ c,k > 0 ∋ |f(n)| ≤ c⋅|g(n)| ∀ n>k

Lim f(n) / g(n) exists n→∞

“f(n) is big-O of g(n)”

Ex: n = O(n2)

33n+17 = O(n)

n8-n7 = O(n123)

n100 = O(2n)

213 = O(1)

165

Lower Bounds

f(n)=Ω(g(n)) ⇔ g(n)=Ο(f(n))

Lim g(n) / f(n) exists n→∞

“f(n) is Omega of g(n)”

Ex: 100n = Ω(n)

33n+17 = Ω(log n)

n8-n7 = Ω(n8)

213 = Ω(1/n)

1= Ω(213)

166

Tight Bounds

f(n) = Θ(g(n)) ⇔ f(n)=Ο(g(n)) ^ g(n)=Ο(f(n)) “f(n) is Theta of g(n)”

Ex: 100n = Θ(n)

33n+17 + log n = Θ(n)

n8-n7-n-13 = Θ(n8)

213 = Θ(1)

3+cos(2n) = Θ(1)

167

Loose Bounds

f(n) = ο(g(n)) ⇔ f(n)=Ο(g(n)) ^ f(n)≠Ω(g(n))

Lim f(n)/g(n) = 0 n→∞

“f(n) is little-o of g(n)”

Ex: 100n = ο(n log n)

33n+17 + log n = ο(n2)

n8-n7-n-13 = ο(2n)

213 = ο(log n)

3+cos(2n) = ο(√n)

168

Growth Laws

Let f1(n)=O(g1(n)) and f2(n)=O(g2(n))

Thm: f1(n) + f2(n) = O(max(g1(n),g2(n)))

Thm: f1(n) • f2(n) = O(g1(n) • g2(n)) Thm: nk = O(cn) ∀ c,k>0

Ex: n1000 = O(1.001n)

169

Recurrences

T(n) = a•T(n/b) + f(n)

let c = logba

Thm: f(n)=O(nc-ε) ⇒ T(n)=Θ(nc) f(n)=Θ(nc) ⇒ T(n)=Θ(nc log n) f(n)=Ω(nc+ε) ^ a•f(n/b) ≤ d•f(n) ∀ d<1, n>n0 ⇒ T(n)=Θ(f(n))

Ex: T(n) = 9T(n/3)+n ⇒ T(n)=Θ(n2)

T(n) = T(2n/3)+1 ⇒ T(n)=Θ(log n)

170

Stirling's Formula

n! = 1•2•3• . . . •(n-2)•(n-1)•n

n! = 2Πn • (ne )

n• (1 + Θ(1

n ))

n! ≈ (ne )

n

log(n!) = O(n log n)

• Useful in analyses and bounds

171

Resource-Bounded Computations

Previously: can it be done? Now: how efficiently can we do it?

Conserve computational resources: ⇒ Time, space, other resources?

Def: L is decidable within time O(t(n)) if some TM M for L halts on all w∈∑* within O(t(|w|)) steps / time.

Def: L is decidable within space O(s(n)) if some TM M for L halts on all w∈∑* while never using more than O(s(|w|)) space / tape cells.

172

Complexity Classes Def: DTIME(t(n))=L | L is decidable

by a O(t(n))-time deterministic TM Def: NTIME(t(n))=L | L is decidable

by a O(t(n))-time non-determ. TM Def: DSPACE(s(n))=L | L decidable

by a O(s(n))-space determ. TM Def: NSPACE(s(n))=L | L decidable

by a O(s(n))-space non-determ. TM

Note: Times depends on #tapes in TM.

173

Note: For multi-tape TM’s, input tape space does not count in s(n).

Theorem: Space is #tapes-independent.

Theorem: If s(n) < s’(n) ∀n>1 then: DSPACE(s(n)) ⊆ DSPACE(s’(n)) NSPACE(s(n)) ⊆ NSPACE(s’(n))

Theorem: If t(n) < t’(n) ∀n>1 then: DTIME(t(n)) ⊆ DTIME(t’(n)) NTIME(t(n)) ⊆ NTIME(t’(n))

Ex: NTIME(n) ⊆ NTIME(n2) Ex: DSPACE(log n) ⊆ DSPACE(n)

174

Ex: L1=0n1n | n>0:

For 1-tape TM’s: • L1 ∈ DTIME(n2) • L1 ∈ DSPACE(n) • L1 ∈ DTIME(n log n)

For 2-tape TM’s: • L1 ∈ DTIME(n) • L1 ∈ DSPACE(log n)

Ex: L2=Σ*

• L2 ∈ DTIME(n) • L2 ∈ DSPACE(1)

Ex: L3=w$w | w in Σ*

• L3 ∈ DTIME(n2) • L3 ∈ DSPACE(n) • L3 ∈ DSPACE(log n)

175

Special Classes Def: P = ∪ DTIME(nk) K

P ≡ deterministic polynomial time

Def: NP = ∪ NTIME(nk) K

NP ≡ non-deterministic polynom. time

Def: PSPACE = ∪ DSPACE(nk) K

= ∪ NSPACE(nk) K

PSPACE ≡ polynomial space

Def: EXPTIME = ∪ DTIME(2nk) K

EXPTIME ≡ exponential time

176

Time Hierarchy

Theorem: for any t(n)>0, there exists a decidable language L∉DTIME(t(n)).

(note: t(n) must computable & everywhere defined)

⇒ No time complexity class contains all decidable languages!

⇒ There are decidable languages that take arbitrarily long times to decide!

Proof: (diagonalization)

Consider lexicographic orders: wi for strings, Mi for TM’s, i>0

Define L=wi | Mi does not accept wi within t(|wi|) time/moves

177

Claim: L is decidable (why?)

Q: is L∈DTIME(t(n)) ?

Assume yes, i.e., L=L(Mi0) for some

t(n)-time-bounded TM Mi0

Consider whether wi0∈L:

wi0∈L ⇒ Mi0

(wi0) halts in t(|wi0

|) time ⇒ wi0

∉L by L’s definition.

wi0∉L ⇒ Mi0

(wi0) does not halts within

t(|wi0|) time ⇒ wi0

∈L by L’s definition.

So, (wi0∈L) ⇔ (wi0

∉L), a contradiction!

⇒ L≠L(Mi0) for any t(n)-time TM Mi0

⇒ L∉DTIME(t(n))

178

Space Hierarchy

Theorem: for any s(n)>0, there exists a decidable language L∉DSPACE(s(n)).

(note: s(n) must computable & everywhere defined)

⇒ No space complexity class contains all decidable languages!

⇒ There are decidable languages that take arbitrarily large space to decide!


Consider lexicographic orders: wi for strings, Mi for TM’s, i>0

Define L=wi | Mi does not accept wi within s(|wi|) space

179

Claim: L is decidable (Q: how can we detect space-bounded ∞ loops?)

Q: is L∈DSAPCE(s(n)) ?

Assume yes, i.e., L=L(Mi0) for some

s(n)-space-bounded TM Mi0

Consider whether wi0∈L:

wi0∈L ⇒ Mi0

(wi0) halts in t(|wi0

|) space ⇒ wi0

∉L by L’s definition.

wi0∉L ⇒ Mi0

(wi0) does not halt within

t(|wi0|) space ⇒ wi0

∈L by definition.

So, (wi0∈L) ⇔ (wi0

∉L), a contradiction!

⇒ L≠L(Mi0) for any s(n)-space TM Mi0

⇒ L∉DSPACE(s(n))

180

Dense Space Hierarchy

Q: How much additional space does it take to recognize more languages?

A: Very little more!

Theorem: Given bounds s1 and s2 such that Lim s1(n) / s2(n)=0 as n→∞, i.e., s1(n) = o(s2(n)), ∃ a decidable language L such that L∈DSPACE(s2(n)) but L∉DSPACE(s1(n)).

(note: s2(n) must computable within s2(n) space)

⇒ DSPACE(o(s(n))) ⊄ DSPACE(s(n))

Ex: DSPACE(n) ⊄ DSPACE(n log n) DSPACE(n2) ⊄ DSPACE(n2.001) DSPACE(nx) ⊄ DSPACE(ny) ∀x<y

181


Consider an encoding Mi of TM’s (note: this represents each R.E. language ∞ times) Construct new TM M’ that for each w: 1) Mark s2(n) cells on tape; if the rest of this

Simulation tries to exit marked calls, reject; 2) If input w is not a valid TM encoding, reject; 3) If w=Mi, simulate TM Mi on w, subject to:

If Mi accepts, reject; If Mi rejects, accept; If Mi runs for longer than 2s

2(|w|), reject; (since Mi is then in an ∞ loop!)

⇒ M’ always halts within s2(n) space. ⇒ L∈DSPACE(s2(n)) Claim: L(M’) is not in DSPACE(s1(n))

182

Assume: L(M’) is in DSPACE(s1(n)), where s1(n) = o(s2(n))

⇒ ∃ s1(n)-space M” that decides L(M’) i.e., L(M”) = L(M’)

Note: s1(n) < s2(n) ∀n>n0

Consider what happens when you run M’ on w=M”$0n

0, i.e., analyze the case of M’ running on “padded” M”$0n

0.

⇒ the simulation of M’ on M”$0n0 in

step (3) will run to completion

But M’ will do the opposite of M”, so L(M’) can not be the same as L(M”), a contradiction!

⇒ L(M’) ∉ DSPACE(s1(n))

183

Dense Time Hierarchy

Q: How much additional time does it take to recognize more languages?

A: At most a logarithmic factor more!

Theorem: Given t1 and t2 such that t1(n)⋅log(t1(n)) = o(t2(n)), ∃ decidable language L such that L∈DTIME(t2(n)) but L∉DTIME(t1(n)).

(note: t2(n) must computable within t2(n) time)

⇒ DTIME(o(t(n) / log t(n))) ⊄ DTIME(t(n))

Ex: DTIME(n) ⊄ DTIME (n log2n) DTIME(2n) ⊄ DTIME(n22n) DTIME(nx) ⊄ DTIME(ny) ∀x<y

184

Time/Space Relationships

Thm: DTIME(f(n)) ⊆ DSPACE(f(n)) Thm: DSPACE(f(n)) ⊆ DTIME(cf(n)) for some c depending on the language. Thm: NTIME(f(n)) ⊆ DTIME(cf(n)) for some c depending on the language. Thm: NSPACE(f(n)) ⊆ DSPACE(f2(n)) Thm: NSPACE(nr) ⊆ DSPACE(nr+ε) for all r>0, ε>0.

185

NP-Completeness

• Tractability

• Polynomial time

• Computation vs. verification

• Non-determinism

• Encodings

• Transformation & reducibilities

• P vs. NP

• "completeness"

186

A problem L is NP-hard if:

1) all problems in NP reduce to L in polynomial time.

A problem L is NP-complete if:

1) L is NP-hard; and 2) L is in NP.

• One NPC problem is in P⇒P=NP

P co-NPNP

NPC

Open question: is P=NP ?

187

Satisfiability

SAT: is a given n-variable boolean formula (in CNF) satisfiable?

CNF (Conjunctive Normal Form): i.e., product-of-sums "satisfiable" ⇒ can be made "true"

Ex: (x+y)(x_ +z) is satisfiable

(x+z)(x_ )(z

_ ) is not satisfiable

3-SAT: is a given n-var boolean formula (in 3-CNF) satisfiable?

3-CNF: three literals per clause

Ex: (x1+x5+x7)(x3+x_ 4+x

_ 5)

188

Cook's Theorem

Thm: SAT is NP-complete [Cook 1971]

Pf idea: given a non-deterministic polynomial-time TM M and input w, construct a CNF formula that is satisfiable iff M accepts w.

Use variables: • q[i,k] ⇒ at step i, M is in state k • h[i,k] ⇒ at step i, read-write head scans tape cell k • s[i,j,k] ⇒ at step i, tape cell j contains symbol Σk

M always halts in polynomial time ⇒ # of variables is polynomial

189

Clauses for necessary restrictions: • At each time i: M is in exactly 1 state r/w head scans exactly 1 cell all cells contain exactly 1 symb • Time 0 ⇒ initial state • Time P(n) ⇒ final state • Transitions from time i to time i+1 obey M's transition function

Resulting formula is satisfiable iff M accepts w.

Thm: 3-SAT is NP-complete

Pf idea: convert each long clause to an equivalent set of short ones: (x+y+z+u+v+w)

⇒(x+y+ )(_

+z+ )(_

+u+ )(_

+v+w)

190

Q: is 1-SAT NP-complete? Q: is 2-SAT NP-complete?

191

COLORABILITY: given a graph G and integer k, is G k-colorable?

(different colors for adjacent nodes) Ex:

⇒

Thm: 3-COLORABILITY is NPC Proof: reduction from 3-SAT

(x+y+z) ⇒ z

T

x

y

gadget is 3-colorable ⇔ x+y+z is true

T

F

x

x ∀x

192

Ex: (x+y+z)(x_ +y

_ +z)(x

_ +y+z

_ )

z

F

z

x

T

y

x

y

193

Ex (cont.): a 3-coloring:

z

F

x

T

x

y

z

y

Solution ⇒ x=true, y=false, z=false

194

Thm: 3-COLORABILITY is NPC for graphs with max degree 4.

Pf: degree-reduction "gadget":

a) max degree 4 b) 3-colorable but not 2-colorable c) all corners get same color

"Super"-gadgets:

Use these "fanout" components to reduce node degrees to 4 or less

195

Ex:

G:

G':

G is 3-colorable ⇔ G' is 3-colorable

196

Q: is 3-COLORABILITY NPC for graphs with max degree 3?

197

Thm: 3-COLORABILITY is NPC for planar graphs.

Pf: planarity-preserving "gadget":

a) planar and 3-colorable b) Opposite Corners get same color c) "independence" of pairs of OC's

Use gadget to avoid edge crossings:

a b

x

y

a b

x

y

198

Ex:

G:

1

2 3

45

G': 1

2

3

54

G is 3-colorable ⇔ G' is 3-colorable

cs302 slides 2006 v22 - University of Virginia School of ...robins/cs302/cs302_slides... · 114...

Documents

Transcript of cs302 slides 2006 v22 - University of Virginia School of ...robins/cs302/cs302_slides... · 114...