Universal Turing Machines and Undecidability

Universal Turing Machines and Undecidability

Nabil Mustafa

Computational Complexity

1 / 174

Turing machine recap

A Turing machine TM is a tuple M = (Γ,Q, δ) where

Γ: set of symbols that TM ’s tapes can contain.

Q: possible states TM can be in.I qstart : the TM starts in this stateI qhalt : the TM halts when this state is reached

Storage for TM :

A special register stores the current state.

1 input tape, 1 output tape and 1 work tape

2 / 174





Storage for TM :



3 / 174





Storage for TM :



4 / 174





Storage for TM :



5 / 174

Robustness of TMMany details of our TM quite arbitrary

Does restricting the alphabet Γ to {0, 1,�,B} matter?

No!If function f is computable by a TM using alphabet Γ, then it iscomputable in time 4 log |Γ| · T (n) by a TM using the alphabet{0, 1,�,B}.

Does using more work tapes make everything much faster?

No!If function f is computable by a TM using k tapes, then it is computablein time 5kT (n)2 by a TM using a single work tape.

Church-Turing HypothesisEvery physically realizable computation device can be simulated by a TM .

6 / 174

Robustness of TMMany details of our TM quite arbitraryDoes restricting the alphabet Γ to {0, 1,�,B} matter?





7 / 174






8 / 174






9 / 174






10 / 174






11 / 174

Alphabet size doesn’t matterClaimIf function f is computable by a TM M using alphabet Γ, then it iscomputable in time 4 log |Γ| · T (n) by a TM M ′ using the alphabet{0, 1,�,B}.

Have to decide:

How to represent each symbol of Γ using {0, 1,�,B}.I Use binary encoding. Each symbol encoded by log |Γ| bits.

How to simulate each step of M.

I Read the encoded symbol via log |Γ| bits. Problem: remember the bitsread.

I Modify the transition table appropriately

I Lookup the (remembered) read bits in the (modified) table

12 / 174


Have to decide:

How to represent each symbol of Γ using {0, 1,�,B}.

I Use binary encoding. Each symbol encoded by log |Γ| bits.





13 / 174


Have to decide:

How to represent each symbol of Γ using {0, 1,�,B}.

I Use binary encoding. Each symbol encoded by log |Γ| bits.





14 / 174


Have to decide:

How to represent each symbol of Γ using {0, 1,�,B}.I Use binary encoding. Each symbol encoded by

log |Γ|

bits.





15 / 174


Have to decide:






16 / 174


Have to decide:



I Read the encoded symbol via log |Γ| bits.

Problem: remember the bitsread.



17 / 174


Have to decide:






18 / 174


Have to decide:






19 / 174


Have to decide:





I Lookup the (remembered) read bits in the (modified) table20 / 174

Remembering bits read

Problem: How to remember the bits read?

y bn x i 3jq′q i Rj

QQ Γ {L,R,S}Γ

21 / 174



q′q i Rj y bn x j 3j

QQ Γ {L,R,S}Γ

22 / 174



q′q i Rj y bn x i 3j

1 0y 0 0 b1

QQ Γ {L,R,S}Γ

23 / 174



y bn x i 3j

1 0y 0 0 b1

q′q i Rj

q′q 01001 R01010

QQ Γ {L,R,S}Γ

24 / 174



y bn x i 3j

1 0y 0 0 b1

q′q i Rj

q0

q R0 0q0

q R0 0

QQ Γ {L,R,S}Γ

25 / 174



y bn x i 3j

1 0y 0 0 b1

q′q i Rj

q0

q R0 0

q0

R1 1q01

QQ Γ {L,R,S}Γ

26 / 174



y bn x i 3j

1 0y 0 0 b1

q′q i Rj

q0

q R0 0

q0

R1 1q01

q010

R0 0q01

QQ Γ {L,R,S}Γ

27 / 174



y bn x i 3j

1 0y 0 0 b1

q′q i Rj

q0

q R0 0

q0

R1 1q01

q010

R0 0q01

q0100 R0 0q

010

QQ Γ {L,R,S}Γ

28 / 174



y bn x i 3j

1 0y 0 0 b1

q′q i Rj

q0

q R0 0

q0

R1 1q01

q010

R0 0q01

q0100 R0 0q

010

q′ R1 1q0100

QQ Γ {L,R,S}Γ

29 / 174

Lets say that i → 01001 and j → 01010. Then,

Q Γ Q Γ {L, S ,R}q 0 q0 0 R

q0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R

Note: need to move back 2 log |Γ| steps as well!

30 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 R

q01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


31 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 R

q010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


32 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 R

q0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


33 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 L

qb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


34 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − L

qb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


35 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − L

qb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


36 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − L

qb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


37 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − S

qw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


38 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 R

qw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


39 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 R

qw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


40 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 R

qw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R


41 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 R

qw5.01001 − q′ 0 R


42 / 174


Q Γ Q Γ {L, S ,R}q 0 q0 0 Rq0 1 q01 1 Rq01 0 q010 0 Rq010 0 q0100 0 Rq0100 1 qb1.01001 1 Lqb1.01001 − qb2.01001 − Lqb2.01001 − qb3.01001 − Lqb3.01001 − qb4.01001 − Lqb4.01001 − qw1.01001 − Sqw1.01001 − qw2.01001 0 Rqw2.01001 − qw3.01001 1 Rqw3.01001 − qw4.01001 0 Rqw4.01001 − qw5.01001 1 Rqw5.01001 − q′ 0 R

Note: need to move back 2 log |Γ| steps as well!43 / 174

Total time required

What is the number of states in M ′?

Each line in the transition table of M creates 4 log |Γ| states.I log |Γ| steps to read the symbol encoding.I log |Γ| steps to go back to the start of the symbol encoding.I log |Γ| steps to write the new symbol encoding.I 2 log |Γ| in case the tape-head moves to the left.

What is the size of the old transition table?I |Q| · |Γ|

Total number of new states required: |Q||Γ|4 log |Γ|.

Each step of M requires 5 log |Γ| steps of M ′.Total time taken: 5 log |Γ|T (n) steps.

44 / 174

Total time required

What is the number of states in M ′?Each line in the transition table of M creates 4 log |Γ| states.

I log |Γ| steps to read the symbol encoding.I log |Γ| steps to go back to the start of the symbol encoding.I log |Γ| steps to write the new symbol encoding.I 2 log |Γ| in case the tape-head moves to the left.




45 / 174

Total time required


I log |Γ| steps to read the symbol encoding.

I log |Γ| steps to go back to the start of the symbol encoding.I log |Γ| steps to write the new symbol encoding.I 2 log |Γ| in case the tape-head moves to the left.




46 / 174

Total time required


I log |Γ| steps to read the symbol encoding.I log |Γ| steps to go back to the start of the symbol encoding.

I log |Γ| steps to write the new symbol encoding.I 2 log |Γ| in case the tape-head moves to the left.




47 / 174

Total time required


I log |Γ| steps to read the symbol encoding.I log |Γ| steps to go back to the start of the symbol encoding.I log |Γ| steps to write the new symbol encoding.

I 2 log |Γ| in case the tape-head moves to the left.




48 / 174

Total time required






49 / 174

Total time required



What is the size of the old transition table?

I |Q| · |Γ|



50 / 174

Total time required






51 / 174

Total time required






52 / 174

Total time required





Each step of M requires 5 log |Γ| steps of M ′.

Total time taken: 5 log |Γ|T (n) steps.

53 / 174

Total time required






54 / 174

More work tapes do not make much difference

ClaimIf a function f is computable by a TM M using k tapes, then itscomputable in time 5kT (n)2 by a TM using a single work tape.

ProofLets consider that a TM M computes the function f and has k tapes(plus additional input and output tapes)Next we consider a single work tape Turing machine MM encodes the k tapes of M on a single tape by using locations1, k + 1, 2k + 1, . . . to encode the first tape, locations2, k + 2, 2k + 2, . . . to encode the second tape etc.For every symbol a in M’s alphabet, M will contain both the symbol aand the symbol a. In the encoding of each tape, exactly one symbolwill be of the ‘ˆ type’, indicating that the corresponding head of M ispositioned in that location.

55 / 174


ClaimIf a function f is computable by a TM M using k tapes, then itscomputable in time 5kT (n)2 by a TM using a single work tape.

ProofLets consider that a TM M computes the function f and has k tapes(plus additional input and output tapes)Next we consider a single work tape Turing machine MM encodes the k tapes of M on a single tape by using locations1, k + 1, 2k + 1, . . . to encode the first tape, locations2, k + 2, 2k + 2, . . . to encode the second tape etc.For every symbol a in M’s alphabet, M will contain both the symbol aand the symbol a. In the encoding of each tape, exactly one symbolwill be of the ‘ˆ type’, indicating that the corresponding head of M ispositioned in that location.

56 / 174


Proof Cont’dTo simulate one step of M, the machine makes two passes of its worktape: first it traverses the tape in the left-to-right direction andrecords (via additional states) the k symbols of the form aThen M uses M’s transition function to determine the new state,symbols, and head movements and sweeps the tape back in theright-to-left direction to update the encoding accordingly.M will never reach more than location kT (n) of its work tape,meaning that for each of the at most T (n) steps of M, M performs atmost 5kT (n) work (sweeping back and forth requires about 2T (n)steps, and some additional steps needed for updating head movementand book keeping).

57 / 174

Universal Turing Machines

So far, constructed Turing machines for a specific task.

I a primitive computer that only performs one task.

Not very useful, since there are many things we want to do.

Question: does there exist a TM that does many things?

I Given a ‘program’ P and an input x for P, run P on x .

I Note that a program is just an algorithm for doing something, i.e., its aTuring machine.

So . . . given any TM M and x , does there exist a TM UTM that willrun M on x

I How to give M to UTM? Since M is a machine.

I How can there be a fixed machine UTM that will run any other TM ?

58 / 174











59 / 174











60 / 174











61 / 174











62 / 174











63 / 174











64 / 174











65 / 174











66 / 174

What is a Turing machine really

Observe that the TM s are not all that different

I Can assume all have 1 input tape, 1 output tape, 1 work tape.

I Can assume Γ = {0, 1,�,B}

The only thing different between different Turing machines is thetransition table

But the transition table can be written up as a string of bits!

Mα: TM represented by string of bits α

bMc ∈ {0, 1}∗: string of bits representing M

So UTM takes α and x , and simulates Mα on x .

67 / 174










68 / 174










69 / 174










70 / 174










71 / 174










72 / 174










73 / 174










74 / 174

The Algorithm for TM

How can a fixed Turing machine UTM run any other TM ?

Basic idea: Running a TM is also just an algorithm!

For example, we ran the TM for PAL(x) by hand.

We just have to follow some rules, i.e., there is a simple algorithm forrunning a TM M:

I Read the symbol of the input x at the tape-head.I Scan the transition table to find a rule which applies.I Apply this rule to write new symbol and move the tape head.I Store the new state of M

So we just have to design a TM that can run the above algorithm.

75 / 174








76 / 174








77 / 174








78 / 174






I Read the symbol of the input x at the tape-head.

I Scan the transition table to find a rule which applies.I Apply this rule to write new symbol and move the tape head.I Store the new state of M


79 / 174






I Read the symbol of the input x at the tape-head.I Scan the transition table to find a rule which applies.

I Apply this rule to write new symbol and move the tape head.I Store the new state of M


80 / 174






I Read the symbol of the input x at the tape-head.I Scan the transition table to find a rule which applies.I Apply this rule to write new symbol and move the tape head.

I Store the new state of M


81 / 174








82 / 174








83 / 174

Universal Turing Machine

Storage:T1: input tape containing α, x .

T2: output tapeThree work tapes:

I T3 contains the current state of Mα

I T4 is used the same way as Mα’s work-tape.I T5 contains the transition function.

Transition function of UTM:Scan T5 to find a match of the state with T3

Scan T5 to find a match of symbol with T1 and T4.Copy new state to T3

Copy new symbol to T4 and move heads of T1 and T3.

84 / 174


Storage:T1: input tape containing α, x .T2: output tape

Three work tapes:I T3 contains the current state of Mα





85 / 174


Storage:T1: input tape containing α, x .T2: output tapeThree work tapes:






86 / 174








87 / 174




I T4 is used the same way as Mα’s work-tape.

I T5 contains the transition function.




88 / 174








89 / 174





Transition function of UTM:

Scan T5 to find a match of the state with T3



90 / 174








91 / 174






Scan T5 to find a match of symbol with T1 and T4.

Copy new state to T3


92 / 174








93 / 174








94 / 174

Simulating Mα

95 / 174

A Quote

“If we were so clever that we could understand our brain, our brain wouldbe so complex that we couldn’t understand it.”

96 / 174

Uncomputable functions

f is computable by a TM M if M(x) = f (x) ∀x

It would seem that any function f : {0, 1}∗ → {0, 1} can be computedby a Turing machine.

I We don’t care how much time is takenI So just spend as much time till the computation of the function

finishes.

Uncomputable functions existThere exists UC : {0, 1}∗ → {0, 1} not computable by any TM .

HALT(P, x): Given program P and x , does P halt on x?

Halting functionHALT is not computable by any TM .

97 / 174


f is computable by a TM M if M(x) = f (x) ∀xIt would seem that any function f : {0, 1}∗ → {0, 1} can be computedby a Turing machine.


finishes.




98 / 174



I We don’t care how much time is taken

I So just spend as much time till the computation of the functionfinishes.




99 / 174




finishes.




100 / 174




finishes.




101 / 174




finishes.




102 / 174




finishes.




103 / 174

A Detour

Let S = {s1, . . . , sn} and P = {p1, . . . , pm} be two finite sets

Question: When does a bijection between S and P exist?I Answer: ∃ a bijection f : S → P iff |S | = |P|

S P

What if S and P are infinite sets?

Any bijection between S = {1, 2, . . .} and P = {2, 3, . . .}?I f : S → P defined as: f (a) = a + 1.I 1↔ 2, 2↔ 3, 3↔ 4 and so on.

104 / 174

A Detour


Question: When does a bijection between S and P exist?

I Answer: ∃ a bijection f : S → P iff |S | = |P|S P



105 / 174

A Detour



S P



106 / 174

A Detour



S P



107 / 174

A Detour



S P



108 / 174

A Detour



S P


Any bijection between S = {1, 2, . . .} and P = {2, 3, . . .}?

I f : S → P defined as: f (a) = a + 1.I 1↔ 2, 2↔ 3, 3↔ 4 and so on.

109 / 174

A Detour



S P


Any bijection between S = {1, 2, . . .} and P = {2, 3, . . .}?I f : S → P defined as: f (a) = a + 1.

I 1↔ 2, 2↔ 3, 3↔ 4 and so on.

110 / 174

A Detour



S P



111 / 174

A Detour

Bijection between S = {1, 2, 3, . . .} and P = {2, 4, 6, . . .}?

I f : S → P defined as: f (a) = 2a.

I 1↔ 2, 2↔ 4, 3↔ 6 and so on.

I Looks weird, but make sure you completely understand this.

TheoremA bijection exists between natural numbers N and rationals Q.

TheoremNo bijection exists between natural numbers N and reals R.

Question: What about an infinite set, and its power set?

112 / 174

A Detour

Bijection between S = {1, 2, 3, . . .} and P = {2, 4, 6, . . .}?I f : S → P defined as: f (a) = 2a.

I 1↔ 2, 2↔ 4, 3↔ 6 and so on.





113 / 174

A Detour


I 1↔ 2, 2↔ 4, 3↔ 6 and so on.





114 / 174

A Detour


I 1↔ 2, 2↔ 4, 3↔ 6 and so on.





115 / 174

A Detour


I 1↔ 2, 2↔ 4, 3↔ 6 and so on.





116 / 174

A Detour


I 1↔ 2, 2↔ 4, 3↔ 6 and so on.





117 / 174

A Detour


I 1↔ 2, 2↔ 4, 3↔ 6 and so on.





118 / 174

Bijection between a set and its powerset

ClaimThere does not exist bijection between infinite set and its power set

ProofConsider an infinite set S and its corresponding power set PWe show a contradiction if ∃ bijection f : S → PLet i be any element of S and A be any element of PIf bijection exists, every A ∈ P is mapped to by some i ∈ SConsider the following set A′ ∈ P :

A′ = {i ∈ S : i /∈ f (i)}

A′: All i which are not present in the set f (i)

119 / 174



ProofConsider an infinite set S and its corresponding power set P

We show a contradiction if ∃ bijection f : S → PLet i be any element of S and A be any element of PIf bijection exists, every A ∈ P is mapped to by some i ∈ SConsider the following set A′ ∈ P :

A′ = {i ∈ S : i /∈ f (i)}


120 / 174



ProofConsider an infinite set S and its corresponding power set PWe show a contradiction if ∃ bijection f : S → P

Let i be any element of S and A be any element of PIf bijection exists, every A ∈ P is mapped to by some i ∈ SConsider the following set A′ ∈ P :

A′ = {i ∈ S : i /∈ f (i)}


121 / 174



ProofConsider an infinite set S and its corresponding power set PWe show a contradiction if ∃ bijection f : S → PLet i be any element of S and A be any element of P

If bijection exists, every A ∈ P is mapped to by some i ∈ SConsider the following set A′ ∈ P :

A′ = {i ∈ S : i /∈ f (i)}


122 / 174



ProofConsider an infinite set S and its corresponding power set PWe show a contradiction if ∃ bijection f : S → PLet i be any element of S and A be any element of PIf bijection exists, every A ∈ P is mapped to by some i ∈ S

Consider the following set A′ ∈ P :

A′ = {i ∈ S : i /∈ f (i)}


123 / 174



ProofConsider an infinite set S and its corresponding power set PWe show a contradiction if ∃ bijection f : S → PLet i be any element of S and A be any element of PIf bijection exists, every A ∈ P is mapped to by some i ∈ SConsider the following set A′ ∈ P :

A′ = {i ∈ S : i /∈ f (i)}


124 / 174

Proof Cont’dNow lets look at A′ more closely

Since f is a bijection, ∃ j such that f (j) = A′

Here is the fatal question: Is j ∈ A′?If the answer is ‘Yes’:

I If j ∈ A′, then by definition of A′, j maps to a set which does notcontain j

I Contradiction!If the answer is ‘No’:

I If j /∈ A′, then j is mapping to a set (A′) which doesn’t contain j . Soby definition of A′, j should be in A′.

I Contradiction!

Important CorollaryGiven any function f : S → P , there always exists a pj ∈ P such that nosi ∈ S maps to pj .

125 / 174

Proof Cont’dNow lets look at A′ more closelySince f is a bijection, ∃ j such that f (j) = A′





I Contradiction!


126 / 174


Here is the fatal question: Is j ∈ A′?

If the answer is ‘Yes’:I If j ∈ A′, then by definition of A′, j maps to a set which does not

contain jI Contradiction!

If the answer is ‘No’:I If j /∈ A′, then j is mapping to a set (A′) which doesn’t contain j . So

by definition of A′, j should be in A′.I Contradiction!


127 / 174






I Contradiction!


128 / 174




I Contradiction!

If the answer is ‘No’:I If j /∈ A′, then j is mapping to a set (A′) which doesn’t contain j . So

by definition of A′, j should be in A′.I Contradiction!


129 / 174






I Contradiction!


130 / 174






I Contradiction!


131 / 174






I Contradiction!


132 / 174

A Quote

“It seemed unworthy of a grown man to spend his time on such trivialities,but what was I to do?” –Bertrand Russell.

133 / 174

Uncountable Sets and Uncomputable functions

Claim: Uncomputable functions exist

S : Infinite set of all binary strings, P : Power-set of S

Each si ∈ S is a binary stringI A Turing Machine can be represented by a stringI Interpret each si to represent some TM Msi

Each pj ∈ P is a set of binary stringsI A language is just a set of stringsI Interpret each pj to represent a language, say, Lj

Now, each TM Msi decides some language Lj

I This defines f (·): f (si ) = pj , where Msi decides Lj

By previous corollary, ∃ pk ∈ P s.t. no si maps to pk

I Then there is no TM that decides the language Lk !

134 / 174










135 / 174




Each si ∈ S is a binary string

I A Turing Machine can be represented by a stringI Interpret each si to represent some TM Msi






136 / 174




Each si ∈ S is a binary stringI A Turing Machine can be represented by a string

I Interpret each si to represent some TM Msi






137 / 174










138 / 174





Each pj ∈ P is a set of binary strings

I A language is just a set of stringsI Interpret each pj to represent a language, say, Lj





139 / 174





Each pj ∈ P is a set of binary stringsI A language is just a set of strings

I Interpret each pj to represent a language, say, Lj





140 / 174










141 / 174










142 / 174










143 / 174










144 / 174










145 / 174

HALTHalting functionHALT is not computable by any TM

Proof.Assume a TM MH computes HALT

I MH takes as input a TM M (as a string) and M’s input xI MH(bMc; x) = 1 if M halts on input x , 0 otherwiseI Note that MH always halts

Consider the case when x = bMc: MH(bMc; bMc)If MH exists, then there also exists a TM D s.t.

I D takes as input a TM MI D works the same way as MH . However, when MH is about to halt

with a ‘yes’, D goes into an infinite loop.I When MH would halt with a ‘No’, D also halts.

146 / 174







147 / 174



I MH takes as input a TM M (as a string) and M’s input x

I MH(bMc; x) = 1 if M halts on input x , 0 otherwiseI Note that MH always halts




148 / 174



I MH takes as input a TM M (as a string) and M’s input xI MH(bMc; x) = 1 if M halts on input x , 0 otherwise

I Note that MH always halts




149 / 174







150 / 174




Consider the case when x = bMc: MH(bMc; bMc)

If MH exists, then there also exists a TM D s.t.I D takes as input a TM MI D works the same way as MH . However, when MH is about to halt


151 / 174







152 / 174





I D takes as input a TM M

I D works the same way as MH . However, when MH is about to haltwith a ‘yes’, D goes into an infinite loop.

I When MH would halt with a ‘No’, D also halts.

153 / 174






with a ‘yes’, D goes into an infinite loop.

I When MH would halt with a ‘No’, D also halts.

154 / 174







155 / 174

HALT

Proof Cont’dSo far: if MH exists, then there also exists D:

I D(bMc) does not halt if M halts on input bMcI D(bMc) halts if M does not halt on input bMc

Fatal question: Does D(bDc) halt?

If D(bDc) halts:I D(bDc) halts =⇒ D does not halt on input bDcI Contradiction!

If D(bDc) does not halt:I D(bDc) does not halt =⇒ D halts on input bDcI Contradiction!

This type of argument is called a diagonalization argument.

156 / 174

HALT


I D(bMc) does not halt if M halts on input bMc

I D(bMc) halts if M does not halt on input bMc





157 / 174

HALT







158 / 174

HALT







159 / 174

HALT




If D(bDc) halts:

I D(bDc) halts =⇒ D does not halt on input bDcI Contradiction!



160 / 174

HALT




If D(bDc) halts:I D(bDc) halts =⇒ D does not halt on input bDc

I Contradiction!



161 / 174

HALT







162 / 174

HALT





If D(bDc) does not halt:

I D(bDc) does not halt =⇒ D halts on input bDcI Contradiction!


163 / 174

HALT





If D(bDc) does not halt:I D(bDc) does not halt =⇒ D halts on input bDc

I Contradiction!


164 / 174

HALT







165 / 174

HALT







166 / 174

The Halting problem spawns more problems

Using reductions, can show many problems to be undecidable

Goal: show that the problem A is undecidable

Establish that, if there were an algorithm for problem A, then therewould be an algorithm for HALT

HALT doesn’t have an algorithm, hence so doesn’t A

Claim: This is undecidable: {M: M halts on all inputs}

Reduce HALT to this problem

Given M; x construct the following machine M ′:M ′(y) = M(x) if y = x , M ′(y) = 0 otherwise

M ′ halts on all inputs if and only if M halts on x

167 / 174


Using reductions, can show many problems to be undecidableGoal: show that the problem A is undecidable







168 / 174









169 / 174









170 / 174









171 / 174









172 / 174









173 / 174









174 / 174

Universal Turing Machines and Undecidability

Documents

Transcript of Universal Turing Machines and Undecidability