1

PCP Theorem Proof: Powering StagePart II

Irit Dinur

Presentation by Boris Danilovich and Jonathan Heimann

2

The Story So FarInput:

• G=(V,E) an (n,d,λ)-expander with alphabet Σ.

• A set of constraints CΣXΣ.

Construction of new G’=(V’,E’):

• V’ = V

• E’ – Any A.S.R.W from a to b => (a,b)E’.

w(a,b) = P[A.S.R.W begins on a and ends on b]

• Σ’ - All binary strings up to length ddtt

• C’C’ = { = {(ε1, ε2)Σ’XΣ’ | ε1 and ε2 agree about neighbors,

derived plurality assignment satisfies C}

3

Back to Basics

A.S.R.W – After Stopping Random Walk

B.S.R.W – Before Stopping Random Walk

Let G=(V,E) be a regular graph.

A.S.R.WB.S.R.W

1) Pick a random vertex aV to start at.

1) Starting from vertex v V.

2) Take a step along a random edge out of the current vertex

2) Stop with probability 1/t.

3) Decide to stop with probability 1/t.

If you didn’t stop, go to step #2.

3) Take a step along a random edge out of the current vertex.

4) Name the final vertex b.4) Go to step #2.

4

Back to Basics

σ:V→Σ – σ(v) = the “popular” σ’(w)v, for w’s that are neighbors of v.

σ’:V’→Σ’ – An assignment which minimizes UNSAT(G’).

F’E is the set of all edges in E which are not satisfied by assignment σ.

FF’E is produced from F’ by removing edges until |F|/|E| = UNSAT(G).

Make an A.S.R.W from a to b in a graph G.

An edge (a,b)E’ is bad if it contains a step u→v for which:

• For every wV a t-neighbor of a and b in graph G, ‘(a)w = ‘(b)w

• distG(u,a) t AND distG(v,b) t

• The assignment ) ‘(a)u,‘(b)v) is a violation of the constraint on (u,v).

5

Back to Basics

Make an A.S.R.W from a to b in a graph G.

A step u→v within such a walk is called faulty if:

• (u,v)F

• distG(u,a) t AND ‘(a)u = (u)

• distG(v,b) t AND ‘(b)v = (v)

N = # number of faulty steps in an A.S.R.W.

6

Back to Basics

UNSAT(G’)

P[A bad edge in G’ will be generated by an A.S.R.W in G]

P[The chosen A.S.R.W walk in G contains a faulty step]

P]N>0]

E[N]2

E[N2]

=

=

≥

≥

t

O(1)min(UNSAT(G),1/

t)

≥?

Second moment method

7

Setting immediate goals

We want to prove: P[N>0] ≥ = [t/O(1)] ּ[|F|/|E|]

We will prove:

1) E[N]2 ([t/O(1)] ּ[|F|/|E|])2

2) E[N2] [t/O(1)] ּ[|F|/|E|]

E[N]2

E[N2]

8

Lemma 1: E[N] [t/4|∑|2]ּ[|F|/|E|]

E[N] = E[#faulty steps] == E[over all uv E : #faulty uv steps] =

= E[over all uv F : #fauly uv steps] [#possible steps F] ּminuvF{E[#faulty uv stepF]}=

= 2|F| ּminuvF {E[#faulty uv step F]}If we prove that:

minuvF {E[#faulty uv step F]} [t/(8|∑|2) ּ[(1/|E|)

We get that: E[N] [t/4|∑|2] ּ[|F|/|E|] as required

זה קבוע...

לפי הגדר

ה

Remember we talk

about A.S.R.W

9

[Note: if we prove this bound to be a lower bound of every {u→vF faulty step}, we are done]

For any step uvF:E[#faulty uv steps] =A fact about expectancy: E[A] = ∑ E[A|B(k)] ּPr[B(k)]

=∑E[#faulty uv steps |exactly k uv steps] ּPr[exactly k uv steps]Remember that in an specific A.S.R.W, if a u→v step is faulty once, it

will is faulty each time in that walk.E[#faulty uv steps | exactly k uv steps] =

= k * Pr[uv steps are faulty | exactly k uv steps] We’ll set this value in and receive:=∑k*Pr[uv steps are faulty |exactly k uv steps] ּPr[exactly k uv steps]

Lemma 1:1) E[N] [t/4|∑|2] ּ[|F|/|E|]

minuvF{E[#faulty uv step F]} [t/(8|∑|2) ּ[(1/|E|)

1

1

k

10

Overall, we got (from previous slide):E[#faulty uv steps] ==∑k * Pr[uv steps are faulty|exactly k uv steps] ּPr[exactly k uv steps]

We’ll claim and soon prove:P[uv steps are faulty | exactly k uv steps] 1/(4|∑|2)We’ll set this value in and receive:=∑1/(4|∑|2) * k ּPr[exactly k uv steps] =

we take a constant value out of the sum

= 1/(4|∑|2) * ∑ kּPr[exactly k uv steps] == 1/(4|∑|2) * E[#of uv steps (Not only faulty)] =

Lemma 1:1) E[N] [t/4|∑|2] ּ[|F|/|E|]

minuvF{E[#faulty uv step F]} [t/(8|∑|2) ּ[(1/|E|)

Continue – 2 out of 3

1

2

1

11

Lemma 1:1) E[N] [t/4|∑|2] ּ[|F|/|E|]

minuvF{E[#faulty uv step F]} [t/(8|∑|2) ּ[(1/|E|)

Continue – 3 out of 3= 1/(4|∑|2) * E[#of uv steps (Not only faulty)] == 1/(4|∑|2) * E[∑ [uv is the ith step]] =1) The chances of each ith step to be uv are equal2) The expected number of steps = t

= 1/(4|∑|2) * t ּ E[ [uv is the ith step]] =Regular G the probability of going over a

particular edge in an A.S.R.W = 1/|E| the probability of making that uv step = 1/(2|E|)

= 1/(4|∑|2) * t /(2|E|) == 1/(8|∑|2) * t /|E|

i

( מציין בהסתברות)

A.S.R.W

d-Regular

12

Lemma 1: 1) E[N] [t/4|∑|2] ּ[|F|/|E|]

minuvF{E[#faulty uv step F]} [t/(8|∑|2)ּ[(1/|E|)

Debt from last week:E[#uv steps] = t/(2|E|)

E[#of uv steps (Not only faulty)] =

= ∑E[#uv steps | exactly k steps] ּ Pr[exactly k steps] =

= ∑ k/(2|E|) ּ Pr[exactly k steps] =

= 1/2|E| ּ ∑ k ּ Pr[exactly k steps] =

= 1/2|E| ּ t == t/2|E|

Each step has Pr=1/2|E|

A Geometric Expectation

13

Yet more debts

Suppose we did an A.S.R.W in a graph G=(V,E), conditioned on making

exactly k u→v steps, where k≥1 is some constant and (u,v) a fixed edge

in E. Then:1. Distribution of the final vertex b ~ Distribution of a B.S.R.W from v.

2. Distribution of the initial vertex a ~ Distribution of a B.S.R.W. from u.

3. a and b are independent.

14

Proof1. Change the condition: instead of taking exactly k u→v steps, we’ll

take at

least k.

Take kth u→v step. A.S.R.W turns into B.S.R.W (choice of stopping on v

taken yet) and there are no further restrictions.

Pw = P[A B.S.R.W from v stops on vertex w] = P[b=w | Y≥k] =? P[b=w | Y=k]

Denote Y = # of u→v steps.

P[b = w | Y≥k] = =P[(b=w) ۸ (Y≥k)] P[(b=w) ۸ (Y=k)] + P[(b=w) ۸

(Y≥k+1)] P[Y≥k] P[Y≥k]

1

Pu→v = P[u→v taken in A.S.R.W]

Pu→v’ = P[u→v taken in A.S.R.W | begin on v]

15

Proof

Set these values in :

Pw = =>P[(b=w) ۸ (Y=k)] + Pw ∙ Pu→v ∙

(Pu→v’)k Pu→v ∙ (Pu→v’) k-1

P[b=w | Y=k] = = = Pw

P[(b=w) ۸ (Y=k)] Pw ∙ Pu→v ∙ (Pu→v’)k-1 ∙ (1 - Pu→v’)

P[Y=k] Pu→v ∙ (Pu→v’)k-1 ∙ (1 - Pu→v’)

P[Y≥k] = Pu→v ∙ (Pu→v’)k-1

P[(b=w) ۸ (Y≥k+1)]

P[Y≥k+1] = P[b = w | Y≥k+1] = Pw - B.S.R.W is

memoryless

=> P[(b=w) ۸ (Y≥k+1)] = P[b = w | Y≥k+1]∙P[Y≥k+1] = Pw ∙ Pu→v ∙ (Pu→v’)k

1

Finally:

P[b = w | Y≥k] = = Pw

P[(b=w) ۸ (Y=k)] + P[(b=w) ۸ (Y≥k+1)] P[Y≥k]

P[(b=w) ۸ (Y=k)] = Pw ∙ Pu→v ∙ (Pu→v’)k-1 ∙ (1 - Pu→v’)

1

16

Where are we standing?Remember: UNSAT(G’) ≥

E[N]2

E[N2]

After bounding E[N]2, we are one step away from completing the

analysis and put a constant-growth lower-bound on UNSAT(G’).

Something is rotten in the state of Denmark:

• Edges of E’ are weighted.

• G’ is too large – Ω(n2).

17

Recalling G’’In the previous lecture, G’’ was created out of G’,

by:

1. Splicing a single edge in E’, into several unweighted parallel edges in E, according to its weight. Growth: by a constant factor.

2. Removing all edges in E’ which were generated by a walk in G that was longer than B=(10ln|Σ|)t steps.

18

Doesn’t this hurt the credibility of our argument?

Note that G’’G’.

Couldn’t the removal of edges hurt the UNSAT?

EB = E’ – E’’

α = |EB|/|E’| = P[Edge eE’ generated by walk longer than B] =

P[A.S.R.W takes more than B steps] = (1-1/t)B ≤ exp(-B/t) =

exp(-10ln|Σ|t/t) = 1/(|Σ|10) - Constant!

Worst case – All edges of EB were unsatisfied

UNSAT(G’’) = (|F’|- α|E’|)/[|E’|(1 – α)] = |F’|/[|E’|(1-α)] – α|E’|/[|E’|(1-α)] =

1/(1-α)∙UNSAT(G’) – α/(1-α) = O(UNSAT(G’))

Change is by small constant factor only!

Note: might be a problem in the first iterations, if UNSAT

is too small

19

A faulty* step

Let there be a walk from a to b chosen by an A.S.R.W from a in graph G.

A step u→v within such a walk is called faulty if:

• (u,v)F

• distG(u,a) t AND ‘(a)u = (u)

• distG(v,b) t AND ‘(b)v = (v)

A faulty step:

A step is faulty* if:

• it is faulty.

• the number of steps in the entire a→b walk is at most B.

Denote the random variable N* = # of faulty* steps in an A.S.R.W

20

A faulty* step

u→v is a faulty* step in an A.S.R.W from a to b =>

(a,b) is a bad edge in G’’

This is because σ’(a) and σ’(b) agree with the popular opinion, and

according to it (u,v)F (meaning, the constraint on the edge is not

satisfied by σ).Reminder: Let there be a walk from a to b chosen by an A.S.R.W from a in graph G.An edge (a,b)E’ is bad if it contains a step u→v for which:

• For every wV a t-neighbor of a and b in graph G, ‘(a)w = ‘(b)w

• distG(u,a) t AND distG(v,b) t

• ) ‘(a)u,‘(b)v) is a violation of the constraint on (u,v).

21

Bounding UNSAT(G’’)

UNSAT(G’’)

P[A bad edge in G’’ will be chosen by an A.S.R.W in G]

P[The chosen A.S.R.W walk in G contains a faulty* step]

P]N*>0]

E[N*]2

E[N*2]

=

=

≥

≥

t

O(1)min(UNSAT(G),1/

t)

≥?

Second Moment Method

Each walk containing a faulty* step generates a bad edge.

22

If we could just prove this…

Lemma 1: E[N*] ≥ 8|Σ|2 |E| t |F|

|F|/|E| = UNSAT(G)

Lemma 2: E[(N*)2] ≤ O(1)∙t∙|E||F|

23

ProofLemma 1: E[N*] ≥ 8|Σ|2 |E|

t |F|

S – # of steps in walk

E[N*] = E[N ∙ 1{S≤B}] = E[N ∙ (1 – 1{S>B})] = E[N] – E[N ∙ 1{S>B}]

≥ – E[N ∙ 1{S>B}]

An indicator

Lower bound on E[N], shown before.

4|Σ|2 |E| t |F|

24

ProofLemma 1: E[N*] ≥ 8|Σ|2 |E|

t |F|

We want to find an upper-bound on E[N ∙ 1{S>B}].

NF – # of steps that were in F

E[N ∙ 1{S>B}] = P[S>B] ∙ E[N | S>B] = (1-1/t)B ∙ E[N | S>B] ≤

exp(-B/t) ∙ E[NF | S>B] = exp(-B/t) ∙ E[S | S>B] ∙ (|F|/|E|) =

exp(-B/t) ∙ (B+t) ∙ |F|/|E| = exp(-10ln|Σ|) ∙ t(10ln|Σ| + 1) ∙ |F|/|E| =

(10ln|Σ| + 1)/|Σ|10 ∙ t(|F|/|E|) ≤ 1/(8|Σ|2) ∙ t(|F|/|E|) =

=> E[N*] ≥ – E[N ∙ 1{S>B}] ≥

8|Σ|2 |E| t |F|

8|Σ|2 |E| t |F|

4|Σ|2 |E| t |F|

Proof follows.

25

ProofLemma 1: E[N*] ≥ 8|Σ|2 |E|

t |F|

Why E[NF | S>B] = E[S | S>B] ∙ (|F|/|E|) ?

NF = # of steps of walk in F

S = # of steps in walk

Xi = 1[ith step is in F]

We’ll show E[NF] = E[S] ∙ (|F|/|E|) . This also holds with the condition.

E[NF] = E[Σi=1Xi] = E[X1 +…+ XN] = E[X1∙P[S≥1] + X2∙P[S≥2] +…]

E[Σi=1Xi∙P[S≥i]] = Σi=1(P[S≥i]∙E[Xi]) = Σi=1P[S≥i]∙P[Xi] = |F|/|E| ∙ ΣPi=1[S≥i] =

|F|/|E| ∙ Σi=1(P[S=i] + P[S=i+1] + …) = (|F|/|E|) ∙ Σi=1Σj=iP[S=j] =

(|F|/|E|) ∙ Σi=1i∙P[S=i] = (|F|/|E|) ∙ E[S]

N

∞ ∞ ∞ ∞

∞ ∞ ∞

∞

26

Time to get the job done

Lemma 2: E[(N*)2] ≤ t ∙ O(1) ∙ |E||F|

Each faulty* step is in F, so:

E[(N*)2] ≤ E[N2] ≤ E[(NF)2]

Denote Χi = 1[ith step is in F]. Then: NF ≤ Σi=1Xi

∞

27

ProofLemma 2: E[(N*)2] ≤ t ∙ O(1) ∙ |E|

|F|

E[(NF)2] ≤ E[(ΣXi)2] = E[Σi=1Xi2 + Σi≠jXiXj] =

Σi=1E[Xi2] + Σi≠jE[XiXj] = Σi,j=1E[XiXj] =

Σi,j=1P[XiXj] = Σi,j=1P[(Xi=1) ۸ (Xj=1)] ≤

2Σi=1Σj≥i(P[Xi=1] ∙ P[Xj=1 | Xi=1]) =

2Σi=1(P[Xi=1] ∙ Σj≥iP[Xj=1 | Xi=1])

P[Xj=1 | Xi=1] =

∞

∞

∞

∞

∞

∞

{1

i=j

P[walk takes at least j-i more steps] X

P[The (j-i)th step of a walk beginning in F, is in F]i≠j

Linearity of expectancy

XiXj is an indicator

∞

∞

∞ ∞

28

ProofLemma 2: E[(N*)2] ≤ t ∙ O(1) ∙ |E|

|F|

P[walk takes at least j-i more steps] = (1-1/t)j-i - Walk didn’t stop j-i times

P[The (j-i)th step of a walk beginning in F, is in F] = ?

Therefore,

P[The (j-i)th step of a walk beginning in F, is in F] = |F|/|E| + (λ/d)j-i-1

A Blast from the Past!

G=(V,E) is an (n,d,λ)-expander and FE. Then, the probability that a random walk which starts on an edge from F will pass on an edge from F on its tth step is bounded by |F|/|E| + (λ/d)t-1

.

29

Yay, calculations!Lemma 2: E[(N*)2] ≤ O(1) ∙ t ∙ |E|

|F|

E[(N*)2] ≤ E[(NF)2] ≤ 2Σi=1(P[Xi=1] ∙ [1 + Σk=1(1-1/t)k ∙ (|F|/|E| + (λ/d)k-

1)]) =

2Σi=1(P[Xi=1] ∙ [1 + Σk=1(1-1/t)k ∙ |F|/|E| + Σk=1(1-1/t)k(λ/d)k-1]) ≤

2Σi=1(P[Xi=1] ∙ [1 + Σk=1(1-1/t)k ∙ |F|/|E| + Σk=1(λ/d)k-1]) ≤

Reminder: if |a|<1, then Σt=0at converges to 1/(1-a).

2Σi=1(P[Xi=1] ∙ [1 + (t-1) ∙ |F|/|E| + O(1)]) ≤

Remember: we bound the growth of the UNSAT by a constant 1/t. Therefore, |F|/|E| ≤ 1/t.

O(1) ∙ Σi=1P[Xi=1] = O(1) ∙ E[NF] = O(1) ∙ t ∙

∞ ∞

∞ ∞ ∞

∞ λ<d, because G is an expander

∞|E||F|

Rejoice!!!

(1-1/t) < 1

30

SummaryGiven an (n,d,λ)-expander G, we manage to build a

new

graph G’’ with a considerably larger UNSAT, in polynomial

time.

We’ve done this by allowing unsatisfied edges to “spread

the disease”, using an expander’s properties.

Some of the graph properties were hurt. This will be dealt

with in the next lecture.

31

The end

Questions?