Yuan ZhouCarnegie Mellon University

Joint works with Boaz Barak, Fernando G.S.L. Brandão, Aram W. Harrow, Jonathan Kelner, Ryan

O'Donnell and David Steurer

Constraint Satisfaction Problems

• Given:– a set of variables: V– a set of values: Ω– a set of "local constraints": E

• Goal: find an assignment σ : V -> Ω to maximize #satisfied constraints in E

• α-approximation algorithm: always outputs a solution of value at least α*OPT

Example 1: Max-Cut

• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Typical local constraint: (i, j) э E wants σ(i) ≠

σ(j)

• Alternative description:– Given G = (V, E), divide V into two parts,– to maximize #edges across the cut

• Best approx. alg.: 0.878-approx. [GW'95]• Best NP-hardness: 0.941 [Has'01, TSSW'00]

Example 2: Balanced Seperator

• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Minimize #satisfied local constraints: (i, j) э E : σ(i) ≠ σ(j)• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤

2n/3

• Alternative description:– given G = (V, E)– divide V into two "balanced" parts,– to minimize #edges across the cut

Example 2: Balanced Seperator (cont'd)

• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1}• Minimize #satisfied local constraints: (i, j) э E : σ(i) ≠ σ(j)• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤

2n/3

• Best approx. alg.: sqrt{log n}-approx. [ARV'04]

• Only (1+ε)-approx. alg. is ruled out even assuming 3-SAT does not have subexp time alg. [AMS'07]

Example 3: Unique Games• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1, 2, ..., q - 1}• Maximize #satisfied local constraints: (i, j) э E : σ(i) - σ(j) = c (mod q)

• Unique Games Conjecture (UGC) [Kho'02, KKMO'07]

No poly-time algorithm, given an instance where optimal solution satisfies (1-ε) constraints, finds a solution satisfying ε constraints

• Stronger than (implies) "no constant approx. alg."

Example 3: Unique Games (cont'd)

• Vertex set: V = {1, 2, 3, ..., n}• Value set: Ω = {0, 1, 2, ..., q - 1}• Maximize #satisfied local constraints: (i, j) э E : σ(i) - σ(j) = c (mod q)

• UG(ε): to tell whether an instance has a solution satisfying (1-ε) constraints, or no solution satisfying ε constraints

• Unique Games Conjecture (UGC). UG(ε) is hard for sufficiently large q

Example 3: Unique Games (cont'd)

• Implications of UGC– For large class of problems, BASIC-SDP

(semidefinite programming relaxation) achieves optimal approximation ratio

Max-Cut: 0.878-approx. Vertex-Cover: 2-approx. Max-CSP [KKMO '07, MOO '10, KV '03, Rag '08]

Open questions

• Is UGC true?

• Are the implications of UGC true?– Is Max-Cut hard to approximate better than

0.878?

– Is Balanced Seperator hard to approximate with in constant factor?

SDP Relaxation hierarchies

• A systematic way to write tighter and tighter SDP relaxations

• Examples– Sherali-Adams+SDP [SA'90]– Lasserre hierarchy [Par'00, Las'01]

…

?

UG(ε)

r rounds SDP relaxation in roughly time

)(rOn

BASIC-SDP

GW SDP for Maxcut (0.878-approx.)ARV SDP for Balanced Seperator

How many rounds of tighening suffice?• Upperbounds

– rounds of SA+SDP suffice for UG(ε) [ABS'10,

BRS'11]

• Lowerbounds [KV'05, DKSV'06, RS'09, BGHMRS '12]

(also known as constructing integrality gap instances)

– rounds of SA+SDP needed for UG(ε)

– rounds of SA+SDP needed for better-than-0.878 approx for Max-Cut

– rounds for SA+SDP needed for constant approx. for Balanced Seperator

)1(n

))logexp((log )1(n

)1()log(log n

))logexp((log )1(n

Our Results

• We study the performance of Lasserre SDP hierarchy against known lowerbound instances for SA+SDP hierarchy, and show that

• 8-round Lasserre solves the Unique Games lowerbound instances [BBHKSZ'12]

• 4-round Lasserre solves the Balanced Seperator lowerbound instances [OZ'12]

• Constant-round Lasserre gives better-than-0.878 approximation for Max-Cut lowerbound instances [OZ'12]

Proof overview

• Integrality gap instance– SDP completeness: a good vector solution– Integral soundness: no good integral

solution

• A common method to construct gaps (e.g. [RS'09])

– Use the instance derived from a hardness reduction

– Lift the completeness proof to vector world– Use the soundness proof directly

Proof overview (cont'd)

• Our goal: to prove there is no good vector solution– Rounding algorithms?

• Instead, – we bound the value of the dual of the SDP– interpret the dual of the SDP as a proof

system ("Sum-of-squares proof system")– lift the soundness proof to the proof

system

Remarks• Using a connection between SDP hierarchies

and algebraic proof systems, we refute all known UG lowerbound instances and many instances for its related problems

• We provide new insight in designing integrality gap instances -- should avoid soundness proofs that can be lifted to the powerful Sum-of-Squares proof system

• We show that Lasserre is strictly stronger than other hierarchies on UG and its related problems (as it was believed to be)

Outline of the rest of the talk

• Sum-of-Squares proof system and Lasserre hierarchy

• Lift the soundness proofs to the SoS proof system

Sum-of-Squares proof system and Lasserre hierarchy

Polynomial optimization

• Maximize/Minimize• Subject to

all functions are low-degree n-variate polynomial functions

• Max-Cut example: Maximize

s.t.

)(xp

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

2)(E jiE(i,j)

xx

ixx ii ,0)1(

Polynomial optimization (cont'd)

• Maximize/Minimize• Subject to

all functions are low-degree n-variate polynomial functions

• Balanced Seperator example: Minimize

s.t.

)(xp

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

32

31 ][E,][E

,0)1(

ii

ii

ii

xx

ixx

2)(E jiE(i,j)

xx

Certifying no good solution

• Maximize• Subject to

• To certify that there is no solution better than , simply say that the following equations & inequalities are infeasible

)(xp

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

)(xp

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

The Sum-of-Squares proof system

• To show the following equations & inequalities are infeasible,

• Show that

• where is a sum of squared polynomials, including 's

• A degree-d "Sum-of-Squares" refutation, where

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

)}deg(),deg(){deg(max hqfd iii

)()()(1...1

xhxqxfmi

ii

)(xh)(xri

Example 1

• To refute

• We simply write

• A degree-2 SoS refutation

2)1()2()1(1 xxxx

0)1(

2

xx

x

Example 2: Max-Cut on triangle graph

• To refute

• We "simply" write ... ...

0)1(,0)1(,0)1( 332211 xxxxxx

2)()()( 213

232

221 xxxxxx

)12)(1(

)3222)(1(

)12)(1(

)1()1()(

2)()()(

212133

313123122

3223

2211

232

221

22313221

213

232

221

xxxxxx

xxxxxxxx

xxxxxx

xxxxxxxxxxx

xxxxxx

Example 2: Max-Cut on triangle graph (cont'd)

• A degree-4 SoS refutation

Relation between SoS proof system and Lasserre SDP hierarchy

Finding SoS refutation by SDP

• A degree-d SoS refutation corresponds to solution of an SDP with variables

• The SDP is the same as the dual of -round Lasserre relaxation

• An SoS refutation => upperbound on the dual of optimum of Lasserre => upperbound on the value of Lasserre– e.g. 4-round Lasserre says that Max-Cut of

the triangle graph is at most 2 (BASIC-SDP gives 9/4)

)( dnO

)(d

Bounding SDP value by SoS refutation

Remarks• Positivestellensatz. [Krivine'64, Stengle'73] If

the given equalities & inequalities are infeasible, there is always an SoS refutation (degree not bounded).

• The degree-d SoS proof system was first proposed by Grigoriev and Vorobjov in 1999

• Grigoriev showed degree is needed to refute unsatisfiable sparse -linear equations– later rediscovered by Schoenbeck in

Lasserre world

)(n2F

SoS proofs (in contrast to refutations)

• Given assumptions

to prove that

• A degree-d SoS proof writes

where are sums of squared

polynomials

• Remark. Degree-d SoS proof => degree-d SoS refutation for

)(xp

0)(,0)(,0)( 21 xqxqxq m0)(,0)(,0)( '21 xrxrxr m

)()()()(...1

xhxqxfxpmi

ii

)}deg(),deg(){deg(max hqfd iii

0,)( xp

)(),( xhxgi

Technical Part:Lift the proofs to SoS proof

system

Components of the soundness proof

• Cauchy-Schwarz/Hölder's inequality• Hypercontractivity inequality• Smallsets expand in the noisy hypercube• Invariance Principle• Influence decoding

(of known UG instances)

Hypercontractivity Inequality

• 2->4 hypercontractivity inequality: for low degree polynomial

we have

• Goal of an SoS proof: write

Note that 's are indeterminates

dSnSi

SiS xxf

||],[

)(

22

}1,1{

4

}1,1{])([E9])([E

xfxf

nn x

d

x

ixx

d hxfxfnn

2}2{}1{

4

}1,1{

22

}1,1{),,,(])([E])([E9

S

Traditional proof of hypercontractivity

• 2->4 hypercontractivity inequality: for low degree polynomial

we have

• (Traditional) proof. Apply induction on d and n.– Let – g and h are (n-1)-variate polynomials,

dSnSi

SiS xxf

||],[

)(

22

}1,1{

4

}1,1{])([E9])([E

xfxf

nn x

d

x

hgxf 1

nhng )deg(,1)deg(

Traditional proof of hypercontractivity (cont'd)

]446[E

])[(E][E33

13

1222

1444

1

41

4

hgxghxhgxhgx

hgxf

][E6][E][E 2244 hghg

][E][E6][E][E 4444 hghg

][E9][E96][E9][E9 42/)1(22/2222 hghg dddd

222 ])[E][(E9 hgd 22 ])[(E9 fd

(Cauchy-Schwartz)

(induction)

All equalities are polynomial identities about indeterminatesS

SoS proof of hypercontractivity?

• The square-root in the Cauchy-Schwartz step looks difficult for polynomials

• Solution: Prove a stronger statement -- two-function hypercontractivity inequality

• Theorem. Suppose

• then

eSnSi

SiS

dSnSi

SiS xxgxxf

||],[||],[

)(,)(

][E][E9][E 2222 2 gfgfed

SoS proof of two-fcn hypercontractivity• Write 101101 , ggxgffxf

]4[E

])()[(E][E

101020

21

21

20

21

21

20

20

2101

2101

22

ggffgfgfgfgf

ggxffxgf

][E2][E2][E 20

21

21

20

20

21

21

20

21

21

20

20 gfgfgfgfgfgf

]33[E 20

21

21

20

21

21

20

20 gfgfgfgf

][E93][E93

][E][E9][E][E920

21

21

20

21

21

20

20

21

21

22

gfgf

gfgfeded

eded

0)( 20110 gfgfusing

(induction)

unroll the induction to get the SoS proof][E][E9

])[E][E])([E][(E922

21

20

21

20

2

2

gf

ggffed

ed

Components of the soundness proof

• Cauchy-Schwarz/Hölder's inequality• Hypercontractivity inequality• Smallsets expand in the noisy hypercube• Invariance Principle• Influence decoding

(of known UG instances)

Smallset expansion of noisy hypercube

• For , let

• Theorem. If

• then

• Traditional proof. Let be the projection operator onto the eigenspace of with eigenvalue . I.e. the space spanned by

Rf n }1,1{: )]([E)(1~

1 yfxfTxy

][E

,0))(1)((

f

xxfxf

)(11 )]()([E xfTxf

x

P

1T

}log:)({ 1 Sxx i

SiS

Traditional proof of SSE of noisy hypercube (cont'd)

])([E)]()([E

)]()([E)]()([E

)]()([E

2

11

1

xfxfPxf

xfPTxfxfPTxf

xfTxf

xx

xx

x

])([E]))([(E])([E 24/144/33/4 xfxfPxfxxx

)]([E]))([(E)]([E 4/144/3 xfxfPxfxxx

)]([E]))([(E3)]([E 2/12log4/3 1

xfxfPxfxxx

)]([E])([E3)]([E 2/12log4/3 1

xfxfxfxxx

)]([E)]([E3 4/5log1

xfxfxx

(SoS friendly)

(Holder's)

(SoS friendly)

(SoS friendly)

(hypercontractivity)

(SoS friendly)

(poly. identity)

Traditional proof of SSE of noisy hypercube (cont'd)

)]()([E 1 xfTxfx

4/5log

4/5log

1

1

3

)]([E)]([E3 xfxfxx

)(1 100/

(SoS friendly)

(take )

Key problem: fractional power involved in the Holder's step

Solution: Cauchy-Schwartz/Holders with no fractional power

SoS-izable Cauchy-Schwartz

• Theorem. For any constant a > 0

where SoS is a sum of squared polynomials of degree at most 2

• Remark. and the equality holds when

• Proof. Skipped.

• Corollary. (Holder's) For any constant a > 0

• Proof. Apply C-S twice

SoSfg-gf aa ]E[]E[]E[ 2

1222

SoSgf-fgf abaab ]E[]E[]E[]E[ 3

214

4424

4

XX aa 2

12

Xa

SoS proof of SSE

axba

xx

ab

xx

xfxfPxf

xfPxfxfPxf

214

4424

4

3

])([E]))([(E])([E

)]()([E)]()([E

aba

x

ab xfP 21

442

4 ]))([(E

aba

x

ab xfP 21

422log2

4 ]))([(E31

abaab

21

4

log44

1

3

4/543

log4/541 1

3

(Holder's)

(SoS friendly)

(take )

4/64/5 , ba

(hypercontractivity)

SoS proof of SSE (cont'd)

])([E)]()([E

)]()([E

2

1

xfxfPxf

xfTxf

xx

x

)(1

4/543

log4/541 1

3

100/ (take )

Components of the soundness proof

• Cauchy-Schwarz/Hölder's inequality• Hypercontractivity inequality• Smallsets expand in the noisy hypercube• Invariance Principle• Influence decoding

(of known UG instances)

A few words on Invariance Principle• trickier • "bump function" is used in the original proof

--- not a polynomial!

• but... a polynomial substitution is enough for UG

Max-Cut and Balanced Seperator• An SoS proof for "Majority Is Stablest" theorem

is needed for Max-Cut instances– We don't know how to get around the bump

function issue in the invariance step– Instead, we proved a weaker theorem: "2/pi

theorem" -- suffices to give better-than-0.878 algorithms for known Max-Cut instances

• Balanced Seperator. Key is to SoS-ize the proof for KKL theorem– Hypercontractivity and SSE is also useful

there – Some more issues to be handled

Summary

• SoS/Lasserre hierarchy refutes all known UG instances and Balanced Seperator instances, gives better-than-0.878 approximation for known Max-Cut instances,– certain types of soundness proof does not

work for showing a gap of SoS/Lasserre hierarchy

Open problems

• Show that SoS/Lasserre hierarchy fully refutes Max-Cut instances?– SoS-ize Majority Is Stablest theorem...

• More lowerbound instances for SoS/Lasserre hierarchy?

Thank you!