Download - Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Transcript
Page 1: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Extended Formulations and Information Complexity

Ankur Moitra Massachusetts Institute of Technology

Dagstuhl, March 2014

Page 2: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Permutahedron

Page 3: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

Page 4: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

Page 5: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have? exponentially many!

Page 6: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

e.g. S [n], Σi in S xi ≥ 1 + 2 + … + |S| = |S|(|S|+1)/2

exponentially many!

Page 7: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

e.g. S [n], Σi in S xi ≥ 1 + 2 + … + |S| = |S|(|S|+1)/2

Let Q = {A| A is doubly-stochastic}

exponentially many!

Page 8: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

e.g. S [n], Σi in S xi ≥ 1 + 2 + … + |S| = |S|(|S|+1)/2

Let Q = {A| A is doubly-stochastic}

Then P is the projection of Q: P = {A t | A in Q }

exponentially many!

Page 9: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Permutahedron

Let t = [1, 2, 3, … n], P = conv{π(t) | π is permutation}

How many facets of P have?

e.g. S [n], Σi in S xi ≥ 1 + 2 + … + |S| = |S|(|S|+1)/2

Let Q = {A| A is doubly-stochastic}

Then P is the projection of Q: P = {A t | A in Q }

Yet Q has only O(n2) facets

exponentially many!

Page 10: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Extended Formulations

The extension complexity (xc) of a polytope P is the minimum number of facets of Q so that P = proj(Q)

Page 11: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Extended Formulations

The extension complexity (xc) of a polytope P is the minimum number of facets of Q so that P = proj(Q)

e.g. xc(P) = Θ(n logn) for permutahedron

Page 12: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Extended Formulations

The extension complexity (xc) of a polytope P is the minimum number of facets of Q so that P = proj(Q)

e.g. xc(P) = Θ(n logn) for permutahedron

xc(P) = Θ(logn) for a regular n-gon, but Ω(√n) for its perturbation

Page 13: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Extended Formulations

The extension complexity (xc) of a polytope P is the minimum number of facets of Q so that P = proj(Q)

e.g. xc(P) = Θ(n logn) for permutahedron

xc(P) = Θ(logn) for a regular n-gon, but Ω(√n) for its perturbation

In general, P = {x | y, (x,y) in Q} E

Page 14: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Extended Formulations

The extension complexity (xc) of a polytope P is the minimum number of facets of Q so that P = proj(Q)

e.g. xc(P) = Θ(n logn) for permutahedron

xc(P) = Θ(logn) for a regular n-gon, but Ω(√n) for its perturbation

In general, P = {x | y, (x,y) in Q} E

…analogy with quantifiers in Boolean formulae

Page 15: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Page 16: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Page 17: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Hence, we can run standard LP solvers instead of the ellipsoid algorithm

Page 18: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Hence, we can run standard LP solvers instead of the ellipsoid algorithm EFs often give, or are based on new combinatorial insights

Page 19: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Hence, we can run standard LP solvers instead of the ellipsoid algorithm EFs often give, or are based on new combinatorial insights

e.g. Birkhoff-von Neumann Thm and permutahedron

Page 20: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Applications of EFs

In general, P = {x | y, (x,y) in Q} E

Through EFs, we can reduce # facets exponentially!

Hence, we can run standard LP solvers instead of the ellipsoid algorithm EFs often give, or are based on new combinatorial insights

e.g. Birkhoff-von Neumann Thm and permutahedron e.g. prove there is low-cost object, through its polytope

Page 21: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Explicit, Hard Polytopes?

Page 22: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Explicit, Hard Polytopes? Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn}

Page 23: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Explicit, Hard Polytopes? Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn} (If we could optimize over this polytope, then P = NP)

Page 24: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Explicit, Hard Polytopes?

Can we prove unconditionally there is no small EF?

Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn} (If we could optimize over this polytope, then P = NP)

Page 25: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Explicit, Hard Polytopes?

Can we prove unconditionally there is no small EF?

Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn}

Caveat: this is unrelated to proving complexity l.b.s

(If we could optimize over this polytope, then P = NP)

Page 26: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Explicit, Hard Polytopes?

Can we prove unconditionally there is no small EF?

[Yannakakis ’90]: Yes, through the nonnegative rank

Definition: TSP polytope:

P = conv{1F| F is the set of edges on a tour of Kn}

Caveat: this is unrelated to proving complexity l.b.s

(If we could optimize over this polytope, then P = NP)

Page 27: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History

Theorem [Yannakakis ’90]: Any symmetric EF for TSP or matching has size 2Ω(n)

Page 28: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History

Theorem [Yannakakis ’90]: Any symmetric EF for TSP or matching has size 2Ω(n)

…but asymmetric EFs can be more powerful

Page 29: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History

Theorem [Yannakakis ’90]: Any symmetric EF for TSP or matching has size 2Ω(n)

…but asymmetric EFs can be more powerful

Page 30: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History

Theorem [Yannakakis ’90]: Any symmetric EF for TSP or matching has size 2Ω(n)

…but asymmetric EFs can be more powerful

Theorem [Fiorini et al ’12]: Any EF for TSP has size 2Ω(√n) (based on a 2Ω(n) lower bd for clique)

Approach: connections to non-deterministic CC

Page 31: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History II

Theorem [Braun et al ’12]: Any EF that approximates clique within n1/2-eps has size exp(neps)

Approach: Razborov’s rectangle corruption lemma

Page 32: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History II

Theorem [Braun et al ’12]: Any EF that approximates clique within n1/2-eps has size exp(neps)

Approach: Razborov’s rectangle corruption lemma

Theorem [Braverman, Moitra ’13]: Any EF that approximates clique within n1-eps has size exp(neps)

Approach: information complexity

Page 33: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History II

Theorem [Braun et al ’12]: Any EF that approximates clique within n1/2-eps has size exp(neps)

Approach: Razborov’s rectangle corruption lemma

Theorem [Braverman, Moitra ’13]: Any EF that approximates clique within n1-eps has size exp(neps)

Approach: information complexity

see also [Braun, Pokutta ’13]: reformulation using common information, applications to avg. case

Page 34: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History III

Theorem [Chan et al ’12]: Any EF that approximates MAXCUT within 2-eps has size nΩ(log n/loglog n)

Approach: reduction to Sherali-Adams

Page 35: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

An Abridged History III

Theorem [Chan et al ’12]: Any EF that approximates MAXCUT within 2-eps has size nΩ(log n/loglog n)

Approach: reduction to Sherali-Adams

Theorem [Rothvoss ’13]: Any EF for perfect matching has size 2Ω(n) (same for TSP)

Approach: hyperplane separation lower bound

Page 36: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 37: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 38: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

Page 39: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

How can we prove lower bounds on EFs?

Page 40: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric Parameter

Algebraic Parameter

Page 41: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric Parameter

Algebraic Parameter

Definition of the slack matrix…

Page 42: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Slack Matrix

Page 43: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Slack Matrix

P

Page 44: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Slack Matrix

P S(P)

vertex

face

t

Page 45: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Slack Matrix

P S(P)

vertex

face

t

vj

Page 46: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Slack Matrix

P S(P)

vertex

face

t

vj

<ai,x> ≤ bi

Page 47: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Slack Matrix

The entry in row i, column j is how slack the jth vertex is on the ith constraint

P S(P)

vertex

face

t

vj

<ai,x> ≤ bi

Page 48: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Slack Matrix

The entry in row i, column j is how slack the jth vertex is on the ith constraint

bi - <ai, vj>

P S(P)

vertex

face

t

vj

<ai,x> ≤ bi

Page 49: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric Parameter

Algebraic Parameter

Definition of the slack matrix…

Page 50: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric Parameter

Algebraic Parameter

Definition of the slack matrix…

Definition of the nonnegative rank…

Page 51: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Nonnegative Rank

S =

Page 52: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Nonnegative Rank

S M1 Mr = + + …

rank one, nonnegative

Page 53: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Nonnegative Rank

S M1 Mr = + + …

Definition: rank+(S) is the smallest r s.t. S can be written as the sum of r rank one, nonneg. matrices

rank one, nonnegative

Page 54: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Nonnegative Rank

S M1 Mr = + + …

Definition: rank+(S) is the smallest r s.t. S can be written as the sum of r rank one, nonneg. matrices

rank one, nonnegative

Note: rank+(S) ≥ rank(S), but can be much larger too!

Page 55: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric Parameter

Algebraic Parameter

Page 56: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric Parameter

Algebraic Parameter

xc(P) = rank+(S(P))

Page 57: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric Parameter

Algebraic Parameter

xc(P) = rank+(S(P))

Intuition: the factorization gives a change of variables that preserves the slack matrix!

Page 58: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Factorization Theorem

[Yannakakis ’90]:

How can we prove lower bounds on EFs?

Geometric Parameter

Algebraic Parameter

xc(P) = rank+(S(P))

Intuition: the factorization gives a change of variables that preserves the slack matrix!

Next we will give a method to lower bound rank+ via information complexity…

Page 59: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 60: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 61: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

=

rank one, nonnegative

S M1 Mr + + …

Page 62: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

=

rank one, nonnegative

M1 Mr + + …

Page 63: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

= + + …

rank one, nonnegative

Mr

Page 64: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

= + + …

rank one, nonnegative

Mr

Page 65: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

= + + …

rank one, nonnegative

Page 66: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

= + + …

rank one, nonnegative

The support of each Mi is a combinatorial rectangle

Page 67: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

= + + …

rank one, nonnegative

The support of each Mi is a combinatorial rectangle

rank+(S) is at least # rectangles needed to cover supp of S

Page 68: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

= + + …

rank one, nonnegative

rank+(S) is at least # rectangles needed to cover supp of S

Page 69: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Rectangle Bound

= + + …

rank one, nonnegative

rank+(S) is at least # rectangles needed to cover supp of S

Non-deterministic Comm. Complexity

Page 70: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 71: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 72: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

Page 73: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Page 74: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Page 75: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Page 76: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Page 77: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

Page 78: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

Page 79: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

Page 80: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

This outputs a uniformly random sample from T

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

Page 81: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

Page 82: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Sampling Argument

= + + …

If r is too small, this procedure uses too little entropy!

T = { }, set of entries in S with same value

Choose Mi proportional to total value on T

Choose (a,b) in T proportional to relative value in Mi

Page 83: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 84: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 85: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al] correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Page 86: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al]

S constraints:

vertices:

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Page 87: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al]

S constraints:

vertices: a in {0,1}n

b in {0,1}n

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Page 88: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al]

S constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Page 89: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al]

S constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

UNIQUE DISJ. Output ‘YES’ if a

and b as sets are disjoint, and ‘NO’ if a and b have one index

in common

correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Page 90: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al] correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Page 91: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al] correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Why is that (a sub-matrix of) the slack matrix?

Page 92: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al] correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Why is that (a sub-matrix of) the slack matrix?

(1-aTb)2 = 1 – 2aTb + (aTb)2

Page 93: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al] correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Why is that (a sub-matrix of) the slack matrix?

(1-aTb)2 = 1 – 2aTb + (aTb)2

= 1 – 2 diag(b),aaT + bbT,aaT

Page 94: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al] correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Why is that (a sub-matrix of) the slack matrix?

(1-aTb)2 = 1 – 2aTb + (aTb)2

= 1 – 2 diag(b),aaT + bbT,aaT

1 ≥ 2diag(b) - bbT,aaT

Page 95: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al] correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Why is that (a sub-matrix of) the slack matrix?

(1-aTb)2 = 1 – 2aTb + (aTb)2

= 1 – 2 diag(b),aaT + bbT,aaT

1 ≥ 2diag(b) - bbT,aaT

What is the slack?

Page 96: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Construction of [Fiorini et al] correlation polytope: Pcorr= conv{aaT|a in {0,1}n }

Why is that (a sub-matrix of) the slack matrix?

(1-aTb)2 = 1 – 2aTb + (aTb)2

= 1 – 2 diag(b),aaT + bbT,aaT

1 ≥ 2diag(b) - bbT,aaT

What is the slack? (1-aTb)2

Page 97: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Hard Distribution

Page 98: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Hard Distribution Let T = {(a,b) | aTb = 0}, |T| = 3n

Page 99: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Hard Distribution Let T = {(a,b) | aTb = 0}, |T| = 3n

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

Page 100: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Hard Distribution Let T = {(a,b) | aTb = 0}, |T| = 3n

How does the sampling procedure specialize to this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

Page 101: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Hard Distribution Let T = {(a,b) | aTb = 0}, |T| = 3n

Sampling Procedure:

How does the sampling procedure specialize to this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

Page 102: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Hard Distribution Let T = {(a,b) | aTb = 0}, |T| = 3n

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

How does the sampling procedure specialize to this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

Page 103: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Hard Distribution Let T = {(a,b) | aTb = 0}, |T| = 3n

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

How does the sampling procedure specialize to this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

Page 104: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

A Hard Distribution Let T = {(a,b) | aTb = 0}, |T| = 3n

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

How does the sampling procedure specialize to this case? (Recall it generates (a,b) unif. from T)

Recall: Sa,b=(1-aTb)2, so Sa,b=1 for all pairs in T

Page 105: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Page 106: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

Page 107: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

≤ n log23

Page 108: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

+ choose i

choose (a,b) conditioned on i

≤ n log23

Page 109: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r + choose i

choose (a,b) conditioned on i

≤ n log23

Page 110: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r (1-δ)n log23 + choose i

choose (a,b) conditioned on i

≤ n log23

Page 111: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Sampling Procedure:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r (1-δ)n log23 + choose i

choose (a,b) conditioned on i

≤ n log23 (?)

Page 112: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum
Page 113: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Suppose that a-j and b-j are fixed

a b bj

aj

Page 114: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Suppose that a-j and b-j are fixed

a b bj

aj

Mi restricted to (a-j,b-j)

Page 115: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Suppose that a-j and b-j are fixed

a b bj

aj

Mi restricted to (a-j,b-j)

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

Page 116: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

Page 117: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

Page 118: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

zero

Page 119: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b)

Mi(a,b)

Mi(a,b)

(…bj=0…) (…bj=1…)

But rank(Mi)=1, hence there must be another zero in either the same row or column

zero

Page 120: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b) Mi(a,b)

(…bj=0…) (…bj=1…)

But rank(Mi)=1, hence there must be another zero in either the same row or column

zero zero

Page 121: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

If aj=1, bj=1 then aTb = 1, hence Mi(a,b) = 0

(a1..j-1,aj=0,aj+1…n)

(a1..j-1,aj=1,aj+1…n)

Mi(a,b) Mi(a,b)

(…bj=0…) (…bj=1…)

But rank(Mi)=1, hence there must be another zero in either the same row or column

zero zero

H(aj,bj| i, a-j, b-j) ≤ 1 < log23

Page 122: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Generate uniformly random (a,b) in T:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r + choose i

choose (a,b) conditioned on i

≤ n log23

Page 123: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Entropy Accounting 101

Generate uniformly random (a,b) in T:

� Let Ri be the sum of Mi(a,b) over (a,b) in T and

let R be the sum of Ri

� Choose i with probability Ri/R

� Choose (a,b) with probability Mi(a,b)/Ri

Total Entropy:

log2r n + choose i

choose (a,b) conditioned on i

≤ n log23

Page 124: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 125: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 126: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Approximate EFs [Braun et al]

S constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

Page 127: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Approximate EFs [Braun et al]

S constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

Is there a K (with small xc) s.t. Pcorr K (C+1)Pcorr?

Page 128: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Approximate EFs [Braun et al]

S constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

Is there a K (with small xc) s.t. Pcorr K (C+1)Pcorr?

+ C

Page 129: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Approximate EFs [Braun et al]

S constraints:

vertices: a in {0,1}n

b in {0,1}n

(1-aTb)2

New Goal: Output the answer to UDISJ with prob. at least ½ + 1/2(C+1)

Is there a K (with small xc) s.t. Pcorr K (C+1)Pcorr?

+ C

Page 130: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum
Page 131: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Is the correlation polytope hard to approximate for large values of C?

Analogy: Is UDISJ hard to compute with prob. ½+1/2(C+1) for large values of C?

Page 132: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Is the correlation polytope hard to approximate for large values of C?

Analogy: Is UDISJ hard to compute with prob. ½+1/2(C+1) for large values of C?

There is a natural barrier at C = √n for proving l.b.s:

Page 133: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Is the correlation polytope hard to approximate for large values of C?

Analogy: Is UDISJ hard to compute with prob. ½+1/2(C+1) for large values of C?

Claim: If UDISJ can be computed with prob. ½+1/2(C+1) using o(n/C2) bits, then UDISJ can be computed with prob. ¾ using o(n) bits

There is a natural barrier at C = √n for proving l.b.s:

Page 134: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Is the correlation polytope hard to approximate for large values of C?

Analogy: Is UDISJ hard to compute with prob. ½+1/2(C+1) for large values of C?

Claim: If UDISJ can be computed with prob. ½+1/2(C+1) using o(n/C2) bits, then UDISJ can be computed with prob. ¾ using o(n) bits

Proof: Run the protocol O(C2) times and take the majority vote

There is a natural barrier at C = √n for proving l.b.s:

Page 135: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Is the correlation polytope hard to approximate for large values of C?

Analogy: Is UDISJ hard to compute with prob. ½+1/2(C+1) for large values of C?

There is a natural barrier at C = √n for proving l.b.s:

Page 136: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Is the correlation polytope hard to approximate for large values of C?

Analogy: Is UDISJ hard to compute with prob. ½+1/2(C+1) for large values of C?

Corollary [from K-S]: Computing UDISJ with probability ½+1/2(C+1) requires Ω(n/C2) bits

There is a natural barrier at C = √n for proving l.b.s:

Page 137: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Is the correlation polytope hard to approximate for large values of C?

Analogy: Is UDISJ hard to compute with prob. ½+1/2(C+1) for large values of C?

Corollary [from K-S]: Computing UDISJ with probability ½+1/2(C+1) requires Ω(n/C2) bits

There is a natural barrier at C = √n for proving l.b.s:

Theorem [B-M]: Computing UDISJ with probability ½+1/2(C+1) requires Ω(n/C) bits

Page 138: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Is the correlation polytope hard to approximate for large values of C?

Analogy: Is UDISJ hard to compute with prob. ½+1/2(C+1) for large values of C?

Theorem [B-M]: Any EF that approximates clique within n1-eps has size exp(neps)

There is a natural barrier at C = √n for proving l.b.s:

Theorem [B-M]: Computing UDISJ with probability ½+1/2(C+1) requires Ω(n/C) bits

Page 139: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 140: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Outline

Part I: Tools for Extended Formulations

� Yannakakis’s Factorization Theorem

� The Rectangle Bound

� A Sampling Argument

Part II: Applications

� Correlation Polytope

� Approximating the Correlation Polytope

� Matching Polytope

Page 141: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Matching Polytope [Edmonds] PPM= conv{1M| M is a perfect matching in Kn}

Page 142: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Matching Polytope [Edmonds]

S

constraints:

vertices: 1M

U [n] with |U| = odd

PPM= conv{1M| M is a perfect matching in Kn}

Page 143: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Matching Polytope [Edmonds]

S

constraints:

vertices: 1M

U [n] with |U| = odd

|δ(U) M|-1

PPM= conv{1M| M is a perfect matching in Kn}

Page 144: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Matching Polytope [Edmonds]

S

constraints:

vertices: 1M

U [n] with |U| = odd

|δ(U) M|-1

Is there a small rectangle covering?

PPM= conv{1M| M is a perfect matching in Kn}

Page 145: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

The Matching Polytope [Edmonds]

S

constraints:

vertices: 1M

U [n] with |U| = odd

|δ(U) M|-1

Is there a small rectangle covering?

PPM= conv{1M| M is a perfect matching in Kn}

Yes! Just guess two edges in M, crossing the cut

Page 146: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Hyperplane Separation Lemma [Rothvoss] attributed to [Fiorini]:

Page 147: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Hyperplane Separation Lemma [Rothvoss] attributed to [Fiorini]:

Lemma: For slack matrix S, any matrix W:

rank+(S) ≥ S, W

||S||∞ α

where α = max W,R s.t. R is rank one, entries in [0,1]

Page 148: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Hyperplane Separation Lemma [Rothvoss] attributed to [Fiorini]:

Lemma: For slack matrix S, any matrix W:

rank+(S) ≥ S, W

||S||∞ α

where α = max W,R s.t. R is rank one, entries in [0,1]

Proof:

W,S = Σ ||Ri||∞ W, Ri/||Ri||∞ ≤ α r ||S||∞

Page 149: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Theorem [Rothvoss ’13]: Any EF for perfect matching has size 2Ω(n) (same for TSP)

Page 150: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Theorem [Rothvoss ’13]: Any EF for perfect matching has size 2Ω(n) (same for TSP)

How do we choose W?

Page 151: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Theorem [Rothvoss ’13]: Any EF for perfect matching has size 2Ω(n) (same for TSP)

WU,M =

-∞ if |δ(U) M| = 1

1/Q3 if |δ(U) M| = 3

-1/Qk if |δ(U) M| = k 0 else

How do we choose W?

Page 152: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Theorem [Rothvoss ’13]: Any EF for perfect matching has size 2Ω(n) (same for TSP)

WU,M =

-∞ if |δ(U) M| = 1

1/Q3 if |δ(U) M| = 3

-1/Qk if |δ(U) M| = k 0 else

Proof is a substantial modification to Razborov’s rectangle corruption lemma

How do we choose W?

Page 153: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Summary:

� Extended formulations and Yannakakis’ factorization theorem

Page 154: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Summary:

� Extended formulations and Yannakakis’ factorization theorem

� Lower bound techniques: rectangle bound, information complexity, hyperplane separation

Page 155: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Summary:

� Extended formulations and Yannakakis’ factorization theorem

� Lower bound techniques: rectangle bound, information complexity, hyperplane separation

� Applications: connections between correlation polytope and disjointness,

Page 156: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Summary:

� Extended formulations and Yannakakis’ factorization theorem

� Lower bound techniques: rectangle bound, information complexity, hyperplane separation

� Applications: connections between correlation polytope and disjointness,

� Open question: Can we prove lower bounds against general SDPs?

Page 157: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Any Questions? Summary:

� Extended formulations and Yannakakis’ factorization theorem

� Lower bound techniques: rectangle bound, information complexity, hyperplane separation

� Applications: connections between correlation polytope and disjointness,

� Open question: Can we prove lower bounds against general SDPs?

Page 158: Extended Formulations and Information Complexitypeople.csail.mit.edu/moitra/docs/tutorialf.pdf · Extended Formulations The extension complexity (xc) of a polytope P is the minimum

Thanks!

P vj