3.8 Strong valid inequalities

By studying the problem structure, we can derive strong valid inequalities which lead tobetter approximations of the ideal formulation conv(X ) and hence to tighter bounds.

Two definitions related to polyhedra and their description:

Consider a polyhedron P = {x ∈ Rn+ : Ax ≤ b}.

Definition: Given two valid inequalities πtx ≤ π0 and µtx ≤ µ0 for P, πtx ≤ π0

dominates µtx ≤ µ0 if ∃ u > 0 such that uµ ≤ π and π0 ≤ uµ0 with (π, π0) 6= (uµ, uµ0).

Poiche uµtx ≤ πtx ≤ π0 ≤ uµ0, chiaramente {x ∈ Rn+ : πtx ≤ π0} ⊆ {x ∈ Rn

+ : µtx ≤ µ0}

Example: x1 + 3x2 ≤ 4 dominates 2x1 + 4x2 ≤ 9 since for (π, π0) = (1, 3, 4) and

(µ, µ0) = (2, 4, 9) we have 12µ ≤ π and π0 ≤ 1

2µ0.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 1 / 27

Definition: A valid inequality πtx ≤ π0 is redundant in the description of P

if there exist k ≥ 2 valid inequalities πix ≤ πi0 for P with ui > 0, 1 ≤ i ≤ k, such that

(k∑

i=1

uiπi )x ≤

k∑i=1

uiπi0 dominates πtx ≤ π0.

Example:

P = {(x1, x2) ∈ R2+ : −x1 + 2x2 ≤ 4, −x1 − 2x2 ≤ −3, −x1 + x2 ≤ 5/3, 1 ≤ x1 ≤ 3}

−x1 + x2 ≤ 5/3 is redundant because it is dominated by −x1 + x2 ≤ 3/2, which isimplied by −x1 + 2x2 ≤ 4 and −x1 ≤ −1 (with u1 = u2 = 1

2)

Observation: When P = conv(X ) is not known explicitly it can be very difficult to checkredundancy. In pratice, we should avoid inequalities that are dominated by other alreadyavailable inequalities.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 2 / 27

3.8.1 Faces and facets of polyhedra

Consider a polyhedron P = {x ∈ Rn : Ax ≤ b}

Definitions

The vectors x1, . . . , xk are affinely independent if the k − 1 vectorsx2 − x1, . . . , xk − x1 are linearly independent.

The dimension of P, dim(P), is equal to the maximum number of affinely linearlyindependent points of P minus 1.

P is full-dimensional if dim(P) = n, i.e., no equation atx = b is satisfied withequality by all the points x ∈ P.

Illustrations:

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 3 / 27

For the sake of simplicity, we assume that P is full dimensional

Theorem: If P is of full dimension, P admits a unique minimal description

P = {x ∈ Rn : ati x ≤ bi , i = 1, . . . ,m}

where each inequality is unique within a positive multiple.

Each inequality is necessary: deleting anyone of them we obtain a polyhedron that differsfrom P.

Moreover, each valid inequality for P which is not a positive multiple of one of theati x ≤ bi is redundant (can be obtained as linear combination with non negative

coefficients of two or more valid inequalities).

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 4 / 27

Alternative characterization of necessary inequalities

Definitions

Let F = {x ∈ P : πtx = π0} for any valid inequality πtx ≤ π0 for P. Then F is aface of P and the inequality πtx ≤ π0 represents or defines F .

If F is a face of P and dim(F ) = dim(P)− 1, then F is a facet of P.

Consequences: The faces of a polyhedron are polyhedra and a polyhedron has a finitenumber of faces.

Theorem: If P is full dimensional, a valid inequality is necessary for the description of Pif and only if it defines a facet of P.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 5 / 27

Example

Consider the polyhedron P ⊂ R2 described by:

x1 + 2x2 ≤ 4 (1)

−x1 − 2x2 ≤ −3 (2)

−x1 + x2 ≤3

2(3)

x1 ≤ 3 (4)

x1 ≥ 1 (5)

Verify that P is full dimensional.

Establish which inequalities define facets of P or are redundant.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 6 / 27

How to show that a valid inequality is facet defining

Consider X ⊂ Z n+ and a valid inequality πtx ≤ π0 for X

Assumption: conv(X ) is bounded and full dimensional

Two simple approaches to show that πtx ≤ π0 defines a facet of P = conv(X ):

1) Direct approach (definition): Find n points x1, . . . , xn ∈ X that satisfy theinequality with equality (πtx = π0) and are affinely independent.

2) Indirect approach:

(i) Select t points x1, . . . , x t ∈ X , with t ≥ n, that satisfy π x = π0. Suppose that theyall belong to a generic hyperplane µtx = µ0.

(ii) Solve the linear system

n∑j=1

µjxkj = µ0 for k = 1, . . . , t

with the n + 1 unknowns µ0, µ1, . . . , µn.

(iii) If the only solution is (µ, µ0) = λ(π, π0) with λ 6= 0, then the inequality π x = π0

defines a facet of conv(X ).

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 7 / 27

Example:

Consider X = {(x , y) ∈ Rm × {0, 1} :∑m

i=1 xi ≤ my , 0 ≤ xi ≤ 1 ∀i}

i) Verify that dim(conv(X )) = m + 1.

ii) Show with the indirect approach that the valid inequality xi ≤ y defines a facet ofconv(X ).

Consider the points (0, 0), (e i , 1) and (e i + e j , 1) for j 6= i which are feasible and satisfyxi = y .

Since (0, 0) belongs to the hyperplane defined by∑m

k=1 µkxk + µm+1y = µ0, then µ0 = 0.

Since (e i , 1) belongs to the hyperplane defined by∑m

k=1 µkxk + µm+1y = µ0, thenµi = −µm+1.

Since (e i + e j , 1) belongs to the hyperplane defined by∑m

k=1 µkxk − µiy = µ0, thenµj = 0 for j 6= i .

Thus the hyperplane is µixi − µiy = 0 and the inequality xi ≤ y defines a facet ofconv(X ).

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 8 / 27

3.8.2 Cover inequalities for the binary knapsack problem

Consider X = {x ∈ {0, 1}n :∑n

j=1 ajxj ≤ b} with b > 0 and N = {1, . . . , n}.

Assumptions: For each j with 1 ≤ j ≤ n, aj > 0 (if aj < 0 set x ′j = 1− xj) and aj ≤ b.

Definition: A subset C ⊆ N is a cover for X if∑

j∈C aj > b. A cover is minimal if, foreach j ∈ C , C \ {j} is not a cover.

Example: For X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}two minimal covers are: {1, 2, 3} and {3, 4, 5, 6}

Proposition: If C is a cover for X , the inequality∑j∈C

xj ≤ |C | − 1

is valid for X , and is called a cover inequality.

Example: For X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}some minimal cover inequalities: x1 + x2 + x3 ≤ 2, x1 + x2 + x6 ≤ 2, x1 + x5 + x6 ≤ 2

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 9 / 27

Can such cover inequalities be strengthened?

Proposition: If C is a cover for X , the extended cover inequality∑j∈E(C)

xj ≤ |C | − 1

is valid for X , where E(C) = C ∪ {j ∈ N : aj ≥ ai for all i ∈ C}.

Example: Consider X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}.The extended cover inequality for C = {3, 4, 5, 6} is

x1 + x2 + x3 + x4 + x5 + x6 ≤ 3

which clearly dominatesx3 + x4 + x5 + x6 ≤ 3. (6)

Observation: Since a1 = 11, ai ≥ 5 for i ∈ {3, 4, 5}, a6 = 4 and b = 19, if x1 = 1 atmost one of the other variables in (6) can take value 1 and the inequality

2x1 + x3 + x4 + x5 + x6 ≤ 3

is valid and in turn dominates (6).

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 10 / 27

Lifting procedure: to strengthen such valid inequalities to obtain facet-defining ones

Let j1, . . . , jr be the indices of N \ C and set t = 1.

Let∑t−1

i=1 αji xji +∑

j∈C xj ≤ |C | − 1 be the inequality obtained at iteration t − 1.

At iteration t: Determine the maximum value of αjt such that

αjt xjt +t−1∑i=1

αji xji +∑j∈C

xj ≤ |C | − 1

is valid for X by solving the (binary knapsack) problem

σt = max∑t−1

i=1 αji xji +∑

j∈C xjs.t.

∑t−1i=1 aji xji +

∑j∈C ajxj ≤ b − ajt

x ∈ {0, 1}|C |+t−1

and by setting αt = |C | − 1− σt .

Terminate when t = r .

N.B.: σt = maximum amount of ”space” used up by the variables of indicesC ∪ {j1, . . . , jt−1} when xjt = 1.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 11 / 27

Example:

Consider X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}.Applying the lifting procedure to

x3 + x4 + x5 + x6 ≤ 3

considering in the order x1, x2 and x7, we obtain the valid inequality

2x1 + x2 + x3 + x4 + x5 + x6 ≤ 3.

Proposition: If C is a minimal cover and ai ≤ b ∀i ∈ N, the lifting procedure isguaranteed to yield a facet-defining inequality of conv(X ).

Example cont.:

The valid inequality2x1 + x2 + x3 + x4 + x5 + x6 ≤ 3

defines a facet of conv(X ).

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 12 / 27

Separation of cover inequalities

Separation problem: Given a fractional solution x∗ with 0 ≤ x∗j ≤ 1, 1 ≤ j ≤ n,

decide whether x∗ satisfies all the cover inequalities or determine a one violated by x∗.

Since∑

j∈C xj ≤ |C | − 1 can be written as∑

j∈C (1− xj) ≥ 1, we have to answer thequestion:

Does there exist a subset C ⊆ N such that∑

j∈C aj > b and∑

j∈C (1− x∗j ) < 1?

If z ∈ {0, 1}n is the incidence vector (binary characteristic vector) of the subset C ⊆ N,it is equivalent to the question:

ζ∗ = min{∑

j∈N(1− x∗j )zj :∑

j∈N ajzj > b, z ∈ {0, 1}n} < 1?

Proposition:

(i) If ζ∗ ≥ 1, x∗ satisfies all the cover inequalities.

(ii) If ζ∗ < 1 with optimal solution z∗, then∑

j∈C xj ≤ |C | − 1

with C = {j : z∗j = 1, 1 ≤ j ≤ n}, cuts x∗ by a quantity 1− ζ∗.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 13 / 27

Example:

Considermax z = 5x1 + 2x2 + x3 + 8x4s.t. 4x1 + 2x2 + 2x3 + 3x4 ≤ 4

x1, x2, x3, x4 ∈ {0, 1}Optimal solution of the LP relaxation x∗LP = (1/4, 0, 0, 1) with z∗LP = 9.25.

The separation problem amounts to the following binary knapsack problem:

ζ∗ = min 34z1 + z2 + z3

s.t. 4z1 + 2z2 + 2z3 + 3z4 > 4z1, z2, z3, z4 ∈ {0, 1}

where the > constraint can be replaced with 4z1 + 2z2 + 2z3 + 3z4 ≥ 5.

Optimal solution z = (1, 0, 0, 1) with ζ∗ = 34.

Thus the cover inequalityx1 + x4 ≤ 1

cuts away the current LP optimal solution x∗LP by 1− ζ∗ = 14.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 14 / 27

3.8.3 Strong valid inequalities for the MaximumIndependent Set problem

Definitions

- A subset S ⊆ V is an independent/stable set if {i , j} /∈ E for each pair of nodesi , j ∈ S .

- A subset K ⊆ V is a clique if {i , j} ∈ E for each pair of nodes i , j ∈ K , .

- A subset S ⊆ V is a maximal independent/stable set (in the sense of inclusion) ifS ∪ {i} is not independent/stable for each i ∈ V \ S .

- A subset K ⊆ V is a maximal clique if K ∪ {i} is not a clique for each i ∈ V \ S .

Illustrations:

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 15 / 27

Maximum Independent Set problem (MIS): (maximum stable set)

Given an undirected G = (V ,E), determine an independent set S ⊆ V of maximum size.

Proposition: MIS is NP-hard.

MIS has a wide range of interesting applications.

For each i ∈ V , let the binary variable xi indicate whether i ∈ S .

ILP formulation:

max∑

i∈V xi (7)

s.t. xi + xj ≤ 1 ∀ {i , j} ∈ E (8)

xi ∈ {0, 1} ∀i ∈ V (9)

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 16 / 27

Definition:The bounded polyhedron

PIS = conv({x ∈ {0, 1}|V | : xi + xj ≤ 1, ∀ {i , j} ∈ E})

is the independent set polytope.

PIS = conv(X ) where X = {x ∈ {0, 1}|V | : x incidence vector of an independent set of G}.

Observation: dim(PIS) = n = |V |

Indeed: since 0 and the n unit vectors e i , 1 ≤ i ≤ n, belong to X , there are n + 1 affinelyindependent vectors in X .

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 17 / 27

1) Clique inequalities

Property: For any clique K ⊆ V of G , the clique inequality∑i∈K

xi ≤ 1

is valid for X .

Observation: The undominated clique inequalities are those corresponding to themaximal cliques.

Consider two cliques K and K ′ with K ⊂ K ′, then∑

i∈K ′ xi ≤ 1 clearly dominates∑i∈K xi ≤ 1.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 18 / 27

Proposition: A clique inequality ∑i∈K

xi ≤ 1 (10)

defines a facet of PIS if and only if K is a maximal clique in G .

Proof:

Suppose K = {1, . . . , k}. Clearly the vectors e i with i = 1, . . . , k, satisfy (10) withequality.

Since K is a maximal clique, for each i 6∈ K , there is a node j(i) ∈ K such that{i , j(i)} 6∈ E .

Denote by x i the binary vector with components i and j(i) equal to 1 and all others to 0.

Then x i ∈ X and it satisfies (10) with equality for i = k + 1, . . . , n.

Since e1, . . . , ek , xk+1, . . . , xn are linearly (affinely) independent, inequality (10) is facetdefining.

Conversely, if K is not maximal ∃ i 6∈ K such that K ∪ {i} is a clique and thus∑l∈K∪{i} xi ≤ 1 is valid for PIS .

Since (10) is the sum of −xi ≤ 0 and∑

l∈K∪{i} xi ≤ 1, it is not facet defining.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 19 / 27

Separation problem for clique inequalities:

Given a fractional optimal solution x∗ of the current linear relaxation of (7)-(9)

max∑

i∈V xi

s.t. xi + xj ≤ 1 ∀ {i , j} ∈ E

constraints (10) generated so far

xi ≥ 0 ∀i ∈ V ,

find a clique inequality that is violated by x∗ or establish that no such inequality exists.

Separation procedure: Given x∗ with 0 ≤ x∗i ≤ 1 for all i ∈ V as above, assign a weight

x∗i to each node i ∈ V and look for a clique K∗ ⊆ V of maximum total weight.

Indeed, if∑

i∈K∗ x∗i > 1, the clique inequality∑

i∈K∗ xi ≤ 1 is violated by x∗,

otherwise all those not yet introduced in the partial ILP formulation are satisfied by x∗.

Since the this separation problem is NP-hard, heuristics are used.

N.B.: The larger class of so-called orthonormal representation (OR) valid inequalities,includes the clique inequalities and can be separated in polynomial time.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 20 / 27

2) Odd hole inequalities

Definition: A subset H ⊆ V is a hole of G if the induced subgraph H is a simple cycle,i.e., the nodes in H are connected via a cycle and for each pair of non adjacent nodes i , jwe have {i , j} /∈ E .

Example: outer cycle of a wheel with n = 6 nodes

Proposition: For each hole with an odd number of nodes, the odd hole inequality∑i∈H

xi ≤ b|H|2c =|H| − 1

2

is valid for X .

We cannot select more than half of the nodes of each hole; for odd holes the upperbound is |H|−1

2.

Observation: The inequalities corresponding to even holes and odd holes with 3 nodes,are implied by (8) and (10).

An odd hole with 3 nodes is a clique; every even hole inequality can be obtained byaggregating the inequalities (8) associated to all the edges of the hole with multipliers 1

2.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 21 / 27

Separation problem: Given x∗ optimal solution of the current linear relaxation, find anodd hole inequality that is violated by x∗ or establish that no such inequality exists.

Proposition: Odd hole inequalities can be separated by computing a shortest pathbetween n = |V | pairs of nodes in an ad hoc bipartite graph with appropriate edge costs.

Separation algorithm:

Given an optimal solution x∗ as above, we look for a violated odd hole inequality.

Construct an auxiliary bipartite graph G ′ = (V ′,E ′), including each node i ∈ V of G inV1 and a copy i ′ in V2.

For each edge {i , j} ∈ E , include in E ′ (G ′) two edge {i , j ′} and {i ′, j}, where i , j ∈ V1

and i ′, j ′ ∈ V2.

To each edge {i , j ′} ∈ E ′ we assign a cost 1− x∗i − x∗j .

This way, a cycle C in G ′ has cost∑{i,j}∈C

(1− x∗i − x∗j ) = |C | − 2∑

i∈V ′(C)

x∗i = |V ′(C)| − 2∑

i∈V ′(C)

x∗i ,

where V ′(C) is the set of nodes of C in G ′.

If such cost < 1, then∑

i∈V (C) xi ≤ |V (C)|−12

is violated by x∗.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 22 / 27

To consider all possible odd holes, for each i ∈ V1 it suffices to look for a minimum costpath p∗i from node i ∈ V1 to its copy i ′ ∈ V2.

- If p∗i is a hole and its cost is < 1, we have a violated inequality.

- If p∗i is not a hole and its cost < 1, then it contains an odd subcycle whose cost isnecessarily smaller than 1 (since the costs x∗i ≥ 0), which corresponds to a violatedodd hole inequality.

- If all the paths found have a costs ≥ 1, x∗ satisfy all the odd hole inequalities.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 23 / 27

Example:

G = (V ,E) is the wheel with 6 nodes and 10 edges (5 nodes in the outer cycle/hole).

i) List all the maximal clique inequalities.

ii) Indicate the unique optimal solution x∗LP of the linear programming relaxationincluding all the maximal cliques inequalities.

iii) Determine the violated odd hole inequality.

iv) Lift this odd hole inequality so that it defines a facet of the Independent Set polytopePIS .

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 24 / 27

3.8.4 Equivalence between separation and optimization

Consider a family of LPs min{ c tx : x ∈ Po} with o ∈ O, wherePo = { x ∈ Rno : Aox ≥ bo } polytope with rational (integer) coefficients and a verylarge (e.g., exponential) number of constraints.

Examples:

1) Linear relaxation of asymmetric TSP with cut-set inequalities (O set of all graphs)

2) Maximum Matching problem: For each G = (V ,E), the matching polytope

conv({x ∈ {0, 1}|E | :∑e∈δ(i)

xe ≤ 1, ∀i ∈ V })

coincides (Edmonds) with

{x ∈ R|E |+ :∑e∈δ(i)

xe ≤ 1, ∀i ∈ V ,∑

e∈E(S)

xe ≤|S | − 1

2, ∀S ⊆ V with |S | ≥ 3 odd}

We adopt a cutting plane approach where constraints are only generated if needed.

Assumption: Even though the number of constraints mo of Po is exponential in no

(e.g., O(2no )), Ao and bo are specified in a concise way (as a function of a polynomialnumber of parameters w.r.t. no).

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 25 / 27

Optimization problem: Given a rational polytope P ⊆ Rn and a rational objectivevector c ∈ Rn, find a x∗ ∈ P minimizing c tx over x ∈ P or establish that P is empty.

N.B.: we assume that P is bounded (polytope) just to avoid unbounded problems.

Separation problem: Given a rational polytope P ⊆ Rn and a rational vector x ′ ∈ Rn,establish that x ′ ∈ P or determine a rational vector π ∈ Rn such that πx < πx ′ for eachx ∈ P (indentify a cut that separate x ′ from P).

Theorem: (consequence of Grotschel, Lovasz, Schriver 1988 theorem)

The separation problem for a family of polyhedra can be solved in polynomial time in n elog U if and only if the optimization for that family can be solved in polynomial time in nand log U, where U is an upper bound on all aij and bi .

Proof based on the Ellipsoid method, first LP polynomial algorithm (Khachiyan 1979).

For now theoretical tool: the resulting algorithm is not efficient but the equivalence mayguide the search for more practical polynomial-time algorithms.

Corollary: The linear relaxation of the ILP formulation for ATSP with cut-set inequalitiescan be solved in polynomial time in spite of the exponential number of constraints.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 26 / 27

3.8.5 Remarks on cutting plane methods

Consider a generic discrete optimization problem

min{ c tx : x ∈ X ⊆ Rn+}

with rational coefficients ci .

When designing a cutting plane method, we should be aware that:

It can be difficult to describe one or more families of strong (possibly facet defining)valid inequalities for conv(X ).

The separation problem for a given family F may require a considerablecomputational effort (if NP-hard we devise heuristics).

Even when finite convergence is guaranteed (e.g., with Gomory cuts), pure cuttingplane methods tend to be very slow.

To solve hard/large problems optimally, it is often necessary to embed strong validinequalities into a Branch-and-Bound framework, leading to a so-called Branch and Cutmethod.

The subfield of Discrete Optimization studying the polyhedral structure of the idealformulations (conv(X )) is known as Polyhedral Combinatorics.

Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 27 / 27