3.8 Strong valid inequalities - Intranet...
Transcript of 3.8 Strong valid inequalities - Intranet...
3.8 Strong valid inequalities
By studying the problem structure, we can derive strong valid inequalities which lead tobetter approximations of the ideal formulation conv(X ) and hence to tighter bounds.
Two definitions related to polyhedra and their description:
Consider a polyhedron P = {x ∈ Rn+ : Ax ≤ b}.
Definition: Given two valid inequalities πtx ≤ π0 and µtx ≤ µ0 for P, πtx ≤ π0
dominates µtx ≤ µ0 if ∃ u > 0 such that uµ ≤ π and π0 ≤ uµ0 with (π, π0) 6= (uµ, uµ0).
Poiche uµtx ≤ πtx ≤ π0 ≤ uµ0, chiaramente {x ∈ Rn+ : πtx ≤ π0} ⊆ {x ∈ Rn
+ : µtx ≤ µ0}
Example: x1 + 3x2 ≤ 4 dominates 2x1 + 4x2 ≤ 9 since for (π, π0) = (1, 3, 4) and
(µ, µ0) = (2, 4, 9) we have 12µ ≤ π and π0 ≤ 1
2µ0.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 1 / 27
Definition: A valid inequality πtx ≤ π0 is redundant in the description of P
if there exist k ≥ 2 valid inequalities πix ≤ πi0 for P with ui > 0, 1 ≤ i ≤ k, such that
(k∑
i=1
uiπi )x ≤
k∑i=1
uiπi0 dominates πtx ≤ π0.
Example:
P = {(x1, x2) ∈ R2+ : −x1 + 2x2 ≤ 4, −x1 − 2x2 ≤ −3, −x1 + x2 ≤ 5/3, 1 ≤ x1 ≤ 3}
−x1 + x2 ≤ 5/3 is redundant because it is dominated by −x1 + x2 ≤ 3/2, which isimplied by −x1 + 2x2 ≤ 4 and −x1 ≤ −1 (with u1 = u2 = 1
2)
Observation: When P = conv(X ) is not known explicitly it can be very difficult to checkredundancy. In pratice, we should avoid inequalities that are dominated by other alreadyavailable inequalities.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 2 / 27
3.8.1 Faces and facets of polyhedra
Consider a polyhedron P = {x ∈ Rn : Ax ≤ b}
Definitions
The vectors x1, . . . , xk are affinely independent if the k − 1 vectorsx2 − x1, . . . , xk − x1 are linearly independent.
The dimension of P, dim(P), is equal to the maximum number of affinely linearlyindependent points of P minus 1.
P is full-dimensional if dim(P) = n, i.e., no equation atx = b is satisfied withequality by all the points x ∈ P.
Illustrations:
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 3 / 27
For the sake of simplicity, we assume that P is full dimensional
Theorem: If P is of full dimension, P admits a unique minimal description
P = {x ∈ Rn : ati x ≤ bi , i = 1, . . . ,m}
where each inequality is unique within a positive multiple.
Each inequality is necessary: deleting anyone of them we obtain a polyhedron that differsfrom P.
Moreover, each valid inequality for P which is not a positive multiple of one of theati x ≤ bi is redundant (can be obtained as linear combination with non negative
coefficients of two or more valid inequalities).
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 4 / 27
Alternative characterization of necessary inequalities
Definitions
Let F = {x ∈ P : πtx = π0} for any valid inequality πtx ≤ π0 for P. Then F is aface of P and the inequality πtx ≤ π0 represents or defines F .
If F is a face of P and dim(F ) = dim(P)− 1, then F is a facet of P.
Consequences: The faces of a polyhedron are polyhedra and a polyhedron has a finitenumber of faces.
Theorem: If P is full dimensional, a valid inequality is necessary for the description of Pif and only if it defines a facet of P.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 5 / 27
Example
Consider the polyhedron P ⊂ R2 described by:
x1 + 2x2 ≤ 4 (1)
−x1 − 2x2 ≤ −3 (2)
−x1 + x2 ≤3
2(3)
x1 ≤ 3 (4)
x1 ≥ 1 (5)
Verify that P is full dimensional.
Establish which inequalities define facets of P or are redundant.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 6 / 27
How to show that a valid inequality is facet defining
Consider X ⊂ Z n+ and a valid inequality πtx ≤ π0 for X
Assumption: conv(X ) is bounded and full dimensional
Two simple approaches to show that πtx ≤ π0 defines a facet of P = conv(X ):
1) Direct approach (definition): Find n points x1, . . . , xn ∈ X that satisfy theinequality with equality (πtx = π0) and are affinely independent.
2) Indirect approach:
(i) Select t points x1, . . . , x t ∈ X , with t ≥ n, that satisfy π x = π0. Suppose that theyall belong to a generic hyperplane µtx = µ0.
(ii) Solve the linear system
n∑j=1
µjxkj = µ0 for k = 1, . . . , t
with the n + 1 unknowns µ0, µ1, . . . , µn.
(iii) If the only solution is (µ, µ0) = λ(π, π0) with λ 6= 0, then the inequality π x = π0
defines a facet of conv(X ).
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 7 / 27
Example:
Consider X = {(x , y) ∈ Rm × {0, 1} :∑m
i=1 xi ≤ my , 0 ≤ xi ≤ 1 ∀i}
i) Verify that dim(conv(X )) = m + 1.
ii) Show with the indirect approach that the valid inequality xi ≤ y defines a facet ofconv(X ).
Consider the points (0, 0), (e i , 1) and (e i + e j , 1) for j 6= i which are feasible and satisfyxi = y .
Since (0, 0) belongs to the hyperplane defined by∑m
k=1 µkxk + µm+1y = µ0, then µ0 = 0.
Since (e i , 1) belongs to the hyperplane defined by∑m
k=1 µkxk + µm+1y = µ0, thenµi = −µm+1.
Since (e i + e j , 1) belongs to the hyperplane defined by∑m
k=1 µkxk − µiy = µ0, thenµj = 0 for j 6= i .
Thus the hyperplane is µixi − µiy = 0 and the inequality xi ≤ y defines a facet ofconv(X ).
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 8 / 27
3.8.2 Cover inequalities for the binary knapsack problem
Consider X = {x ∈ {0, 1}n :∑n
j=1 ajxj ≤ b} with b > 0 and N = {1, . . . , n}.
Assumptions: For each j with 1 ≤ j ≤ n, aj > 0 (if aj < 0 set x ′j = 1− xj) and aj ≤ b.
Definition: A subset C ⊆ N is a cover for X if∑
j∈C aj > b. A cover is minimal if, foreach j ∈ C , C \ {j} is not a cover.
Example: For X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}two minimal covers are: {1, 2, 3} and {3, 4, 5, 6}
Proposition: If C is a cover for X , the inequality∑j∈C
xj ≤ |C | − 1
is valid for X , and is called a cover inequality.
Example: For X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}some minimal cover inequalities: x1 + x2 + x3 ≤ 2, x1 + x2 + x6 ≤ 2, x1 + x5 + x6 ≤ 2
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 9 / 27
Can such cover inequalities be strengthened?
Proposition: If C is a cover for X , the extended cover inequality∑j∈E(C)
xj ≤ |C | − 1
is valid for X , where E(C) = C ∪ {j ∈ N : aj ≥ ai for all i ∈ C}.
Example: Consider X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}.The extended cover inequality for C = {3, 4, 5, 6} is
x1 + x2 + x3 + x4 + x5 + x6 ≤ 3
which clearly dominatesx3 + x4 + x5 + x6 ≤ 3. (6)
Observation: Since a1 = 11, ai ≥ 5 for i ∈ {3, 4, 5}, a6 = 4 and b = 19, if x1 = 1 atmost one of the other variables in (6) can take value 1 and the inequality
2x1 + x3 + x4 + x5 + x6 ≤ 3
is valid and in turn dominates (6).
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 10 / 27
Lifting procedure: to strengthen such valid inequalities to obtain facet-defining ones
Let j1, . . . , jr be the indices of N \ C and set t = 1.
Let∑t−1
i=1 αji xji +∑
j∈C xj ≤ |C | − 1 be the inequality obtained at iteration t − 1.
At iteration t: Determine the maximum value of αjt such that
αjt xjt +t−1∑i=1
αji xji +∑j∈C
xj ≤ |C | − 1
is valid for X by solving the (binary knapsack) problem
σt = max∑t−1
i=1 αji xji +∑
j∈C xjs.t.
∑t−1i=1 aji xji +
∑j∈C ajxj ≤ b − ajt
x ∈ {0, 1}|C |+t−1
and by setting αt = |C | − 1− σt .
Terminate when t = r .
N.B.: σt = maximum amount of ”space” used up by the variables of indicesC ∪ {j1, . . . , jt−1} when xjt = 1.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 11 / 27
Example:
Consider X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}.Applying the lifting procedure to
x3 + x4 + x5 + x6 ≤ 3
considering in the order x1, x2 and x7, we obtain the valid inequality
2x1 + x2 + x3 + x4 + x5 + x6 ≤ 3.
Proposition: If C is a minimal cover and ai ≤ b ∀i ∈ N, the lifting procedure isguaranteed to yield a facet-defining inequality of conv(X ).
Example cont.:
The valid inequality2x1 + x2 + x3 + x4 + x5 + x6 ≤ 3
defines a facet of conv(X ).
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 12 / 27
Separation of cover inequalities
Separation problem: Given a fractional solution x∗ with 0 ≤ x∗j ≤ 1, 1 ≤ j ≤ n,
decide whether x∗ satisfies all the cover inequalities or determine a one violated by x∗.
Since∑
j∈C xj ≤ |C | − 1 can be written as∑
j∈C (1− xj) ≥ 1, we have to answer thequestion:
Does there exist a subset C ⊆ N such that∑
j∈C aj > b and∑
j∈C (1− x∗j ) < 1?
If z ∈ {0, 1}n is the incidence vector (binary characteristic vector) of the subset C ⊆ N,it is equivalent to the question:
ζ∗ = min{∑
j∈N(1− x∗j )zj :∑
j∈N ajzj > b, z ∈ {0, 1}n} < 1?
Proposition:
(i) If ζ∗ ≥ 1, x∗ satisfies all the cover inequalities.
(ii) If ζ∗ < 1 with optimal solution z∗, then∑
j∈C xj ≤ |C | − 1
with C = {j : z∗j = 1, 1 ≤ j ≤ n}, cuts x∗ by a quantity 1− ζ∗.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 13 / 27
Example:
Considermax z = 5x1 + 2x2 + x3 + 8x4s.t. 4x1 + 2x2 + 2x3 + 3x4 ≤ 4
x1, x2, x3, x4 ∈ {0, 1}Optimal solution of the LP relaxation x∗LP = (1/4, 0, 0, 1) with z∗LP = 9.25.
The separation problem amounts to the following binary knapsack problem:
ζ∗ = min 34z1 + z2 + z3
s.t. 4z1 + 2z2 + 2z3 + 3z4 > 4z1, z2, z3, z4 ∈ {0, 1}
where the > constraint can be replaced with 4z1 + 2z2 + 2z3 + 3z4 ≥ 5.
Optimal solution z = (1, 0, 0, 1) with ζ∗ = 34.
Thus the cover inequalityx1 + x4 ≤ 1
cuts away the current LP optimal solution x∗LP by 1− ζ∗ = 14.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 14 / 27
3.8.3 Strong valid inequalities for the MaximumIndependent Set problem
Definitions
- A subset S ⊆ V is an independent/stable set if {i , j} /∈ E for each pair of nodesi , j ∈ S .
- A subset K ⊆ V is a clique if {i , j} ∈ E for each pair of nodes i , j ∈ K , .
- A subset S ⊆ V is a maximal independent/stable set (in the sense of inclusion) ifS ∪ {i} is not independent/stable for each i ∈ V \ S .
- A subset K ⊆ V is a maximal clique if K ∪ {i} is not a clique for each i ∈ V \ S .
Illustrations:
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 15 / 27
Maximum Independent Set problem (MIS): (maximum stable set)
Given an undirected G = (V ,E), determine an independent set S ⊆ V of maximum size.
Proposition: MIS is NP-hard.
MIS has a wide range of interesting applications.
For each i ∈ V , let the binary variable xi indicate whether i ∈ S .
ILP formulation:
max∑
i∈V xi (7)
s.t. xi + xj ≤ 1 ∀ {i , j} ∈ E (8)
xi ∈ {0, 1} ∀i ∈ V (9)
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 16 / 27
Definition:The bounded polyhedron
PIS = conv({x ∈ {0, 1}|V | : xi + xj ≤ 1, ∀ {i , j} ∈ E})
is the independent set polytope.
PIS = conv(X ) where X = {x ∈ {0, 1}|V | : x incidence vector of an independent set of G}.
Observation: dim(PIS) = n = |V |
Indeed: since 0 and the n unit vectors e i , 1 ≤ i ≤ n, belong to X , there are n + 1 affinelyindependent vectors in X .
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 17 / 27
1) Clique inequalities
Property: For any clique K ⊆ V of G , the clique inequality∑i∈K
xi ≤ 1
is valid for X .
Observation: The undominated clique inequalities are those corresponding to themaximal cliques.
Consider two cliques K and K ′ with K ⊂ K ′, then∑
i∈K ′ xi ≤ 1 clearly dominates∑i∈K xi ≤ 1.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 18 / 27
Proposition: A clique inequality ∑i∈K
xi ≤ 1 (10)
defines a facet of PIS if and only if K is a maximal clique in G .
Proof:
Suppose K = {1, . . . , k}. Clearly the vectors e i with i = 1, . . . , k, satisfy (10) withequality.
Since K is a maximal clique, for each i 6∈ K , there is a node j(i) ∈ K such that{i , j(i)} 6∈ E .
Denote by x i the binary vector with components i and j(i) equal to 1 and all others to 0.
Then x i ∈ X and it satisfies (10) with equality for i = k + 1, . . . , n.
Since e1, . . . , ek , xk+1, . . . , xn are linearly (affinely) independent, inequality (10) is facetdefining.
Conversely, if K is not maximal ∃ i 6∈ K such that K ∪ {i} is a clique and thus∑l∈K∪{i} xi ≤ 1 is valid for PIS .
Since (10) is the sum of −xi ≤ 0 and∑
l∈K∪{i} xi ≤ 1, it is not facet defining.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 19 / 27
Separation problem for clique inequalities:
Given a fractional optimal solution x∗ of the current linear relaxation of (7)-(9)
max∑
i∈V xi
s.t. xi + xj ≤ 1 ∀ {i , j} ∈ E
constraints (10) generated so far
xi ≥ 0 ∀i ∈ V ,
find a clique inequality that is violated by x∗ or establish that no such inequality exists.
Separation procedure: Given x∗ with 0 ≤ x∗i ≤ 1 for all i ∈ V as above, assign a weight
x∗i to each node i ∈ V and look for a clique K∗ ⊆ V of maximum total weight.
Indeed, if∑
i∈K∗ x∗i > 1, the clique inequality∑
i∈K∗ xi ≤ 1 is violated by x∗,
otherwise all those not yet introduced in the partial ILP formulation are satisfied by x∗.
Since the this separation problem is NP-hard, heuristics are used.
N.B.: The larger class of so-called orthonormal representation (OR) valid inequalities,includes the clique inequalities and can be separated in polynomial time.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 20 / 27
2) Odd hole inequalities
Definition: A subset H ⊆ V is a hole of G if the induced subgraph H is a simple cycle,i.e., the nodes in H are connected via a cycle and for each pair of non adjacent nodes i , jwe have {i , j} /∈ E .
Example: outer cycle of a wheel with n = 6 nodes
Proposition: For each hole with an odd number of nodes, the odd hole inequality∑i∈H
xi ≤ b|H|2c =|H| − 1
2
is valid for X .
We cannot select more than half of the nodes of each hole; for odd holes the upperbound is |H|−1
2.
Observation: The inequalities corresponding to even holes and odd holes with 3 nodes,are implied by (8) and (10).
An odd hole with 3 nodes is a clique; every even hole inequality can be obtained byaggregating the inequalities (8) associated to all the edges of the hole with multipliers 1
2.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 21 / 27
Separation problem: Given x∗ optimal solution of the current linear relaxation, find anodd hole inequality that is violated by x∗ or establish that no such inequality exists.
Proposition: Odd hole inequalities can be separated by computing a shortest pathbetween n = |V | pairs of nodes in an ad hoc bipartite graph with appropriate edge costs.
Separation algorithm:
Given an optimal solution x∗ as above, we look for a violated odd hole inequality.
Construct an auxiliary bipartite graph G ′ = (V ′,E ′), including each node i ∈ V of G inV1 and a copy i ′ in V2.
For each edge {i , j} ∈ E , include in E ′ (G ′) two edge {i , j ′} and {i ′, j}, where i , j ∈ V1
and i ′, j ′ ∈ V2.
To each edge {i , j ′} ∈ E ′ we assign a cost 1− x∗i − x∗j .
This way, a cycle C in G ′ has cost∑{i,j}∈C
(1− x∗i − x∗j ) = |C | − 2∑
i∈V ′(C)
x∗i = |V ′(C)| − 2∑
i∈V ′(C)
x∗i ,
where V ′(C) is the set of nodes of C in G ′.
If such cost < 1, then∑
i∈V (C) xi ≤ |V (C)|−12
is violated by x∗.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 22 / 27
To consider all possible odd holes, for each i ∈ V1 it suffices to look for a minimum costpath p∗i from node i ∈ V1 to its copy i ′ ∈ V2.
- If p∗i is a hole and its cost is < 1, we have a violated inequality.
- If p∗i is not a hole and its cost < 1, then it contains an odd subcycle whose cost isnecessarily smaller than 1 (since the costs x∗i ≥ 0), which corresponds to a violatedodd hole inequality.
- If all the paths found have a costs ≥ 1, x∗ satisfy all the odd hole inequalities.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 23 / 27
Example:
G = (V ,E) is the wheel with 6 nodes and 10 edges (5 nodes in the outer cycle/hole).
i) List all the maximal clique inequalities.
ii) Indicate the unique optimal solution x∗LP of the linear programming relaxationincluding all the maximal cliques inequalities.
iii) Determine the violated odd hole inequality.
iv) Lift this odd hole inequality so that it defines a facet of the Independent Set polytopePIS .
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 24 / 27
3.8.4 Equivalence between separation and optimization
Consider a family of LPs min{ c tx : x ∈ Po} with o ∈ O, wherePo = { x ∈ Rno : Aox ≥ bo } polytope with rational (integer) coefficients and a verylarge (e.g., exponential) number of constraints.
Examples:
1) Linear relaxation of asymmetric TSP with cut-set inequalities (O set of all graphs)
2) Maximum Matching problem: For each G = (V ,E), the matching polytope
conv({x ∈ {0, 1}|E | :∑e∈δ(i)
xe ≤ 1, ∀i ∈ V })
coincides (Edmonds) with
{x ∈ R|E |+ :∑e∈δ(i)
xe ≤ 1, ∀i ∈ V ,∑
e∈E(S)
xe ≤|S | − 1
2, ∀S ⊆ V with |S | ≥ 3 odd}
We adopt a cutting plane approach where constraints are only generated if needed.
Assumption: Even though the number of constraints mo of Po is exponential in no
(e.g., O(2no )), Ao and bo are specified in a concise way (as a function of a polynomialnumber of parameters w.r.t. no).
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 25 / 27
Optimization problem: Given a rational polytope P ⊆ Rn and a rational objectivevector c ∈ Rn, find a x∗ ∈ P minimizing c tx over x ∈ P or establish that P is empty.
N.B.: we assume that P is bounded (polytope) just to avoid unbounded problems.
Separation problem: Given a rational polytope P ⊆ Rn and a rational vector x ′ ∈ Rn,establish that x ′ ∈ P or determine a rational vector π ∈ Rn such that πx < πx ′ for eachx ∈ P (indentify a cut that separate x ′ from P).
Theorem: (consequence of Grotschel, Lovasz, Schriver 1988 theorem)
The separation problem for a family of polyhedra can be solved in polynomial time in n elog U if and only if the optimization for that family can be solved in polynomial time in nand log U, where U is an upper bound on all aij and bi .
Proof based on the Ellipsoid method, first LP polynomial algorithm (Khachiyan 1979).
For now theoretical tool: the resulting algorithm is not efficient but the equivalence mayguide the search for more practical polynomial-time algorithms.
Corollary: The linear relaxation of the ILP formulation for ATSP with cut-set inequalitiescan be solved in polynomial time in spite of the exponential number of constraints.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 26 / 27
3.8.5 Remarks on cutting plane methods
Consider a generic discrete optimization problem
min{ c tx : x ∈ X ⊆ Rn+}
with rational coefficients ci .
When designing a cutting plane method, we should be aware that:
It can be difficult to describe one or more families of strong (possibly facet defining)valid inequalities for conv(X ).
The separation problem for a given family F may require a considerablecomputational effort (if NP-hard we devise heuristics).
Even when finite convergence is guaranteed (e.g., with Gomory cuts), pure cuttingplane methods tend to be very slow.
To solve hard/large problems optimally, it is often necessary to embed strong validinequalities into a Branch-and-Bound framework, leading to a so-called Branch and Cutmethod.
The subfield of Discrete Optimization studying the polyhedral structure of the idealformulations (conv(X )) is known as Polyhedral Combinatorics.
Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 27 / 27