Introduction to Lemke’s Method (Scheme I)
description
Transcript of Introduction to Lemke’s Method (Scheme I)
Introduction to Lemke’s Method (Scheme I)
This algorithm works with a principal rearrangement of the ALCP.
q =
[q0
q
]and M =
[0 −dT
d M
].
Instead of using λ, θ, and σ, we use q0, z0 and w0, respectively. Thus
we have
w0 = q0 + 0 · z0 − dTz ≥ 0, z0 ≥ 0, z0w0 = 0
w = q + dz0 + Mz ≥ 0, z ≥ 0, zTw = 0.
We assume that the constant column q 6≥ 0 and the covering vector
d > 0. Then the smallest positive z0 such that q + dz0 ≥ 0 for all
z0 ≥ z0 is given by
z0 = maxi{−qi/di}.
Recall the theorem stating that the ALCP always has a solution.
The algorithm we are about to study proves this constructively under
the assumption that the system of equations[
w0
w
]=
[q0
q
]+
[0 −dT
d M
] [z0
z
]
is nondegenerate.
Statement of Lemke’s Method (Scheme I)
Algorithm (Lemke, Scheme I – Augmented LCP)
Step 0. Initialization. Input (q, M). If q ≥ 0, then stop: z = 0 solves
(q, M). Otherwise, let z0 = maxi{−qi/di}. Let wr denote
the (unique, by the nondegeneracy assumption) component
of w that equals zero when z0 = z0. Pivot 〈wr, z0〉. Choose
the driving variable to be the complement of wr, namely zr.
Step 1. Determination of the blocking variable. Use the minimum
ratio test to find the basic variable that blocks the increase
of the driving variable. Stop if w0 is the blocking variable.
(Interpret this outcome, if possible.)
Step 2. Pivoting. The driving variable is blocked.
• If z0 is the blocking variable, then pivot
〈z0, driving variable 〉
and stop. A solution to (q, M) is at hand.
• If some other variable blocks the driving variable, then
pivot
〈 blocking variable, driving variable 〉.
Return to Step 1 using the complement of the most recent
blocking variable as the new driving variable.
Justification of Lemke’s Method (Scheme I)
Existence of pivot elements
After the first pivot in Lemke’s algorithm, all basic variables (and only
basic variables) are eligible to block the driving variable.
But why does blocking always occur?
Lemma. Let M denote a square matrix of the form
M =
[aT δ
B c
]
in which a ≤ 0, δ < 0, and B is nonsingular. Then the pivotal
transform of M obtained by using B as the pivot block has at least
one negative entry in its last column.
Proof. The indicated pivotal transform is[
aTB−1 δ − aTB−1c
B−1 −B−1c
]
If −B−1c ≥ 0, then the hypotheses imply δ − aTB−1c < 0.
Theorem. In Lemke’s Algorithm, when w0 and z0 are basic, the
column of the driving variable contains at least one negative entry.
Proof. Consider the w0 row and the set of rows corresponding to basic
z-variables (including the artificial variable, z0).
Let α be the index set of z-variables (including z0) that are currently
basic. Let β be the index set of the w-variables that are currently
nonbasic.
These index sets have the same cardinality, but differ by just one
element, r: α will contain the index 0, and β will contain the index
of the most recent blocking variable (which is now nonbasic). The
submatrix B in M of the lemma corresponds to Mαβ. The remaining
column of M corresponds to Mγ r of the current driving variable.
This is illustrated in the next panel.
[aT δ
B c
]−→
[aTB−1 δ − aTB−1c
B−1 −B−1c
]
1 z0 zα zr zρ
w0 q0 0 −dTα −dT
r −dTρ
wβ qβ dβ Mβα Mβ r Mβρ
wρ qρ dρ Mρα Mρ r Mρρ
aT =[0 − dT
α
], δ = −dT
r B =[
dβ Mβα
]c = Mβ r.
Choosing the covering vector
To initialize the algorithm, it is enough to choose the column vector d
so as to make the n×2 matrix [d, q] have lexicographically nonnegative
rows.
Nevertheless, it is customary to let d = e, the vector of ones.
The choice of the covering vector can affect the number of iterations
of the algorithm.
Justification of Lemke’s Method (Continued)
Almost complementary paths
Making z0 ≥ z0 and z = 0 yields a set of “feasible solutions” to the
ALCP (q, M).
These points constitute what is called the primary ray. Points along
the primary ray satisfy the conditions
z0w0 > 0 and ziwi = 0 for all i 6= 0.
Thus, they are almost complementary in the sense that complementary
slackness holds for all but one index.
In between almost complementary extreme points, we have almost
complementary edges.
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
d
d
d d
dd
d
d
d d
Theorem. When applied to a nondegenerate instance of the Aug-
mented LCP (q, M), Lemke’s Algorithm terminates in finitely many
steps.
Introduction to Lemke’s Method—New Streamlined Format
w = q + dz0 + Mz ≥ 0, z0 ≥ 0, z ≥ 0, zTw = 0.
We denote this system by (q, d, M). A solution of (q, d, M) with
z0 = 0 furnishes a solution to the original LCP (q, M).
The version of Lemke’s Scheme I for (q, d, M) is a simple variant of
the one for (q, M).
Secondary rays
Without the constraint w0 ≥ 0 that is used in (q, M), we have to
allow for the possibility that the column of a driving variable may be
nonnegative and hence that the variable may be unblocked.
Such a situation corresponds to the case where q0 is large and w0 is
the blocking variable in the earlier version of Lemke’s Algorithm for the
full augmented LCP. In the event that no blocking variable is found
when a driving variable is increased, a secondary ray is generated. A
secondary ray is also an almost complementary edge of FEA(q, d, M),
but it is necessarily unbounded.
Algorithm (Lemke—Streamlined Version).
Step 0. Initialization. Input (q, d, M). If q ≥ 0, then stop: z = 0 solves(q, M). Otherwise, let z0 = maxi{−qi/di}. Let wr denote the (unique,by the nondegeneracy assumption) component of w that equals zerowhen z0 = z0. Pivot 〈wr, z0〉. Choose zr as the driving variable.
Step 1. Determination of the blocking variable (if any). If the driving variableis blocked, use the minimum ratio test to determine the basic variablethat blocks the increase of the driving variable. If the driving variableis unblocked, then stop. (Interpret this outcome, if possible.)
Step 2. Pivoting. The driving variable is blocked.
• If z0 blocks the driving variable, then pivot 〈z0, driving variable 〉and stop. A solution to (q, M) is at hand.
• If some other variable blocks the driving variable, then pivot
〈 blocking variable, driving variable 〉.
Return to Step 1 using the complement of the most recent block-ing variable as the new driving variable.
Miscellaneous Comments on Lemke’s Algorithm
(a) The n-vector e whose components are all 1 is often used as the
default covering vector. Other choices are possible, however. The
choice of the covering vector can drastically affect the outcome
of the procedure.
(b) After the initial pivot which makes the artificial variable z0 basic,
there is one (and only one) complementary pair of nonbasic vari-
ables, initially, wr and zr are such a pair. In general, the nonbasic
complementary variables make up the nonbasic pair.
(c) When z0 ≥ z0, all the basic variables are nonnegative and are
eligible to block the driving variable. In general, after a pivot
occurs, the new driving variable is the complement of the variable
that just became nonbasic.
(d) One can implement the algorithm in a “revised simplex method
fashion.” Only two columns are needed to execute the minimum
ratio test in Step 1. These are the updated constant column and
the updated column of the current driving variable. Thus, if the
current basis is known, these updated columns can be determined
by solving two systems of linear equations.
(e) There are certain classes of matrices for which the nonexistence
of a blocking variable implies the infeasibility of the given LCP
(q, M).
Example Using Lemke’s Algorithm 1 z0 z1 z2 z3
w1 −3 1 0 −1 2
w2 6 1 2 0 −2
w3 −1 1 −1 1 0
1 w1 z1 z2 z3
z0 3 1 0 1 −2
w2 9 1 2 1 −4
w3 2 1 −1 2 −2
1 w1 w3 z2 z3
z0 3 1 0 1 −2
w2 13 3 −2 5 −8
z1 2 1 −1 2 −2
Example Continued 1 w1 w3 z2 z1
z0 1 0 1 −1 1
w2 5 −1 2 −3 4
z3 1 12 −1
2 1 −12
1 w2 w3 z2 z1
z0 1 0 1 −1 1
w1 5 −1 2 −3 4
z3 3 −12
12 −1
232
1 w2 w3 z0 z1
z2 1 0 1 −1 1
w1 2 −1 −1 3 1
z3 3 −12 0 1
2 1
Thus, we have found the solution (z1, z2, z3) = (0, 1, 3).
Some Theory Behind Lemke’s Algorithm
Theorem. When applied to a nondegenerate instance of (q, d, M),
Lemke’s Algorithm (Streamlined) terminates after finitely many steps
with either a secondary ray or else a complementary feasible solution
of (q, d, M) and hence with a solution of (q, M).
Proof. This is just like the proof of finiteness in the unstreamlined case.
Termination on a secondary ray corresponds to the situation where the
latter algorithm terminates with w0 as the blocking variable.
Consequences of termination on a secondary ray
Theorem. If Lemke’s Algorithm applied to (q, d, M) terminates with a sec-ondary ray, then M reverses the sign of some nonzero nonnegative vector; thatis
zi(Mz)i ≤ 0 i = 1, . . . , n
for some nonzero nonnegative vector z.
Proof. We are working with the system
w = q + dz0 + Mz, w ≥ 0, z0 ≥ 0, z ≥ 0.
Suppose the algorithm reaches an almost-complementary extreme point (basicfeasible solution) (w, z0, z) at which we have z0 > 0 and wizi = 0 for all i > 0.
If the driving variable is unblocked, we have termination on a secondary ray.Hence there is a nonzero solution (w, z0, z) of
w = dz0 + Mz, w ≥ 0, z0 ≥ 0, z ≥ 0.
Moreover, we have
(w + λw)i(z + λz)i = 0 for all i = 1, . . . , n and all λ ≥ 0.
Now z 6= 0, for otherwise z0 > 0 (else (w, z0, z) is zero). But z0 > 0 impliesw > 0 and hence that the secondary ray is the primary ray, which is impossible.
For i = 1, . . . , n and all λ ≥ 0, we have
wizi + λwizi + λwizi + λ2wizi = 0
which implieswizi = wizi = wizi = wizi = 0
Thus for each i = 1, . . . n,
0 = ziwi = zidiz0 + zi(Mz)i.
Since zidiz0 ≥ 0, we have the asserted result: there exists a nonnegative nonzerovector z such that
zi(Mz)i ≤ 0 for all i = 1, . . . , n.
More Theory of Lemke’s Method
Theorem. Lemke’s Algorithm will solve any nondegenerate LCP
(q, M) such that M ∈ E.
In particular, Lemke’s Algorithm will solve any nondegenerate LCP
(q, M) such that M ∈ P or M ∈ SCP (strictly copositive).
Proof. It is impossible for a strictly semimonotone matrix to reverse
the sign of a nonzero, nonnegative vector.
Recall that a matrix M ∈ Rn×n is said to be semimonotone if
[0 6= x ≥ 0] ⇒ [xk > 0, (Mx)k ≥ 0 for some k]
The class of semimonotone matrices is denoted E0.
Theorem. Let M ∈ Rn×n ∩ E0. If Lemke’s Algorithm applied to
(q, d, M) terminates with a secondary ray, then SOL(0, M) 6= {0}.
Proof. From the proof of the theorem on consequences of termination
on a ray, we know that there exists a nonzero nonnegative vector
(z0, z) such that w = dz0 + Mz ≥ 0 and zTw = 0. Since M ∈ E0
we cannot have z0 > 0, for otherwise, 0 = wσ = dσz0 + Mσσzσ
where σ = supp(z). Hence Mσσzσ < 0. But now it is clear that
z ∈ SOL(0, M).
Results for copositive matrices
Corollary. If M ∈ CP and Lemke’s Algorithm applied to (q, d, M)
terminates with a secondary ray, then the vector y = z/‖z‖ satisfies:
(i) 0 6= y ∈ SOL(0, M), (ii) qTy < 0.
Proof. Since SOL(0, M) is clearly a closed cone, it is clear that (i)
holds. As for (ii), note that for all λ ≥ 0 we have
0 = (z + λz)T(q + dz + M(z + λz)) ≥ (z + λz)T(q + dz0).
Dividing through by ‖z + λz‖ and letting λ −→ ∞ we obtain
yT(q + dz0) ≤ 0. Since yTdz0 > 0 we obtain (ii).
Theorem. If M is copositive and q ∈ (SOL(0, M))∗, then Lemke’s
Algorithm will compute a solution of the LCP (q, M) if the problem
is nondegenerate.
Proof. This is just a corollary of the corollary.
Constructive proof that CP+ ⊂ Q0
Theorem. If M ∈ CP+ and Lemke’s Algorithm applied to a nonde-
generate instance of the problem (q, d, M) terminates on a secondary
ray, then the LCP (q, M) is infeasible.
Proof. If the algorithm terminates on a secondary ray, we have
0 = zTw = zTdz0 + zTMz.
Since M is copositive, both terms on the right are nonnegative, hence
both are zero. But since M ∈ CP+, we have (M + MT)z = 0.
We also know that z0 = 0, hence Mz ≥ 0 and MTz ≤ 0. The latter
is equivalent to zTM ≤ 0. Moreover zTq < 0; thus
z ≥ 0, zTM ≤ 0, zTq < 0.
These conditions imply that FEA(q, M) = ∅.
Can Lemke’s Algorithm find global minima of nonconvex QPs?
Example. Consider the quadratic program
minimize 12x1 − 1
2x2 − 12x
21 + 1
2x22
subject to 2x1 + x2 ≤ 6
−x1 + 4x2 ≤ 6
x1 ≥ 0, x2 ≥ 0
The problem has a nonempty compact constraint set, hence a global optimum.
The KKT conditions give the LCP (q, M) where
q =
12
−12
6
6
and M =
−1 0 2 −1
0 1 1 4
−2 −1 0 0
1 −4 0 0
Choose the covering vector d ∈ R4++ arbitrarily. Then scale it so that d2 = 1.
1 z0 x1 x2 y3 y4
y112 d1 −1 0 2 −1
y2 −12 1 0 1 1 4
x3 6 d3 −2 −1 0 0
x4 6 d4 1 −4 0 0
The pivot to perform is 〈y2, z0〉. It leads to the tableau
1 z0 x1 x2 y3 y4
y112 d1 −1 0 2 −1
y2 −12 1 0 1 1 4
x3 6 d3 −2 −1 0 0
x4 6 d4 1 −4 0 0
The pivot to perform is 〈y2, z0〉. It leads to the tableau
1 y2 x1 x2 y3 y4
y112(1 + d1) d1 −1 −d1 2 − d1 −1 − 4d1
z012 1 0 −1 −1 −4
x312(12 + d3) d3 −2 −1 − d3 −d3 −4d3
x412(12 + d4) d4 1 −4 − d4 −d4 −4d4
The pivot to perform is 〈z0, x2〉. It leads to the tableau
1 y2 x1 x2 y3 y4
y112(1 + d1) d1 −1 −d1 2 − d1 −1 − 4d1
z012 1 0 −1 −1 −4
x312(12 + d3) d3 −2 −1 − d3 −d3 −4d3
x412(12 + d4) d4 1 −4 − d4 −d4 −4d4
The pivot to perform is 〈z0, x2〉. It leads to the tableau
1 y2 x1 z0 y3 y4
y112 0 −1 d1 2 −1
x212 1 0 −1 −1 −4
x3112 −1 −2 1 + d3 1 4
x4 4 −4 1 4 + d4 4 16
1 y2 x1 z0 y3 y4
y112 0 −1 d1 2 −1
x212 1 0 −1 −1 −4
x3112 −1 −2 1 + d3 1 4
x4 4 −4 1 4 + d4 4 16
The solution of the KKT conditions found is (x1, x2, y3, y4) = (0, 12 , 0, 0).
The point (x1, x2) = (0, 12) is a local minimum.
The global minimum is (x1, x2) = (3, 0).
It cannot be reached by Lemke’s Algorithm, no matter what covering vector isused.