Introduction to Lemke’s Method (Scheme I)

30
Introduction to Lemke’s Method (Scheme I) This algorithm works with a principal rearrangement of the ALCP. ˜ q = q 0 q and ˜ M = 0 -d T d M . Instead of using λ, θ, and σ , we use q 0 ,z 0 and w 0 , respectively. Thus we have w 0 = q 0 +0 · z 0 - d T z 0,z 0 0,z 0 w 0 =0 w = q + dz 0 + Mz 0, z 0, z T w =0. We assume that the constant column q 0 and the covering vector d> 0. Then the smallest positive ¯ z 0 such that q + dz 0 0 for all z 0 ¯ z 0 is given by ¯ z 0 = max i {-q i /d i }.

description

Lemke’s Method, Linear complementarity problems, LCP, Multiple degree of freedom friction and contact, 2 dimensional friction,

Transcript of Introduction to Lemke’s Method (Scheme I)

Page 1: Introduction to Lemke’s Method (Scheme I)

Introduction to Lemke’s Method (Scheme I)

This algorithm works with a principal rearrangement of the ALCP.

q =

[q0

q

]and M =

[0 −dT

d M

].

Instead of using λ, θ, and σ, we use q0, z0 and w0, respectively. Thus

we have

w0 = q0 + 0 · z0 − dTz ≥ 0, z0 ≥ 0, z0w0 = 0

w = q + dz0 + Mz ≥ 0, z ≥ 0, zTw = 0.

We assume that the constant column q 6≥ 0 and the covering vector

d > 0. Then the smallest positive z0 such that q + dz0 ≥ 0 for all

z0 ≥ z0 is given by

z0 = maxi{−qi/di}.

Page 2: Introduction to Lemke’s Method (Scheme I)

Recall the theorem stating that the ALCP always has a solution.

The algorithm we are about to study proves this constructively under

the assumption that the system of equations[

w0

w

]=

[q0

q

]+

[0 −dT

d M

] [z0

z

]

is nondegenerate.

Page 3: Introduction to Lemke’s Method (Scheme I)

Statement of Lemke’s Method (Scheme I)

Algorithm (Lemke, Scheme I – Augmented LCP)

Step 0. Initialization. Input (q, M). If q ≥ 0, then stop: z = 0 solves

(q, M). Otherwise, let z0 = maxi{−qi/di}. Let wr denote

the (unique, by the nondegeneracy assumption) component

of w that equals zero when z0 = z0. Pivot 〈wr, z0〉. Choose

the driving variable to be the complement of wr, namely zr.

Step 1. Determination of the blocking variable. Use the minimum

ratio test to find the basic variable that blocks the increase

of the driving variable. Stop if w0 is the blocking variable.

(Interpret this outcome, if possible.)

Page 4: Introduction to Lemke’s Method (Scheme I)

Step 2. Pivoting. The driving variable is blocked.

• If z0 is the blocking variable, then pivot

〈z0, driving variable 〉

and stop. A solution to (q, M) is at hand.

• If some other variable blocks the driving variable, then

pivot

〈 blocking variable, driving variable 〉.

Return to Step 1 using the complement of the most recent

blocking variable as the new driving variable.

Page 5: Introduction to Lemke’s Method (Scheme I)

Justification of Lemke’s Method (Scheme I)

Existence of pivot elements

After the first pivot in Lemke’s algorithm, all basic variables (and only

basic variables) are eligible to block the driving variable.

But why does blocking always occur?

Page 6: Introduction to Lemke’s Method (Scheme I)

Lemma. Let M denote a square matrix of the form

M =

[aT δ

B c

]

in which a ≤ 0, δ < 0, and B is nonsingular. Then the pivotal

transform of M obtained by using B as the pivot block has at least

one negative entry in its last column.

Proof. The indicated pivotal transform is[

aTB−1 δ − aTB−1c

B−1 −B−1c

]

If −B−1c ≥ 0, then the hypotheses imply δ − aTB−1c < 0.

Page 7: Introduction to Lemke’s Method (Scheme I)

Theorem. In Lemke’s Algorithm, when w0 and z0 are basic, the

column of the driving variable contains at least one negative entry.

Proof. Consider the w0 row and the set of rows corresponding to basic

z-variables (including the artificial variable, z0).

Let α be the index set of z-variables (including z0) that are currently

basic. Let β be the index set of the w-variables that are currently

nonbasic.

These index sets have the same cardinality, but differ by just one

element, r: α will contain the index 0, and β will contain the index

of the most recent blocking variable (which is now nonbasic). The

submatrix B in M of the lemma corresponds to Mαβ. The remaining

column of M corresponds to Mγ r of the current driving variable.

This is illustrated in the next panel.

Page 8: Introduction to Lemke’s Method (Scheme I)

[aT δ

B c

]−→

[aTB−1 δ − aTB−1c

B−1 −B−1c

]

1 z0 zα zr zρ

w0 q0 0 −dTα −dT

r −dTρ

wβ qβ dβ Mβα Mβ r Mβρ

wρ qρ dρ Mρα Mρ r Mρρ

aT =[0 − dT

α

], δ = −dT

r B =[

dβ Mβα

]c = Mβ r.

Page 9: Introduction to Lemke’s Method (Scheme I)

Choosing the covering vector

To initialize the algorithm, it is enough to choose the column vector d

so as to make the n×2 matrix [d, q] have lexicographically nonnegative

rows.

Nevertheless, it is customary to let d = e, the vector of ones.

The choice of the covering vector can affect the number of iterations

of the algorithm.

Page 10: Introduction to Lemke’s Method (Scheme I)

Justification of Lemke’s Method (Continued)

Almost complementary paths

Making z0 ≥ z0 and z = 0 yields a set of “feasible solutions” to the

ALCP (q, M).

These points constitute what is called the primary ray. Points along

the primary ray satisfy the conditions

z0w0 > 0 and ziwi = 0 for all i 6= 0.

Thus, they are almost complementary in the sense that complementary

slackness holds for all but one index.

In between almost complementary extreme points, we have almost

complementary edges.

Page 11: Introduction to Lemke’s Method (Scheme I)

.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

d

d

d d

dd

d

d

d d

Theorem. When applied to a nondegenerate instance of the Aug-

mented LCP (q, M), Lemke’s Algorithm terminates in finitely many

steps.

Page 12: Introduction to Lemke’s Method (Scheme I)

Introduction to Lemke’s Method—New Streamlined Format

w = q + dz0 + Mz ≥ 0, z0 ≥ 0, z ≥ 0, zTw = 0.

We denote this system by (q, d, M). A solution of (q, d, M) with

z0 = 0 furnishes a solution to the original LCP (q, M).

The version of Lemke’s Scheme I for (q, d, M) is a simple variant of

the one for (q, M).

Page 13: Introduction to Lemke’s Method (Scheme I)

Secondary rays

Without the constraint w0 ≥ 0 that is used in (q, M), we have to

allow for the possibility that the column of a driving variable may be

nonnegative and hence that the variable may be unblocked.

Such a situation corresponds to the case where q0 is large and w0 is

the blocking variable in the earlier version of Lemke’s Algorithm for the

full augmented LCP. In the event that no blocking variable is found

when a driving variable is increased, a secondary ray is generated. A

secondary ray is also an almost complementary edge of FEA(q, d, M),

but it is necessarily unbounded.

Page 14: Introduction to Lemke’s Method (Scheme I)

Algorithm (Lemke—Streamlined Version).

Step 0. Initialization. Input (q, d, M). If q ≥ 0, then stop: z = 0 solves(q, M). Otherwise, let z0 = maxi{−qi/di}. Let wr denote the (unique,by the nondegeneracy assumption) component of w that equals zerowhen z0 = z0. Pivot 〈wr, z0〉. Choose zr as the driving variable.

Step 1. Determination of the blocking variable (if any). If the driving variableis blocked, use the minimum ratio test to determine the basic variablethat blocks the increase of the driving variable. If the driving variableis unblocked, then stop. (Interpret this outcome, if possible.)

Step 2. Pivoting. The driving variable is blocked.

• If z0 blocks the driving variable, then pivot 〈z0, driving variable 〉and stop. A solution to (q, M) is at hand.

• If some other variable blocks the driving variable, then pivot

〈 blocking variable, driving variable 〉.

Return to Step 1 using the complement of the most recent block-ing variable as the new driving variable.

Page 15: Introduction to Lemke’s Method (Scheme I)

Miscellaneous Comments on Lemke’s Algorithm

(a) The n-vector e whose components are all 1 is often used as the

default covering vector. Other choices are possible, however. The

choice of the covering vector can drastically affect the outcome

of the procedure.

(b) After the initial pivot which makes the artificial variable z0 basic,

there is one (and only one) complementary pair of nonbasic vari-

ables, initially, wr and zr are such a pair. In general, the nonbasic

complementary variables make up the nonbasic pair.

(c) When z0 ≥ z0, all the basic variables are nonnegative and are

eligible to block the driving variable. In general, after a pivot

occurs, the new driving variable is the complement of the variable

that just became nonbasic.

Page 16: Introduction to Lemke’s Method (Scheme I)

(d) One can implement the algorithm in a “revised simplex method

fashion.” Only two columns are needed to execute the minimum

ratio test in Step 1. These are the updated constant column and

the updated column of the current driving variable. Thus, if the

current basis is known, these updated columns can be determined

by solving two systems of linear equations.

(e) There are certain classes of matrices for which the nonexistence

of a blocking variable implies the infeasibility of the given LCP

(q, M).

Page 17: Introduction to Lemke’s Method (Scheme I)

Example Using Lemke’s Algorithm 1 z0 z1 z2 z3

w1 −3 1 0 −1 2

w2 6 1 2 0 −2

w3 −1 1 −1 1 0

1 w1 z1 z2 z3

z0 3 1 0 1 −2

w2 9 1 2 1 −4

w3 2 1 −1 2 −2

1 w1 w3 z2 z3

z0 3 1 0 1 −2

w2 13 3 −2 5 −8

z1 2 1 −1 2 −2

Page 18: Introduction to Lemke’s Method (Scheme I)

Example Continued 1 w1 w3 z2 z1

z0 1 0 1 −1 1

w2 5 −1 2 −3 4

z3 1 12 −1

2 1 −12

1 w2 w3 z2 z1

z0 1 0 1 −1 1

w1 5 −1 2 −3 4

z3 3 −12

12 −1

232

1 w2 w3 z0 z1

z2 1 0 1 −1 1

w1 2 −1 −1 3 1

z3 3 −12 0 1

2 1

Thus, we have found the solution (z1, z2, z3) = (0, 1, 3).

Page 19: Introduction to Lemke’s Method (Scheme I)

Some Theory Behind Lemke’s Algorithm

Theorem. When applied to a nondegenerate instance of (q, d, M),

Lemke’s Algorithm (Streamlined) terminates after finitely many steps

with either a secondary ray or else a complementary feasible solution

of (q, d, M) and hence with a solution of (q, M).

Proof. This is just like the proof of finiteness in the unstreamlined case.

Termination on a secondary ray corresponds to the situation where the

latter algorithm terminates with w0 as the blocking variable.

Page 20: Introduction to Lemke’s Method (Scheme I)

Consequences of termination on a secondary ray

Theorem. If Lemke’s Algorithm applied to (q, d, M) terminates with a sec-ondary ray, then M reverses the sign of some nonzero nonnegative vector; thatis

zi(Mz)i ≤ 0 i = 1, . . . , n

for some nonzero nonnegative vector z.

Proof. We are working with the system

w = q + dz0 + Mz, w ≥ 0, z0 ≥ 0, z ≥ 0.

Suppose the algorithm reaches an almost-complementary extreme point (basicfeasible solution) (w, z0, z) at which we have z0 > 0 and wizi = 0 for all i > 0.

If the driving variable is unblocked, we have termination on a secondary ray.Hence there is a nonzero solution (w, z0, z) of

w = dz0 + Mz, w ≥ 0, z0 ≥ 0, z ≥ 0.

Page 21: Introduction to Lemke’s Method (Scheme I)

Moreover, we have

(w + λw)i(z + λz)i = 0 for all i = 1, . . . , n and all λ ≥ 0.

Now z 6= 0, for otherwise z0 > 0 (else (w, z0, z) is zero). But z0 > 0 impliesw > 0 and hence that the secondary ray is the primary ray, which is impossible.

For i = 1, . . . , n and all λ ≥ 0, we have

wizi + λwizi + λwizi + λ2wizi = 0

which implieswizi = wizi = wizi = wizi = 0

Thus for each i = 1, . . . n,

0 = ziwi = zidiz0 + zi(Mz)i.

Since zidiz0 ≥ 0, we have the asserted result: there exists a nonnegative nonzerovector z such that

zi(Mz)i ≤ 0 for all i = 1, . . . , n.

Page 22: Introduction to Lemke’s Method (Scheme I)

More Theory of Lemke’s Method

Theorem. Lemke’s Algorithm will solve any nondegenerate LCP

(q, M) such that M ∈ E.

In particular, Lemke’s Algorithm will solve any nondegenerate LCP

(q, M) such that M ∈ P or M ∈ SCP (strictly copositive).

Proof. It is impossible for a strictly semimonotone matrix to reverse

the sign of a nonzero, nonnegative vector.

Recall that a matrix M ∈ Rn×n is said to be semimonotone if

[0 6= x ≥ 0] ⇒ [xk > 0, (Mx)k ≥ 0 for some k]

The class of semimonotone matrices is denoted E0.

Page 23: Introduction to Lemke’s Method (Scheme I)

Theorem. Let M ∈ Rn×n ∩ E0. If Lemke’s Algorithm applied to

(q, d, M) terminates with a secondary ray, then SOL(0, M) 6= {0}.

Proof. From the proof of the theorem on consequences of termination

on a ray, we know that there exists a nonzero nonnegative vector

(z0, z) such that w = dz0 + Mz ≥ 0 and zTw = 0. Since M ∈ E0

we cannot have z0 > 0, for otherwise, 0 = wσ = dσz0 + Mσσzσ

where σ = supp(z). Hence Mσσzσ < 0. But now it is clear that

z ∈ SOL(0, M).

Page 24: Introduction to Lemke’s Method (Scheme I)

Results for copositive matrices

Corollary. If M ∈ CP and Lemke’s Algorithm applied to (q, d, M)

terminates with a secondary ray, then the vector y = z/‖z‖ satisfies:

(i) 0 6= y ∈ SOL(0, M), (ii) qTy < 0.

Proof. Since SOL(0, M) is clearly a closed cone, it is clear that (i)

holds. As for (ii), note that for all λ ≥ 0 we have

0 = (z + λz)T(q + dz + M(z + λz)) ≥ (z + λz)T(q + dz0).

Dividing through by ‖z + λz‖ and letting λ −→ ∞ we obtain

yT(q + dz0) ≤ 0. Since yTdz0 > 0 we obtain (ii).

Theorem. If M is copositive and q ∈ (SOL(0, M))∗, then Lemke’s

Algorithm will compute a solution of the LCP (q, M) if the problem

is nondegenerate.

Proof. This is just a corollary of the corollary.

Page 25: Introduction to Lemke’s Method (Scheme I)

Constructive proof that CP+ ⊂ Q0

Theorem. If M ∈ CP+ and Lemke’s Algorithm applied to a nonde-

generate instance of the problem (q, d, M) terminates on a secondary

ray, then the LCP (q, M) is infeasible.

Proof. If the algorithm terminates on a secondary ray, we have

0 = zTw = zTdz0 + zTMz.

Since M is copositive, both terms on the right are nonnegative, hence

both are zero. But since M ∈ CP+, we have (M + MT)z = 0.

We also know that z0 = 0, hence Mz ≥ 0 and MTz ≤ 0. The latter

is equivalent to zTM ≤ 0. Moreover zTq < 0; thus

z ≥ 0, zTM ≤ 0, zTq < 0.

These conditions imply that FEA(q, M) = ∅.

Page 26: Introduction to Lemke’s Method (Scheme I)

Can Lemke’s Algorithm find global minima of nonconvex QPs?

Example. Consider the quadratic program

minimize 12x1 − 1

2x2 − 12x

21 + 1

2x22

subject to 2x1 + x2 ≤ 6

−x1 + 4x2 ≤ 6

x1 ≥ 0, x2 ≥ 0

The problem has a nonempty compact constraint set, hence a global optimum.

The KKT conditions give the LCP (q, M) where

q =

12

−12

6

6

and M =

−1 0 2 −1

0 1 1 4

−2 −1 0 0

1 −4 0 0

Choose the covering vector d ∈ R4++ arbitrarily. Then scale it so that d2 = 1.

Page 27: Introduction to Lemke’s Method (Scheme I)

1 z0 x1 x2 y3 y4

y112 d1 −1 0 2 −1

y2 −12 1 0 1 1 4

x3 6 d3 −2 −1 0 0

x4 6 d4 1 −4 0 0

The pivot to perform is 〈y2, z0〉. It leads to the tableau

Page 28: Introduction to Lemke’s Method (Scheme I)

1 z0 x1 x2 y3 y4

y112 d1 −1 0 2 −1

y2 −12 1 0 1 1 4

x3 6 d3 −2 −1 0 0

x4 6 d4 1 −4 0 0

The pivot to perform is 〈y2, z0〉. It leads to the tableau

1 y2 x1 x2 y3 y4

y112(1 + d1) d1 −1 −d1 2 − d1 −1 − 4d1

z012 1 0 −1 −1 −4

x312(12 + d3) d3 −2 −1 − d3 −d3 −4d3

x412(12 + d4) d4 1 −4 − d4 −d4 −4d4

The pivot to perform is 〈z0, x2〉. It leads to the tableau

Page 29: Introduction to Lemke’s Method (Scheme I)

1 y2 x1 x2 y3 y4

y112(1 + d1) d1 −1 −d1 2 − d1 −1 − 4d1

z012 1 0 −1 −1 −4

x312(12 + d3) d3 −2 −1 − d3 −d3 −4d3

x412(12 + d4) d4 1 −4 − d4 −d4 −4d4

The pivot to perform is 〈z0, x2〉. It leads to the tableau

1 y2 x1 z0 y3 y4

y112 0 −1 d1 2 −1

x212 1 0 −1 −1 −4

x3112 −1 −2 1 + d3 1 4

x4 4 −4 1 4 + d4 4 16

Page 30: Introduction to Lemke’s Method (Scheme I)

1 y2 x1 z0 y3 y4

y112 0 −1 d1 2 −1

x212 1 0 −1 −1 −4

x3112 −1 −2 1 + d3 1 4

x4 4 −4 1 4 + d4 4 16

The solution of the KKT conditions found is (x1, x2, y3, y4) = (0, 12 , 0, 0).

The point (x1, x2) = (0, 12) is a local minimum.

The global minimum is (x1, x2) = (3, 0).

It cannot be reached by Lemke’s Algorithm, no matter what covering vector isused.