A computational study of Exterior Point Simplex Algorithm ...users.uom.gr/~samaras/pdf/C41.pdf ·...

Επιχειρησιακή Έρευνα και Τουριστική Ανάπτυξη 777

Ενότητα 7 – Αλγοριθμική και Επιχειρησιακή Έρευνα 777

A computational study of Exterior Point Simplex Algorithm variations

Paparrizos Konstantinos Department of Applied Informatics, University of Macedonia

156 Egnatia Str., 54006 Thessaloniki [email protected]

Samaras Nikolaos

Department of Applied Informatics, University of Macedonia 156 Egnatia Str., 54006 Thessaloniki

[email protected]

Triantafyllidis Charalampos Department of Applied Informatics, University of Macedonia

156 Egnatia Str., 54006 Thessaloniki [email protected]

Abstract This paper analyses three variations of a relatively new algorithmic family for the linear problem, the Exterior Point Algorithms. These Simplex type-methods make use also of exterior points, along with the feasible path in order to converge to the optimal solution. We implement them using the well-known 2-phase method for general linear problems, and we use three different strategies to enter phase-2, the one which actually solves the initial problem. All these variations use the same second phase, but not also the first one. We use the advantage of these algorithms that if the current direction crosses the feasible region of our problem, then the algorithm will surely converge to the optimal solution. This attribute gives us the ability to avoid a significant amount of iterations in phase-1 or totally the phase-1 part, if we manage to initialize always the algorithm with such a direction. We compare them to the revised Primal Simplex algorithm and the one that is already built-in in the Matlab R2007b software, using a computational study. The algorithms were tested on randomly generated sparse linear problems of various sizes and densities, the majority of which were optimal. We compared them for the total covered number of iterations as well as the cpu-time needed to solve the problems. As shown, these algorithms outperform the revised Primal Simplex algorithm, both in iterations and solution times. Keywords: Linear Programming, Simplex-type algorithms, Computational Study, Matlab.

1. INTRODUCTION

This paper exploits the computational behavior of the Exterior Point algorithms, a quite recent algorithmic family for the solution of the general linear problem. Most of real world problems can be formulated as linear problems. Linear Optimization consists of optimizing, maximizing or minimizing, a linear function over a set of linear constraints. The kind of the constraints can be either inequalities or equalities. The computational performance of simplex algorithm on practical problems is usually far better that the theoretical worst case. The big challenge with the simplex type algorithms is to implement computational versions of them that can solve large-scale linear problems efficiently. We



assume that the problem is in its general form. Formulating the linear problem in mathematical terms now, we can describe it as shown below:

min subject to 0.

Tc xAx bx

=≥

(LP. 1).

mnmxn RbRxcRAwhere ∈∈∈ ,),(, and T denotes transposition. We assume that A has full

rank. The simplex method searches for an optimal solution by moving from one feasible solution to another, along the edges of the feasible set, in a cost reducing direction. Exterior Point Simplex Algorithm (EPSA) differs radically from Primal Simplex Algorithm (PSA) because its basic solutions are not feasible (Paparrizos et al., 2001, Paparrizos, 1997). EPSA relies on the idea (Paparrizos et al., 2003) that making steps in directions that are linear combinations of attractive descent directions can lead to faster practical convergence than that achievable by PSA. For all known pivoting rules (Paparrizos, 1989, Roos, 1990) sequences of examples have been constructed, such that the number of iterations is exponential in m+n. None of the existing simplex type algorithms admits polynomial complexity.

This paper presents three variations of EPSA using the well-known 2-phase method. More specifically, three different strategies to enter phase-2 are presented. Also, we perform a computational study on randomly generated sparse linear problems of various sizes and densities, in order to compare the three variations of EPSA.

The paper is organized as follows. In section 2 we present the EPSA algorithm. In section 3 the three different strategies (initialization methods) are presented. In section 4 we give the computational results. Finally, in section 5we give our conclusions and discuss our future work. 2. ALGORITHM DESCRIPTION

We find it convenient to describe the algorithm using the revised form. The algorithm starts with a primal feasible basic partition (B, N). The variables corresponding to B will be called basic. The others will be referred to as non-basic. Later on the following sets of indexes are computed:

P = {j ∈ N : sj < 0} (1)

Q = {j ∈ N : sj ≥ 0} (2)

If P = ∅ then the current basis B and the corresponding solution xT = (xB, xN) is optimal to (L.P.1). EPSA defines first the leaving and then the entering variable. The leaving variable xB[r]

= xk is computed by

<−

=−

= 0d:d

xmin

dx

a ]i[B]i[B

]i[B

]r[B

]r[B (3)

where d is an improving direction. This direction is constructed in such way that the ray {x + td : t > 0} crosses the feasible region of (L.P.1). The sub-index ‘B’ (dB) means that we select those components from d which correspond to the basic variables. The dB is



computed by

B j jj P

d hλ∈

= −∑ (4)

where hj = B-1A.j. If a = +∞ problem (L.P.1) is unbounded. The entering variable xl must now be computed. We must first compute the following ratios :

jP1 rj

rP rj

sstheta min{ : H 0 j P}H H

−−= − = > ∧ ∈ (5)

}Qj 0H:Hs-

min{H

stheta rj

rj

j

rQ

Q2 ∈∧<=

−−= (6)

If theta1 ≤ theta2 then l = p, otherwise (theta1 > theta2) l = q. The non-basic variable xl enters the basis. A formal description of the EPSA is given below.

EPSA algorithm

Step 0. (Initialization). Start with a feasible partition (B, N). Compute B-1 and vectors xB, w and sN. Find the sets of indices P and Q using relations (1) and (2) correspondingly. Define an arbitrary vector λ = (λ1, λ2, ..., λ|P|) > 0 and compute s0 by

0 j j

j Ps sλ

∈

= ∑

and the direction dB from (4). Step 1. (Termination test).

i. (Optimality test). If P = ∅, STOP. The (L.P.1) is optimal. ii. (Leaving variable selection). If dB ≥ 0, STOP. If s0 = 0 the problem (L.P.1)

is optimal. If s0 < 0 the problem (L.P.1) is unbounded. Otherwise choose the leaving variable xB[r] = xk using relation (3).

Step 2. (Entering variable selection). Compute the row vectors HrP = (B-1)r.AP and HrQ = (B-1)r.AQ

Compute the ratios theta1 and theta2 using relations (5) and (6). Determine the indices t1 and t2 such that P[t1] = p and Q[t2] = q. If theta1 ≤ theta2, set l = p Otherwise set l=q. The non basic variable xl enters the basis.

Step 3. (Pivoting). Set B[r] = l. If theta1 ≤ theta2, set P ← P\{l} and Q ← Q∪{k}. Otherwise, set Q[t2] = k. Using the new partition (B, N) where Ν = (P, Q), update the matrix B-1 and the vectors xB, w and sN. Also updatedB by

dB = E-1dB

If l∈P set [ ]B rd ← dB[r]+λl. Go to step 1. Proof of correctness of the above algorithm can be found in (Paparrizos, 1996).



3. INITIALIZATION METHODS

EPSA can be terminated in phase 1 even if the set P is not empty. This is due to the fact that the algorithm also uses infeasible paths in order to converge and its not bound to feasibility. In this case the criterion to move to phase 2 is that the direction dB should be crossing the feasible region. EPSA was implemented using three different strategies to move from Phase 1 to Phase 2. Those were:

• EPSA1 (Hybrid EPSA). This version embodies unchanged the phase 1 of PSA and actually applies EPSA in phase 2. Thus, in phase 1 this algorithm works exactly as PSA, and when a feasible partition is found EPSA is applied. Therefore this algorithm is the union of PSA in phase 1 and EPSA in phase 2 and as a result it’s a hybrid algorithm.

• EPSA2. In this version both phases use EPSA. This means that we apply the main algorithm steps described in the previous section to the modified problem of phase 1. We move to phase II only when i) the artificial variable has left the basis and at the same time ii) dB direction crosses the feasible region. That means that the set P isn’t necessarily empty. Of course we can move also to Phase 2 if the current point is feasible (and again the artificial variable has already left the basis) because feasibility also means that the current direction crosses the feasible region. In the worst case, this version has to reach modified problem of Phase 1 to optimality, to obtain a feasible partition for the initial problem.

• EPSA3. We also use EPSA in both phases here. EPSA moves to phase 2 only when the modified problem of phase 1 is reached to optimality. This means that the P set must be empty (P=∅). It is obvious that EPSA2 behaves in the worst case as EPSA3. Although we expect EPSA2 to be faster than EPSA3 in practice, and it is true for the number of total iterations needed, we should not forget that this algorithm checks at each iteration after the removal of the artificial variable from the current basis, if the direction crosses the feasible region. This check can cost some significant amount of cputime. Depending on the number of iterations this cost may overlap the total cputime needed and make EPSA2 slower than the EPSA3 algorithm.

4. COMPUTATIONAL STUDY

There are three different approaches to analyzing algorithms: worst-case analysis, average-case analysis and experimental analysis. Computational studies have proven useful tools to examine the practical effectiveness of an algorithm, or even compare algorithms by using the same pool of problems. Although generating the random problems and applying the algorithms can be easy, there are some other measures which we should keep in mind before performing the computational study. This applies of course for large scale linear programming, since small problems are not capable of causing problems to our solvers. Scaling techniques for example may be necessary to apply if our test problems have poorly scaled matrices. In our study, we used the equilibration scaling technique. In this method, all elements of the coefficient matrix A have values between -1 and +1. There are also some issues concerning mathematical accuracy errors during the iterations which can cause our algorithms to converge using much more pivots because of them. So, applying tolerances is almost certain.



Furthermore, as it is broadly known, the most time consuming calculation in such algorithms is the inversion of the current basis. Therefore in order to achieve the optimal balance between accuracy and efficiency we use mostly the eta-matrices inversion procedure. In order to guarantee the accuracy, periodically we compute from scratch the inverse of the basis. We invert the basis only every a specific number of iterations set by us (usual values are between 80-100), using the built-in function of Matlab ‘inv’ (matrix inversion). The tables below are the results from the measurements we made for the computational study that we performed. The reported CPU times were measured in seconds with MATLAB’s built-in function ‘cputime’. For 10% density, instances of increasing sized are randomly generated and run on all codes. We used an Intel Pentium 4 CPU 3.4 Ghz, with 1 GB RAM and Microsoft Windows XP SP2, under the environment of MATLAB R2007b x86. Also, PSA_MATLAB here stands for the built-in implementation of simplex algorithm in MATLAB.

Sizes PSA EPSA1 EPSA2 EPSA3 PSA_MATLAB 500x500 2271.60 1197.30 962.90 1002.10 2600.40 600x600 3320.30 1660.70 1238.00 1225.80 3199.50 700x700 4621.90 2218.40 1717.40 1753.40 3611.10 800x800 5712.10 2272.50 1679.30 1714.30 3286.30 900x900 7473.70 2564.00 2066.60 2040.30 3947.00

1000x1000 9721.90 2767.70 2053.90 2047.20 4328.10 1100x1100 12100.30 2064.70 1692.40 1692.40 4460.70 1200x1200 13314.80 3309.40 2646.90 25880 3883.70 1300x1300 17193.10 2549.80 2098.60 2078.00 4253.60 1400x1400 20080.40 1821.30 1516.30 1490.90 4478.10 1500x1500 21399.40 3317.80 2706.10 2662.50 4121.10

Table 1: Total number of iterations

Sizes PSA EPSA1 EPSA2 EPSA3 PSA_MATLAB 500x500 18.64 10.34 7.213 7.11 370.96 600x600 41.75 21.97 13.53 12.38 689.31 700x700 82.59 39.89 24.99 24.66 1173.58 800x800 127.05 51.73 31.42 29.89 1754.81 900x900 217.43 73.95 49.54 45.29 2611.45

1000x1000 336.36 94.62 55.27 51.57 4034.80 1100x1100 514.30 79.40 52.81 50.10 5777.72 1200x1200 672.39 159.21 100.49 93.49 7679.71 1300x1300 1058.14 135.41 88.03 82.39 10732.35 1400x1400 1426.25 104.16 68.52 64.04 12702.11 1500x1500 1710.25 237.72 156.02 145.75 17250.85

Table 2: Total cputime

Sizes PSA EPSA1 EPSA2 EPSA3 PSA_MATLAB 500x500 2.26 1.19 0.96 1 2.59 600x600 2.70 1.35 1.00 1 2.61 700x700 2.63 1.26 0.97 1 2.05 800x800 3.33 1.32 0.97 1 1.91 900x900 3.66 1.25 1.01 1 1.93

1000x1000 4.74 1.35 1.00 1 2.11 1100x1100 7.14 1.21 1.00 1 2.63 1200x1200 5.14 1.27 1.02 1 1.50



1300x1300 8.27 1.22 1.00 1 2.04 1400x1400 13.46 1.22 1.01 1 3.00 1500x1500 8.03 1.24 1.01 1 1.54

Table 3: Normalized Ratios for number of iterations over EPSA3

Sizes PSA EPSA1 EPSA2 EPSA3 PSA_MATLAB 500x500 2.61 1.45 1.01 1 52.13 600x600 3.37 1.77 1.09 1 55.60 700x700 3.34 1.61 1.01 1 47.57 800x800 4.24 1.73 1.05 1 58.68 900x900 4.80 1.63 1.09 1 57.65

1000x1000 6.52 1.83 1.07 1 78.22 1100x1100 10.26 1.58 1.05 1 115.32 1200x1200 7.19 1.70 1.07 1 82.13 1300x1300 12.84 1.64 1.06 1 130.25 1400x1400 22.27 1.62 1.07 1 198.34 1500x1500 11.73 1.63 1.07 1 118.35

Table 4: Normalized Ratios for cputime over EPSA3

Graph 1 : Total about number of iterations



Graph 2 : Cputime in seconds (log10 format)

Graph 3 : Normalized Ratios for number of iterations over EPSA3



Graph 4 : Normalized Ratios for cputime over EPSA3

As we see best in cputime algorithm from all competitive, is the EPSA3 implementation. We can also observe that all EPSA algorithms are better in both number of iterations and cputime from our PSA implementation. Furthermore the PSA implementation of MATLAB is much slower in cputime from our PSA and therefore much slower from all EPSA algorithms. 4. CONCLUSION

Exterior Point algorithms are clearly superior over the classical Revised Simplex Algorithm. In fact they outperform Simplex in both total number of covered iterations as well as the amount of processing time needed from the cpu to solve the problems. This is because Exterior Point algorithms use infeasible points, out of the feasible region, and as a result they are not bound to visit each adjacent vertex of the polyhedron one by one, as Simplex does. Thus, they can avoid a significant amount of pivoting steps and converge quicker to the optimal solution. This algorithmic family also gives us the chance to improve its performance even better. That is if we manage to supply initially the algorithm with a direction which crosses the feasible region, so then we are able to fully avoid the construction of phase I. That would be an advantage comparing to Simplex algorithm. There is also left to examine whether this type of algorithms are scaling – invariant or not. It is known that PSA with Dantzig’s - pivoting rule is not scaling - invariant. In fact, without scaling techniques applied, the total number of iterations needed is much larger than when applying scaling. If the Exterior Point algorithms are also scaling – invariant, that means they have one more desirable property also, empowering their presence in the world of linear programming. 5. REFERENCES

Paparrizos, K. (1997), “Pivoting algorithms generating two paths”, presented in ISMP ’97, Lausanne, EPFL Switzerland.



Paparrizos K, Samaras N, Stephanides G. (2003): “An efficient simplex type algorithm for sparse and dense linear programs”, European Journal of Operational Research, Vol. 148(2), pp. 323-334. Paparrizos, K., Samaras, N., Tsiplidis, K. (2001). “Pivoting algorithms for (LP) generating two paths” in : M.P PArdalos, A.C. Floudas (eds.) Encyclopedia of Optimization, Vol. 4, Kluwer Academic Publishers, pp. 302-306. Roos, C. (1990). “An exponential example for Terlaky’s pivoting rule for the Criss-cross Simplex method”, Mathematical Programming, Vol. 46, pp. 78-94. Paparrizos, K.(1989). “Pivoting rules directing the Simplex method though all feasible vertices of Klee-Minty examples”, Operations Research, Vol. 26, pp.77-95. Paparrizos, K. (1996). “Exterior point simplex algorithm : Simple and short proof of correctness”. Proceedings of SYMOPIS’96, pp. 13-18.

A computational study of Exterior Point Simplex Algorithm ...users.uom.gr/~samaras/pdf/C41.pdf ·...

Documents

Transcript of A computational study of Exterior Point Simplex Algorithm ...users.uom.gr/~samaras/pdf/C41.pdf ·...