Computer programs for linear SDP

Eigenvalue Optimisationvia Semidefinite Programming

Michal Kocvara

DCAMM Course, June 2007

Computer programs for linear SDP

Consider the eigenvalue problem

A(x)z = λz (or A(x)z = λM(x)z)

with A (and M) depending linearly on a parameter x :

A(x) =n∑

i=1

xiAi + A0

where Ai are given symmetric matrices. Assume further thatA(x) is positive semidefinite for all x .

Computer programs for linear SDP

Consider further the optimisation problem

minx∈Rn

cT x

subject toλmin ≥ λ0

where λmin is the smallest eigenvalue and λ0 some givenpositive number.

Looking at λmin as a function of x , the constraint is nonconvexand non-differentiable, and thus the optimisation problem israther difficult to solve.

Computer programs for linear SDP

However, requiring that λmin ≥ λ0 is just the same as sayingthat all eigenvalues are greater than or equal to λ0. This, inturn, is equivalent to the constraint

A(x)− λ0I is positive semidefinite

or, with the SDP notation, A(x)− λ0I � 0.Thus the optimisation problem can be written as

minx∈Rn

cT x

subject toA(x)− λ0I � 0 .

This is a convex optimisation problem that can be solved by thebarrier (or interior-point) method in a way analogous tostandard convex NLP.

Computer programs for linear SDP

Primal-dual SDP pair

Primal semidefinite programming program:

minx∈Rn

{cT x | A(x)− B � 0} , (SDP-P)

Remark: (SDP-P) often written as

minx∈Rn,S∈Sm

{cT x | A(x)− B = S, S � 0}

with a slack variable S.

Dual semidefinite programming program:

maxY∈Sm

{Tr(BY ) | Tr(AjY ) = cj , j = 1, . . . , n, Y � 0} . (SDP-D)

Computer programs for linear SDP

SDP Optimality conditions

Remark. For any pair X , Y ∈ Sm+, we have

Tr(SY ) = 0 ⇔ SY = YS = 0 .

Theorem (Optimality conditions): Let (P) and (D) be strictlyfeasible. The following is a system of necessary and sufficientoptimality conditions for (P) and (D):

Tr(AiY ) = ci , Y � 0, i = 1, . . . , nA(x)− B = S, S � 0YS = SY = 0.

Computer programs for linear SDP

Logarithmic barrier methodsPrimal log-barrier methodFor µ ↘ 0 solve

minx ,S

cT x − µ log det(S)

subject toA(x)− B = S

Dual log-barrier methodFor µ ↘ 0 solve

maxY

Tr(BY )− µ log det(Y )

subject toTr(AjY ) = cj , j = 1, . . . , n

Computer programs for linear SDP

Logarithmic barrier methodsPrimal-dual log-barrier methodMinimize the duality gap

Tr(BY )− cT x = Tr(SY )

using the primal-dual barrier function

−(log det(S) + log det(Y )) = − log det(SY )

For µ ↘ 0 solve

minx ,S,Y

Tr(SY )− µ log det(SY )

subject toTr(AjY ) = cj , j = 1, . . . , nA(x)− B = S

Computer programs for linear SDP

Central path

Perturb OC by µ > 0:

Tr(AiY ) = ci , Y � 0, i = 1, . . . , n (OCµ)A(x)− B = S, S � 0

YS = SY = µI.

Theorem: System (OCµ) has a unique solution.

Computer programs for linear SDP

Central path

Definition: The curve defined by solutions (x(µ), S(µ), Y (µ)) of(OCµ) is called central path.

Theorem: The central path exists if (SDP-P) and (SDP-D) arestrictly feasible.

Theorem: The pair

S∗ = limµ↘0

S(µ), Y ∗ = limµ↘0

Y (µ)

is the maximally complementary solution pair (matrices withhighest rank).

Computer programs for linear SDP

Primal-dual path-following methods

Given µ > 0, the pair (S(µ), Y (µ)) is the target point on thecentral path, associated with target duality gap Tr(YS) = nµ.

Idea: iteratively compute approximations of (S(µ), Y (µ)) andthus follow the central path while decreasing µ.

Assume S � 0, Y � 0, solve the OC for the P-D problem

Tr(AiY ) = ci , i = 1, . . . , nA(x)− B = S

YS = µI

by the Newton method.

Computer programs for linear SDP

Primal-dual path-following methods

Tr(AiY ) = ci , i = 1, . . . , nA(x)− B = S

YS = µI

Find ∆x ,∆S,∆Y :

(i) Tr(Ai∆Y ) = ci − Tr(AiY ), i = 1, . . . , n(ii) A(∆x)−∆S = A(x)

(iii) Y∆S + ∆YS = µI − YS (−∆Y∆S)

Remark: Solutions ∆S,∆Y of (iii) generally nonsymmetrix.

(∆S symmetric from (ii) but ∆Y may be nonsymmetric)

Symmetrization of (iii) needed.

Computer programs for linear SDP

Symmetrization techniquesReplace YS by symmetrization

HP(YS) = µI

where HP(M) = 12(PMP−1 + P−T MT PT ).

Thus (iii) becomes

(iii)′ HP(Y∆S + ∆YS) = µI − HP(YS)

The scaling matrix determines the symmetrization strategy.

P =[Y12 (Y

12 SY

12 )−

12 Y

12 ]

12 Nesterov-Todd

Y− 12 Monteiro

S12 Monteiro, Helmberg

I Alizadeh-Haeberly-Overton

Computer programs for linear SDP

Primal-dual path-following algorithms

Input: (S0, Y 0) ∈ P ×D

Parameters: τ < 1 and µ0 > 0

Algorithm (generic): Set S := S0, Y := Y 0, µ := µ0

while Tr(SY ) > ε do

compute ∆S,∆Y by solving (i), (ii), (iii)′

S := S + ∆S, Y := Y + ∆Ychoose 0 < θ < 1 and set µ := (1− θ)µ

end

Computer programs for linear SDP

Primal-dual path-following algorithms

Theorem: If τ = 1√2

and θ = 12√

n , then the primal-dualpath-following algorithm with NT symmetrization terminatesafter at most

O(

2√

n lognµ0

ε

)iterations.

Computer programs for linear SDP

Available codes

Primal-dual IP codes (available):CSDP-5.0 http://www.nmt.edu/∼borchers/csdp.html

very good overallSDPA-6.00 http://grid.r.dendai.ac.jp/sdpa/SDPT3-4.0 http://www.math.nus.edu.sg/∼mattohkc/

very good overallSeDuMi-1.1 http://fewcal.kub.nl/sturm/software/sedumi.html

very robust, slow on large-scale problems

Dual IP code:DSDP-5.8 http://www-unix.mcs.anl.gov/∼benson/

very good new version

Computer programs for linear SDP

Available codes

Augmented Lagrangian code:PENSDP-2.2 http://www.penopt.com/

First order code:SDPLR-1.02

http://dollar.biz.uiowa.edu/∼burer/software/SDPLR/very good for certain large-scale problems; maybe very slow, limited accuracy

Other codes:SBmethod-1.1.2 http://www-user.tu-

chemnitz.de/∼helmberg/SBmethod/very efficient; limited applicability

Computer programs for linear SDP

SDP input

MATLAB most popular; most codes have their own MATLABinterfaces; gathered in YALMIP

http://control.ee.ethz.ch/˜joloef/yalmip.msql

ASCII file in SDPA sparse format; good for testingC/Fortran available by few codes; best for heavy-duty

applications

Computer programs for linear SDP

YALMIP exampleLyapunov stability:

P � 0

AT P + PA � 0

A = [-1 2;-3 -4];P = sdpvar(2,2);

Looking for feasible solutions only

F = set(P > 0) + set(A’*P+P*A < 0);F = F + set(trace(P) == 1); %to avoid zero/unbounded solutionssolvesdp(F);

Minimizing top-left element of P

F = set(P > 0) + set(A’*P+P*A < 0);solvesdp(F,P(1,1),sdpsettings(’solver’,’sedumi’));

Computer programs for linear SDP

SDPA examplemin 10 x1 + 20 x2st X = F1 x1 + F2 x2 - F0

X >= 0

F1=[1 0 0 0 F2=[1 0 0 0 F3=[0 0 0 00 2 0 0 0 1 0 0 0 1 0 00 0 3 0 0 0 0 0 0 0 5 20 0 0 4] 0 0 0 0] 0 0 2 6]

A sample problem.2 =mdim ...cont...2 =nblocks 1 1 1 1 1.0{2, 2} 1 1 2 2 1.010.0 20.0 2 1 2 2 1.00 1 1 1 1.0 2 2 1 1 5.00 1 2 2 2.0 2 2 1 2 2.00 2 1 1 3.0 2 2 2 2 6.00 2 2 2 4.0...cont...

Computer programs for linear SDP

Useful WWW sites

http://www-neos.mcs.anl.gov/neos/Server for optimization. All codes mentioned here areimplemented.SDPA and Matlab binary input.

http://plato.la.asu.edu/bench.htmlBenchmarks of latest versions of all available codes.