Eigenvalue Optimisation via Semidefinite...
Transcript of Eigenvalue Optimisation via Semidefinite...
Computer programs for linear SDP
Eigenvalue Optimisationvia Semidefinite Programming
Michal Kocvara
DCAMM Course, June 2007
Computer programs for linear SDP
Consider the eigenvalue problem
A(x)z = λz (or A(x)z = λM(x)z)
with A (and M) depending linearly on a parameter x :
A(x) =n∑
i=1
xiAi + A0
where Ai are given symmetric matrices. Assume further thatA(x) is positive semidefinite for all x .
Computer programs for linear SDP
Consider further the optimisation problem
minx∈Rn
cT x
subject toλmin ≥ λ0
where λmin is the smallest eigenvalue and λ0 some givenpositive number.
Looking at λmin as a function of x , the constraint is nonconvexand non-differentiable, and thus the optimisation problem israther difficult to solve.
Computer programs for linear SDP
However, requiring that λmin ≥ λ0 is just the same as sayingthat all eigenvalues are greater than or equal to λ0. This, inturn, is equivalent to the constraint
A(x)− λ0I is positive semidefinite
or, with the SDP notation, A(x)− λ0I � 0.Thus the optimisation problem can be written as
minx∈Rn
cT x
subject toA(x)− λ0I � 0 .
This is a convex optimisation problem that can be solved by thebarrier (or interior-point) method in a way analogous tostandard convex NLP.
Computer programs for linear SDP
Primal-dual SDP pair
Primal semidefinite programming program:
minx∈Rn
{cT x | A(x)− B � 0} , (SDP-P)
Remark: (SDP-P) often written as
minx∈Rn,S∈Sm
{cT x | A(x)− B = S, S � 0}
with a slack variable S.
Dual semidefinite programming program:
maxY∈Sm
{Tr(BY ) | Tr(AjY ) = cj , j = 1, . . . , n, Y � 0} . (SDP-D)
Computer programs for linear SDP
SDP Optimality conditions
Remark. For any pair X , Y ∈ Sm+, we have
Tr(SY ) = 0 ⇔ SY = YS = 0 .
Theorem (Optimality conditions): Let (P) and (D) be strictlyfeasible. The following is a system of necessary and sufficientoptimality conditions for (P) and (D):
Tr(AiY ) = ci , Y � 0, i = 1, . . . , nA(x)− B = S, S � 0YS = SY = 0.
Computer programs for linear SDP
Logarithmic barrier methodsPrimal log-barrier methodFor µ ↘ 0 solve
minx ,S
cT x − µ log det(S)
subject toA(x)− B = S
Dual log-barrier methodFor µ ↘ 0 solve
maxY
Tr(BY )− µ log det(Y )
subject toTr(AjY ) = cj , j = 1, . . . , n
Computer programs for linear SDP
Logarithmic barrier methodsPrimal-dual log-barrier methodMinimize the duality gap
Tr(BY )− cT x = Tr(SY )
using the primal-dual barrier function
−(log det(S) + log det(Y )) = − log det(SY )
For µ ↘ 0 solve
minx ,S,Y
Tr(SY )− µ log det(SY )
subject toTr(AjY ) = cj , j = 1, . . . , nA(x)− B = S
Computer programs for linear SDP
Central path
Perturb OC by µ > 0:
Tr(AiY ) = ci , Y � 0, i = 1, . . . , n (OCµ)A(x)− B = S, S � 0
YS = SY = µI.
Theorem: System (OCµ) has a unique solution.
Computer programs for linear SDP
Central path
Definition: The curve defined by solutions (x(µ), S(µ), Y (µ)) of(OCµ) is called central path.
Theorem: The central path exists if (SDP-P) and (SDP-D) arestrictly feasible.
Theorem: The pair
S∗ = limµ↘0
S(µ), Y ∗ = limµ↘0
Y (µ)
is the maximally complementary solution pair (matrices withhighest rank).
Computer programs for linear SDP
Primal-dual path-following methods
Given µ > 0, the pair (S(µ), Y (µ)) is the target point on thecentral path, associated with target duality gap Tr(YS) = nµ.
Idea: iteratively compute approximations of (S(µ), Y (µ)) andthus follow the central path while decreasing µ.
Assume S � 0, Y � 0, solve the OC for the P-D problem
Tr(AiY ) = ci , i = 1, . . . , nA(x)− B = S
YS = µI
by the Newton method.
Computer programs for linear SDP
Primal-dual path-following methods
Tr(AiY ) = ci , i = 1, . . . , nA(x)− B = S
YS = µI
Find ∆x ,∆S,∆Y :
(i) Tr(Ai∆Y ) = ci − Tr(AiY ), i = 1, . . . , n(ii) A(∆x)−∆S = A(x)
(iii) Y∆S + ∆YS = µI − YS (−∆Y∆S)
Remark: Solutions ∆S,∆Y of (iii) generally nonsymmetrix.
(∆S symmetric from (ii) but ∆Y may be nonsymmetric)
Symmetrization of (iii) needed.
Computer programs for linear SDP
Symmetrization techniquesReplace YS by symmetrization
HP(YS) = µI
where HP(M) = 12(PMP−1 + P−T MT PT ).
Thus (iii) becomes
(iii)′ HP(Y∆S + ∆YS) = µI − HP(YS)
The scaling matrix determines the symmetrization strategy.
P =[Y12 (Y
12 SY
12 )−
12 Y
12 ]
12 Nesterov-Todd
Y− 12 Monteiro
S12 Monteiro, Helmberg
I Alizadeh-Haeberly-Overton
Computer programs for linear SDP
Primal-dual path-following algorithms
Input: (S0, Y 0) ∈ P ×D
Parameters: τ < 1 and µ0 > 0
Algorithm (generic): Set S := S0, Y := Y 0, µ := µ0
while Tr(SY ) > ε do
compute ∆S,∆Y by solving (i), (ii), (iii)′
S := S + ∆S, Y := Y + ∆Ychoose 0 < θ < 1 and set µ := (1− θ)µ
end
Computer programs for linear SDP
Primal-dual path-following algorithms
Theorem: If τ = 1√2
and θ = 12√
n , then the primal-dualpath-following algorithm with NT symmetrization terminatesafter at most
O(
2√
n lognµ0
ε
)iterations.
Computer programs for linear SDP
Available codes
Primal-dual IP codes (available):CSDP-5.0 http://www.nmt.edu/∼borchers/csdp.html
very good overallSDPA-6.00 http://grid.r.dendai.ac.jp/sdpa/SDPT3-4.0 http://www.math.nus.edu.sg/∼mattohkc/
very good overallSeDuMi-1.1 http://fewcal.kub.nl/sturm/software/sedumi.html
very robust, slow on large-scale problems
Dual IP code:DSDP-5.8 http://www-unix.mcs.anl.gov/∼benson/
very good new version
Computer programs for linear SDP
Available codes
Augmented Lagrangian code:PENSDP-2.2 http://www.penopt.com/
First order code:SDPLR-1.02
http://dollar.biz.uiowa.edu/∼burer/software/SDPLR/very good for certain large-scale problems; maybe very slow, limited accuracy
Other codes:SBmethod-1.1.2 http://www-user.tu-
chemnitz.de/∼helmberg/SBmethod/very efficient; limited applicability
Computer programs for linear SDP
SDP input
MATLAB most popular; most codes have their own MATLABinterfaces; gathered in YALMIP
http://control.ee.ethz.ch/˜joloef/yalmip.msql
ASCII file in SDPA sparse format; good for testingC/Fortran available by few codes; best for heavy-duty
applications
Computer programs for linear SDP
YALMIP exampleLyapunov stability:
P � 0
AT P + PA � 0
A = [-1 2;-3 -4];P = sdpvar(2,2);
Looking for feasible solutions only
F = set(P > 0) + set(A’*P+P*A < 0);F = F + set(trace(P) == 1); %to avoid zero/unbounded solutionssolvesdp(F);
Minimizing top-left element of P
F = set(P > 0) + set(A’*P+P*A < 0);solvesdp(F,P(1,1),sdpsettings(’solver’,’sedumi’));
Computer programs for linear SDP
SDPA examplemin 10 x1 + 20 x2st X = F1 x1 + F2 x2 - F0
X >= 0
F1=[1 0 0 0 F2=[1 0 0 0 F3=[0 0 0 00 2 0 0 0 1 0 0 0 1 0 00 0 3 0 0 0 0 0 0 0 5 20 0 0 4] 0 0 0 0] 0 0 2 6]
A sample problem.2 =mdim ...cont...2 =nblocks 1 1 1 1 1.0{2, 2} 1 1 2 2 1.010.0 20.0 2 1 2 2 1.00 1 1 1 1.0 2 2 1 1 5.00 1 2 2 2.0 2 2 1 2 2.00 2 1 1 3.0 2 2 2 2 6.00 2 2 2 4.0...cont...
Computer programs for linear SDP
Useful WWW sites
http://www-neos.mcs.anl.gov/neos/Server for optimization. All codes mentioned here areimplemented.SDPA and Matlab binary input.
http://plato.la.asu.edu/bench.htmlBenchmarks of latest versions of all available codes.