Calculation of Lagrange multipliers and their use for
sensitivity of optimal solutions
Jaco F. Schutte
EGM 6365 - Structural OptimizationFall 2005
Constrained optimization
x1
x2
Infeasible regions
Feasible region
OptimumDecreasing f(x)
h(x)g(x)
Constraint normalization
0
1 0
a
a
a
g
g
σ σσ σ
σσ
≤= − ≤
= − ≥
Poor optimizer performance often encountered when constraints are not normalized
When normalized g = 0.1 → 10% margin in responce
Equality constraints
� Convert equality constraint to two equality constraints
� Increases the number of constraints
( ) ( )( )
00
0i
ii
h xh x
h x ≤= ⇔ ≥
Reduction of inequality constraints
� Using KS-function (Kreisselmeier-Steinhauser)
� KS bounded by
( )( )
( )
( ) ( )
1
1
00 1 ln
0
jg xj
j
j
g xg x
KS g x e
g x
ρ
ρ−
≥≥ ⇔ = −
≥
∑!
( ) ( )min min
lnj
mg KS g x g
ρ ≤ ≤ −
Using KS to approximate hi(x)
For
the solution lies at hi(x) = - hi(x) = 0
Gradient of KS function of ± hi pair vanishes at solution hi = 0Value of KS function approaches 0 for ρ → ∞
( ) ( )( )
00
0i
ii
h xh x
h x ≤= ⇔ − ≤
( ) ln(2)0 ,KS h hρ
≥ − ≥ −
� Lagrangian function
where λ j are unknown Lagrange multipliers� Stationary point conditions for inequality
constraints:
Kuhn-Tucker conditions
Stationary points
Kuhn-Tucker conditions (contd.)
� Conditions only apply at a regular point (constraint gradients linearly independent)
� Equations
yield n + ne total equations� n stationary point coordinates� ne Lagrange multipliers
� Inequality constraints require transformation to equality constraints:
� This yields the following Lagrangian:
Kuhn-Tucker conditions (contd.)
Kuhn-Tucker conditions (contd.)
� Conditions for stationary points are then:
� If inequality constraint is inactive (t ≠ 0) then Lagrange multiplier = 0
� For inequality constraints a regular point is when gradients of active constraint are linearly independent
Kuhn-Tucker conditions (contd.)
� For an inequality constrained problem x is a minimum of a set of no-negative λ i can be found such that:
1)
2) The corresponding λ i is zero if constraint gj is not active
Kuhn-Tucker conditions (contd.)
Kuhn-Tucker graphical representation
Sufficient conditions� Kuhn-Tucker conditions are satisfied when no
-∇ f can be obtained without violating constraints� Possible to move perpendicular to constraints
and improve objective function (necessary conditions are met, but not sufficient)
� Kuhn tucker conditions for optimality are sufficient when� no. design vars. = no. of active constraints.� or, Hessian of Lagrangian function is positive definite
for subspace tangent to active constraints
Sufficient conditions (contd.)
� For equality constraints
where
� For inequality constraints
A function is convex if
or
Convex problems
� Convex optimization problem has� convex objective function� convex feasible domain if
� All inequality constraints are concave (or �gj = convex)
� All equality constraints are linear
� only one optimum� Kuhn-Tucker conditions necessary and will also
always be sufficient for global minimum
Convex problems (contd.)
Calculating Lagrange multipliers
� Lagrange equation in matrix notation:
where N is
assuming number of active constraints are r define a residual vector u (to be minimized)
Calculating Lagrange multipliers (contd.)
Obtain a least squares solution of u
By differentiating w.r.t. each λ
Or
And substituting into
We obtain where
Calculating Lagrange multipliers (contd.)
� P is a projection matrix which projects a vector into a subspace which is tangent to the constraints
� For the Kuhn Tucker conditions to be satisfied, ∇ f has to be orthogonal to this subspace
� The method of using is often ill-conditioned matrices and inefficient
Alternate method for calculating Lagrange multipliers
QR factorization of N gives a more efficient way ofcalculating λ
Because Q is orthogonal
║u║2 is then minimized by choosing
Alternate method for calculating Lagrange multipliers
Sensitivity of optimum solution to problem parameters
Assuming problem fitness and constraintsdepend on parameter p
The solution is
and the corresponding fitness value
Sensitivity of optimum solution to problem parameters (contd.)
We would like to obtain derivatives of f* w.r.t. p
Equations that govern the optimum solution are
After manipulating governing equations we obtain
Top Related