Post on 25-Mar-2022
Numerical Methods for Multi-ObjectiveOptimal Control Problems
C. Yalcın Kaya
University of South Australia
Helmut Maurer
University of Munster
Workshop on Optimaland Model Predictive Control
Bayreuth, 9-13 September 2013
Outline
1 Multi-Objective Optimal Control problem
Outline
1 Multi-Objective Optimal Control problem
2 The Pareto Front
Outline
1 Multi-Objective Optimal Control problem
2 The Pareto Front
3 Scalarization : Weighted-Sum andTschebychev Approach
Outline
1 Multi-Objective Optimal Control problem
2 The Pareto Front
3 Scalarization : Weighted-Sum andTschebychev Approach
4 Example 1: Tumour Anti-Angiogenesis
Outline
1 Multi-Objective Optimal Control problem
2 The Pareto Front
3 Scalarization : Weighted Sum andTschebychev Approach
4 Example 1: Tumour Anti-Angiogenesis
5 Example 2: Fed-Batch Bioreactor
Single-Objective Optimal Control Problem
(OCPm)
min (ϕ1(x(tf), tf)
s.t. x(t) = f(x(t), u(t), t) , a.e.t ∈ [0, tf ] ,
φ(x(0), x(tf), tf) = 0 ,
φ(x(0), x(tf), tf) ≤ 0 ,
C(x(t), u(t), t) ≤ 0 , a.e.t ∈ [0, tf ] ,
S(x(t), t) ≤ 0 , all t ∈ [0, tf ] ,
statevariablex ∈ IRn, controlvariableu ∈ IRm,terminal time tf is fixed or free.
Multi -Objective Optimal Control Problem
(OCPm)
min (ϕ1(x(tf), tf), . . . , ϕr(x(tf), tf))
s.t. x(t) = f(x(t), u(t), t) , a.e. t ∈ [0, tf ] ,
φ(x(0), x(tf), tf) = 0 ,
φ(x(0), x(tf), tf) ≤ 0 ,
C(x(t), u(t), t) ≤ 0 , a.e. t ∈ [0, tf ] ,
S(x(t), t) ≤ 0 , all t ∈ [0, tf ] ,
statevariablex ∈ IRn, controlvariableu ∈ IRm,terminal time tf fixed or free.
Multi -Objective Optimal Control Problem
(OCPm)
min (ϕ1(x(tf), tf), . . . , ϕr(x(tf), tf))
s.t. x(t) = f(x(t), u(t), t) , a.e. t ∈ [0, tf ] ,
φ(x(0), x(tf), tf) = 0 ,
φ(x(0), x(tf), tf) ≤ 0 ,
C(x(t), u(t), t) ≤ 0 , a.e. t ∈ [0, tf ] ,
S(x(t), t) ≤ 0 , all t ∈ [0, tf ] ,
statevariablex ∈ IRn, controlvariableu ∈ IRm,terminal time tf fixed or free.Assumption:ϕi(x(tf), tf) ≥ 0 for all i = 1, . . . , r.
Multi-Objective Optimal Control Problem
Thefeasible setX consists of triplets
(x, u, tf) ∈ W 1,∞(0, tf ; IRn) × L∞(0, tf ; IR
n) × IR+
satisfying the dynamic and terminal constraints as well as thecontrol and state constraints.
Multi-Objective Optimal Control Problem
The feasible triplet(x∗, u∗, t∗f) is said to be aPareto minimum,if there doesnot exista feasible triplet(x, u, tf) ∈ X such that
ϕi(x(tf), tf)) ≤ ϕi(x∗(t∗f), t
∗f) for all i = 1, . . . , r .
ϕk(x(tf), tf)) < ϕk(x∗(t∗f), t
∗f) for one k ∈ {1, . . . , r} .
Multi-Objective Optimal Control Problem
The feasible triplet(x∗, u∗, t∗f) is said to be aPareto minimum,if there doesnot exista feasible triplet(x, u, tf) ∈ X such that
ϕi(x(tf), tf)) ≤ ϕi(x∗(t∗f), t
∗f) for all i = 1, . . . , r .
ϕk(x(tf), tf)) < ϕk(x∗(t∗f), t
∗f) for one k ∈ {1, . . . , r} .
On the other hand,(x∗, u∗, t∗f) is said to be aweak Paretominimum, if there doesnot exist(x, u, tf) ∈ X such that
ϕi(x(tf), tf)) < ϕi(x∗(t∗f), t
∗f) for all i = 1, . . . , r .
The Pareto Front
The set of all objective functional values at the Pareto andweak Pareto minima is said to be thePareto front(or efficientset) of Problem (OCPm) in the objective value space.
The Pareto Front
The set of all objective functional values at the Pareto andweak Pareto minima is said to be thePareto front(or efficientset) of Problem (OCPm) in the objective value space.
Examples to “generated” Pareto fronts (withr = 2):
0 0.2 0.4 0.6 0.8 1 1.2 1.40
0.2
0.4
0.6
0.8
1
1.2
ϕ1
ϕ2
The Pareto Front
The set of all objective functional values at the Pareto andweak Pareto minima is said to be thePareto front(or efficientset) of Problem (OCPm) in the objective value space.
Examples to “generated” Pareto fronts (withr = 2):
0 0.2 0.4 0.6 0.8 1 1.2 1.40
0.2
0.4
0.6
0.8
1
1.2
ϕ1
ϕ2
6 8 10 12 14 16 18 200
1
2
3
4
5
6
7
8
9
10
ϕ1
ϕ2
Scalarization
The “popular”weighted-sum scalarization:
(Pws) min(x,u,tf )∈X
r∑
i=1
wi ϕi(x(tf), tf) .
wherew1, . . . , wr ≥ 0 areweightswith w1 + . . . + wr = 1 .
TheBolza problemcan equivalently be written in this form.
NOT GOOD FOR NON-CONVEX PROBLEMS WITH A NON-CONVEX PARETO FRONT !
We shall illustrate this on an example.
Scalarization
Weighted Tschebychev problem( Tschebychev scalarization) :
(Pw) min(x,u,tf )∈X
max{w1 ϕ1(x(tf), tf), . . . , wr ϕr(x(tf), tf)} ,
wherew1, . . . , wr ≥ 0 areweightswith w1 + . . . + wr = 1 .
Scalarization
Weighted Tschebychev problem(Tschebychev scalarization) :
(Pw) min(x,u,tf )∈X
max{w1 ϕ1(x(tf), tf), . . . , wr ϕr(x(tf), tf)} ,
wherew1, . . . , wr ≥ 0 areweightswith w1 + . . . + wr = 1 .
Theorem 1 The triplet(x∗, u∗, t∗f) is a weak Pareto minimumof (OCPm), if and only if (x∗, u∗, t∗f) is a solution of(Pw) forsomew1, . . . , wr > 0.
Scalarization
A smoothre-formulation of (Pw):
(OCPw)
minα≥0
(x,u,tf )∈X
α
subject to w1 ϕ1(x(tf), tf) ≤ α ,...
wr ϕr(x(tf), tf) ≤ α .
Numerical method: ”Discretize then Optimize”.
Use Applied Modeling Language AMPL (Fourer et al.) andInterior-Point Optimization Solver IPOPT (Wachter et al.).
ScalarizationCaser = 2 : w = w1 ∈ [0, 1] , w2 = 1 − w .Let β∗ = (β∗
1, β∗2) ≤ 0 be a so-calledutopia point.
Consider thesmoothcontrol problem (Pw):
(OCPw)
minα≥0
(x,u,tf )∈X
α
subject to w (ϕ1(x(tf), tf) − β∗1) ≤ α ,
(1 − w) (ϕ2(x(tf), tf) − β∗2) ≤ α .
We shall determine a ”meaningful interval”
w ∈ [w0, wf ] , 0 ≤ w0 < wf ≤ 1 ,
such that thesolution is the sameforw ∈ [0, w0] and w ∈ [wf , 1] .
Boundary weights
Pareto front
ϕ1
ϕ2
(ϕ∗1, ϕ2)
(ϕ1, ϕ∗2)
wf (ϕ1 − β∗1) = (1 − wf) (ϕ2 − β∗
2)
w0 (ϕ1 − β∗1) = (1 − w0) (ϕ2 − β∗
2)
(β∗1, β
∗2)
Example 1 - Tumour Anti-AngiogenesisTumour Anti-Angiogenesis: J. Folkman (1972) et al.
Control model: Ledzewicz, M., Schattler (2007,2011).State and controlvariables:
p : primary tumour volume [mm3]
q : carrying capacity, or endothelial support [mm3]
u : anti-angiogenic agent
Example 1 - Tumour Anti-angiogenesis
Two competing objective functionalsto minimize:
• Final tumour volume:p(tf)
• Final tumour volume,plus a factor of thetotal amount ofanti-angiogenic (toxic) agent administered:
p(tf) + 140
∫ tf
0
u(t) dt .
The duration of therapytf is free.
Example 1 - Tumour Anti-angiogenesis
(E1)
min (p(tf), p(tf) + 140 y(tf))
s.t. p = −0.084 p lnp
q, p(0) = 8000 ,
q = 5.85 q2/3 − 0.00873 q4/3 − 0.02 q − 0.15 q u ,
q(0) = 10000 ,
y = u , y(0) = 0 ,
y(tf) ≤ 45 , 0 ≤ u(t) ≤ 15 .
Controlu appearslinearly: bang-bang and singular arcs.
Example 1 - Tumour Anti-angiogenesisScalarization of Problem (E1):For a fixed w ∈ [0, 1] , solve
(E1w)
min α
s.t. p = −0.084 p lnp
q, p(0) = 8000 ,
q = 5.85 q2/3 − 0.00873 q4/3 − 0.02 q − 0.15 q u ,
q(0) = 10000 ,
y = u , y(0) = 0 ,
y(tf) ≤ 45 , 0 ≤ u(t) ≤ 15 ,
w p(tf) ≤ α ,
(1 − w) (p(tf) + 140 y(tf)) ≤ α .
Example 1 - Tumour Anti-angiogenesis
Theswitching function:
σu(t) = −0.15λq(t) q(t) + λy(t) .
Example 1 - Tumour Anti-angiogenesis
Theswitching function:
σu(t) = −0.15λq(t) q(t) + λy(t) .
Singularfeedbackcontrol(Ledzewicz, Schattler, 2007, 2011):
using(t) =1
0.02
(5.85 − 0.00873 q2/3(t)
q1/3(t)
+ 3 (0.084)5.85 + 0.00873 q2/3(t)
5.85 − 0.00873 q2/3(t)− 0.02
).
Example 1 - Tumour Anti-angiogenesis
The optimal controlturns out to bebang–singular–bang, inparticular, to befull dose–partial dose–no dose.
The switching times found accurately by means ofarcparameterization: (Kaya & Noakes 2003, Maurer et al. 2005)
weights\switching times t1 t2 tf0.792 ≤ w ≤ 1 1.341 5.063 9.378
w = 0.7 1.341 3.141 8.1210 ≤ w ≤ 0.628 1.341 1.605 7.189
Themeaningful intervalis w ∈ [ 0.628 , 0.792 ] .
Example 1 - Tumour Anti-angiogenesis
2000 2500 3000 3500 4000 4500
7400
7600
7800
8000
8200
8400
8600
p(tf ) [mm3]
p(t
f)+
140
y(t
f)
0.792 ≤ w ≤ 1
ϕ
0 2 4 6 8 10−10
0
10
20
30
40
50
60
switc
hing
func
tion,
σu
time t [days]
0.792 ≤ w ≤ 1
0 2000 4000 6000 8000 100000
1000
2000
3000
4000
5000
6000
7000
8000
9000
endothelial support q
tum
or v
olum
e p
0.792 ≤ w ≤ 1
0 2 4 6 8 10
0
2
4
6
8
10
12
14
16
time t [days]
angi
ogen
ic c
ontr
ol u
0.792 ≤ w ≤ 1
Example 1 - Tumour Anti-angiogenesis
2000 2500 3000 3500 4000 4500
7400
7600
7800
8000
8200
8400
8600
p(tf ) [mm3]
p(t
f)+
140
y(t
f)
0.792 ≤ w ≤ 10 ≤ w ≤ 0.628
ϕ
0 2 4 6 8 10−10
0
10
20
30
40
50
60
switc
hing
func
tion,
σu
time t [days]
0.792 ≤ w ≤ 1
0 ≤ w ≤ 0.628
0 2000 4000 6000 8000 100000
1000
2000
3000
4000
5000
6000
7000
8000
9000
endothelial support q
tum
or v
olum
e p
0.792 ≤ w ≤ 1
0 ≤ w ≤ 0.628
0 2 4 6 8 10
0
2
4
6
8
10
12
14
16
time t [days]
angi
ogen
ic c
ontr
ol u
0.792 ≤ w ≤ 1
0 ≤ w ≤ 0.628
Example 1 - Tumour Anti-angiogenesis
2000 2500 3000 3500 4000 4500
7400
7600
7800
8000
8200
8400
8600
p(tf ) [mm3]
p(t
f)+
140
y(t
f)
0.792 ≤ w ≤ 1w = 0.70 ≤ w ≤ 0.628
ϕ
0 2 4 6 8 10−10
0
10
20
30
40
50
60
switc
hing
func
tion,
σu
time t [days]
0.792 ≤ w ≤ 1
w = 0.7
0 ≤ w ≤ 0.628
0 2000 4000 6000 8000 100000
1000
2000
3000
4000
5000
6000
7000
8000
9000
endothelial support q
tum
or v
olum
e p
0.792 ≤ w ≤ 1
w = 0.7
0 ≤ w ≤ 0.628
0 2 4 6 8 10
0
2
4
6
8
10
12
14
16
time t [days]
angi
ogen
ic c
ontr
ol u
0.792 ≤ w ≤ 1
w = 0.7
0 ≤ w ≤ 0.628
Example 2 - Fed-batch Bioreactor
(Logist, Houska, Diehl, van Impe, 2010)
Thestate and controlvariables:
x1 : biomass [g]
x2 : substrate [g]
x3 : product, lysine [g]
x4 : fermenter volume [g]
u : volumetric rate of the feed stream [lt/h]
Example 2 - Fed-batch BioreactorTwo competing objective functionalsto maximize:
• The ratio of the product formed and the process duration,i.e., theproductivity:
ϕ1(x(tf), tf) =x3(tf)
tf.
• The ratio of the product formed and the mass of the substrateadded, i.e., theyield :
ϕ2(x(tf), tf) =x3(tf)
2.8 (x4(tf) − 5).
The duration of the processtf is free.
Example 2 - Fed-batch BioreactorTwo competing objective functionalsto maximize:
• The ratio of the product formed and the process duration,i.e., theproductivity:
ϕ1(x(tf), tf) =x3(tf)
tf.
• The ratio of the product formed and the mass of the substrateadded, i.e., theyield :
ϕ2(x(tf), tf) =x3(tf)
2.8 (x4(tf) − 5).
The duration of the processtf is free.
Example 2 - Fed-batch BioreactorTwo competing objective functionalsto maximize:
• The ratio of the product formed and the process duration,i.e., theproductivity:
ϕ1(x(tf), tf) =x3(tf)
tf.
• The ratio of the product formed and the mass of the substrateadded, i.e., theyield :
ϕ2(x(tf), tf) =x3(tf)
2.8 (x4(tf) − 5).
The duration of the processtf is free.
Example 2 - Fed-batch Bioreactor
(E2)
min
(−
x3(tf)
tf, −
x3(tf)
2.8 (x4(tf) − 5)
)
s.t. x1 =
(0.125
x2
x4
)x1 , x1(0) = 0.1 ,
x2 = −
(0.125
0.135
x2
x4
)x1 + 2.8u , x2(0) = 14 ,
x3 = −384
(0.125
x2
x4
)2
+ 134
(0.125
x2
x4
), x3(0) = 0 ,
x4 = u , x4(0) = 5 ,
0 ≤ u(t) ≤ 2 , 5 ≤ x4(t) ≤ 20
0 ≤ tf ≤ 40 , 20 ≤ 2.8 (x4(tf) − 5) ≤ 42 .
Expectbang-bang, singular and boundary arcs.
Example 2 - Fed-batch Bioreactor
Scalarization of Problem (E2):For a fixed w ∈ [0, 1] , solve
(E2w)
min α
s.t. dynamical and terminal constraints,
control and state constraints,
−w
(x3(tf)
tf+ β∗
1
)≤ α ,
−(1 − w)
(x3(tf)
2.8 (x4(tf) − 5)+ β∗
2
)≤ α ,
whereβ∗ ≤ 0 is a suitableutopia point.
Example 2 - Fed-batch Bioreactor
Thestate constraint5 ≤ x4(t) ≤ 20 here is oforder one.
Theswitching functionis
σu(t) = 2.8λ2(t) q(t) + λ4(t) .
Thesingular arcs are of order one, but the singular control cannot be obtained in feedback form.
Example 2 - Fed-batch Bioreactor
−25 −20 −15 −10
−22
−20
−18
−16
−14
−12
0 ≤ w ≤ 0.217
−(y
ield
)
−(productivity)0 10 20 30 40
0
0.2
0.4
0.6
0.8
1
σ u
t [hour]
0 ≤ w ≤ 0.217
0 10 20 30 400
5
10
15
20
t [hour]
x 3/100
, x 4 (
thic
ker
curv
e)
0 10 20 30 400
0.5
1
1.5
2
t [hour]
feed
ing
rate
u [l
/h]
0 ≤ w ≤ 0.217
Example 2 - Fed-batch Bioreactor
−25 −20 −15 −10
−22
−20
−18
−16
−14
−12
0 ≤ w ≤ 0.217w = 0.3
−(y
ield
)
−(productivity)0 10 20 30 40
0
0.2
0.4
0.6
0.8
1
σ u
t [hour]
0 ≤ w ≤ 0.217
w = 0.3
0 10 20 30 400
5
10
15
20
t [hour]
x 3/100
, x 4 (
thic
ker
curv
e)
0 10 20 30 400
0.5
1
1.5
2
t [hour]
feed
ing
rate
u [l
/h]
0 ≤ w ≤ 0.217
w = 0.3
Example 2 - Fed-batch Bioreactor
−25 −20 −15 −10
−22
−20
−18
−16
−14
−12
0 ≤ w ≤ 0.217w = 0.3w = 0.4
−(y
ield
)
−(productivity)0 10 20 30 40
0
0.2
0.4
0.6
0.8
1
σ u
t [hour]
0 ≤ w ≤ 0.217
w = 0.3
w = 0.4
0 10 20 30 400
5
10
15
20
t [hour]
x 3/100
, x 4 (
thic
ker
curv
e)
0 10 20 30 400
0.5
1
1.5
2
t [hour]
feed
ing
rate
u [l
/h]
0 ≤ w ≤ 0.217
w = 0.3w = 0.4
Example 2 - Fed-batch Bioreactor
−25 −20 −15 −10
−22
−20
−18
−16
−14
−12
0 ≤ w ≤ 0.217
w = 0.3
w = 0.4
0.517 ≤ w ≤ 1
−(y
ield
)
−(productivity)0 10 20 30 40
0
0.2
0.4
0.6
0.8
1
σ u
t [hour]
0 ≤ w ≤ 0.217
w = 0.3
w = 0.4
0.517 ≤ w ≤ 1
0 10 20 30 400
5
10
15
20
t [hour]
x 3/100
, x 4 (
thic
ker
curv
e)
0 10 20 30 400
0.5
1
1.5
2
t [hour]
feed
ing
rate
u [l
/h]
0 ≤ w ≤ 0.217
w = 0.3w = 0.40.517 ≤ w ≤ 1
Example 2 - Fed-batch Bioreactor
Weighted-sum scalarizationcannot generate “nonconvexparts” of the Pareto front:
−25 −20 −15 −10
−22
−20
−18
−16
−14
−12
productivity
yiel
d
References
References[1] H. BONNEL AND Y. C. KAYA , Optimization over the efficient set of convex multi-objective optimal control problems. J.
Optimization Theory and Applications147, 93–112 (2010).
[2] J. JAHN, Vector Optimization: Theory, Applications, and Extensions, Springer-Verlag, Berlin, Heidelberg, 2004.
[3] Y. K AYA AND H. MAURER, A numerical method for generating the Pareto front of nonconvex multi-objective optimalcontrol problems, to appear in Computational Optimization and Applications.
[4] U. L EDZEWICZ, H. MAURER AND H. SCHATTLER, Optimal and suboptimal protocols for a mathematical model for tumoranti-angiogenesis in combination with chemotherapy, Mathematical Bioscieneces and Engineering,8, 307–323, 2011.
[5] F. LOGIST, B. HOUSKA, M. DIEHL AND J. VAN IMPE, Fast Pareto set generation for nonlinear optimal control problemswith multiple objectives, Struct. Multidisciplinary Optimization,42, 591–603, 2010.
Future Work
1. Multi-objective control problems withr ≥ 3 objectives.
2. Optimization over the Pareto front using sensitivity analysis.
3. Multi-objective control problems for elliptic and parabolicequations.
Conclusion
We hope you’ll find your way now !
PSfrag
Thank you for your attention!