Numerical Methods for Multi-Objective Optimal Control Problems

Numerical Methods for Multi-ObjectiveOptimal Control Problems

C. Yalcın Kaya

University of South Australia

Helmut Maurer

University of Munster

Workshop on Optimaland Model Predictive Control

Bayreuth, 9-13 September 2013

Outline

1 Multi-Objective Optimal Control problem

Outline


2 The Pareto Front

Outline


2 The Pareto Front

3 Scalarization : Weighted-Sum andTschebychev Approach

Outline


2 The Pareto Front

3 Scalarization : Weighted-Sum andTschebychev Approach

4 Example 1: Tumour Anti-Angiogenesis

Outline


2 The Pareto Front

3 Scalarization : Weighted Sum andTschebychev Approach

4 Example 1: Tumour Anti-Angiogenesis

5 Example 2: Fed-Batch Bioreactor

Single-Objective Optimal Control Problem

(OCPm)

min (ϕ1(x(tf), tf)

s.t. x(t) = f(x(t), u(t), t) , a.e.t ∈ [0, tf ] ,

φ(x(0), x(tf), tf) = 0 ,

φ(x(0), x(tf), tf) ≤ 0 ,

C(x(t), u(t), t) ≤ 0 , a.e.t ∈ [0, tf ] ,

S(x(t), t) ≤ 0 , all t ∈ [0, tf ] ,

statevariablex ∈ IRn, controlvariableu ∈ IRm,terminal time tf is fixed or free.

Multi -Objective Optimal Control Problem

(OCPm)

min (ϕ1(x(tf), tf), . . . , ϕr(x(tf), tf))

s.t. x(t) = f(x(t), u(t), t) , a.e. t ∈ [0, tf ] ,

φ(x(0), x(tf), tf) = 0 ,

φ(x(0), x(tf), tf) ≤ 0 ,

C(x(t), u(t), t) ≤ 0 , a.e. t ∈ [0, tf ] ,

S(x(t), t) ≤ 0 , all t ∈ [0, tf ] ,

statevariablex ∈ IRn, controlvariableu ∈ IRm,terminal time tf fixed or free.

Multi -Objective Optimal Control Problem

(OCPm)

min (ϕ1(x(tf), tf), . . . , ϕr(x(tf), tf))

s.t. x(t) = f(x(t), u(t), t) , a.e. t ∈ [0, tf ] ,

φ(x(0), x(tf), tf) = 0 ,

φ(x(0), x(tf), tf) ≤ 0 ,

C(x(t), u(t), t) ≤ 0 , a.e. t ∈ [0, tf ] ,

S(x(t), t) ≤ 0 , all t ∈ [0, tf ] ,

statevariablex ∈ IRn, controlvariableu ∈ IRm,terminal time tf fixed or free.Assumption:ϕi(x(tf), tf) ≥ 0 for all i = 1, . . . , r.

Multi-Objective Optimal Control Problem

Thefeasible setX consists of triplets

(x, u, tf) ∈ W 1,∞(0, tf ; IRn) × L∞(0, tf ; IR

n) × IR+

satisfying the dynamic and terminal constraints as well as thecontrol and state constraints.


The feasible triplet(x∗, u∗, t∗f) is said to be aPareto minimum,if there doesnot exista feasible triplet(x, u, tf) ∈ X such that

ϕi(x(tf), tf)) ≤ ϕi(x∗(t∗f), t

∗f) for all i = 1, . . . , r .

ϕk(x(tf), tf)) < ϕk(x∗(t∗f), t

∗f) for one k ∈ {1, . . . , r} .


The feasible triplet(x∗, u∗, t∗f) is said to be aPareto minimum,if there doesnot exista feasible triplet(x, u, tf) ∈ X such that

ϕi(x(tf), tf)) ≤ ϕi(x∗(t∗f), t

∗f) for all i = 1, . . . , r .

ϕk(x(tf), tf)) < ϕk(x∗(t∗f), t

∗f) for one k ∈ {1, . . . , r} .

On the other hand,(x∗, u∗, t∗f) is said to be aweak Paretominimum, if there doesnot exist(x, u, tf) ∈ X such that

ϕi(x(tf), tf)) < ϕi(x∗(t∗f), t

∗f) for all i = 1, . . . , r .

The Pareto Front

The set of all objective functional values at the Pareto andweak Pareto minima is said to be thePareto front(or efficientset) of Problem (OCPm) in the objective value space.

The Pareto Front


Examples to “generated” Pareto fronts (withr = 2):

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.2

0.4

0.6

0.8

1

1.2

ϕ1

ϕ2

The Pareto Front


Examples to “generated” Pareto fronts (withr = 2):

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.2

0.4

0.6

0.8

1

1.2

ϕ1

ϕ2

6 8 10 12 14 16 18 200

1

2

3

4

5

6

7

8

9

10

ϕ1

ϕ2

Scalarization

The “popular”weighted-sum scalarization:

(Pws) min(x,u,tf )∈X

r∑

i=1

wi ϕi(x(tf), tf) .

wherew1, . . . , wr ≥ 0 areweightswith w1 + . . . + wr = 1 .

TheBolza problemcan equivalently be written in this form.

NOT GOOD FOR NON-CONVEX PROBLEMS WITH A NON-CONVEX PARETO FRONT !

We shall illustrate this on an example.

Scalarization

Weighted Tschebychev problem( Tschebychev scalarization) :

(Pw) min(x,u,tf )∈X

max{w1 ϕ1(x(tf), tf), . . . , wr ϕr(x(tf), tf)} ,


Scalarization

Weighted Tschebychev problem(Tschebychev scalarization) :

(Pw) min(x,u,tf )∈X

max{w1 ϕ1(x(tf), tf), . . . , wr ϕr(x(tf), tf)} ,


Theorem 1 The triplet(x∗, u∗, t∗f) is a weak Pareto minimumof (OCPm), if and only if (x∗, u∗, t∗f) is a solution of(Pw) forsomew1, . . . , wr > 0.

Scalarization

A smoothre-formulation of (Pw):

(OCPw)

minα≥0

(x,u,tf )∈X

α

subject to w1 ϕ1(x(tf), tf) ≤ α ,...

wr ϕr(x(tf), tf) ≤ α .

Numerical method: ”Discretize then Optimize”.

Use Applied Modeling Language AMPL (Fourer et al.) andInterior-Point Optimization Solver IPOPT (Wachter et al.).

ScalarizationCaser = 2 : w = w1 ∈ [0, 1] , w2 = 1 − w .Let β∗ = (β∗

1, β∗2) ≤ 0 be a so-calledutopia point.

Consider thesmoothcontrol problem (Pw):

(OCPw)

minα≥0

(x,u,tf )∈X

α

subject to w (ϕ1(x(tf), tf) − β∗1) ≤ α ,

(1 − w) (ϕ2(x(tf), tf) − β∗2) ≤ α .

We shall determine a ”meaningful interval”

w ∈ [w0, wf ] , 0 ≤ w0 < wf ≤ 1 ,

such that thesolution is the sameforw ∈ [0, w0] and w ∈ [wf , 1] .

Boundary weights

Pareto front

ϕ1

ϕ2

(ϕ∗1, ϕ2)

(ϕ1, ϕ∗2)

wf (ϕ1 − β∗1) = (1 − wf) (ϕ2 − β∗

2)

w0 (ϕ1 − β∗1) = (1 − w0) (ϕ2 − β∗

2)

(β∗1, β

∗2)

Example 1 - Tumour Anti-AngiogenesisTumour Anti-Angiogenesis: J. Folkman (1972) et al.

Control model: Ledzewicz, M., Schattler (2007,2011).State and controlvariables:

p : primary tumour volume [mm3]

q : carrying capacity, or endothelial support [mm3]

u : anti-angiogenic agent

Example 1 - Tumour Anti-angiogenesis

Two competing objective functionalsto minimize:

• Final tumour volume:p(tf)

• Final tumour volume,plus a factor of thetotal amount ofanti-angiogenic (toxic) agent administered:

p(tf) + 140

∫ tf

0

u(t) dt .

The duration of therapytf is free.


(E1)

min (p(tf), p(tf) + 140 y(tf))

s.t. p = −0.084 p lnp

q, p(0) = 8000 ,

q = 5.85 q2/3 − 0.00873 q4/3 − 0.02 q − 0.15 q u ,

q(0) = 10000 ,

y = u , y(0) = 0 ,

y(tf) ≤ 45 , 0 ≤ u(t) ≤ 15 .

Controlu appearslinearly: bang-bang and singular arcs.

Example 1 - Tumour Anti-angiogenesisScalarization of Problem (E1):For a fixed w ∈ [0, 1] , solve

(E1w)

min α

s.t. p = −0.084 p lnp

q, p(0) = 8000 ,

q = 5.85 q2/3 − 0.00873 q4/3 − 0.02 q − 0.15 q u ,

q(0) = 10000 ,

y = u , y(0) = 0 ,

y(tf) ≤ 45 , 0 ≤ u(t) ≤ 15 ,

w p(tf) ≤ α ,

(1 − w) (p(tf) + 140 y(tf)) ≤ α .


Theswitching function:

σu(t) = −0.15λq(t) q(t) + λy(t) .


Theswitching function:

σu(t) = −0.15λq(t) q(t) + λy(t) .

Singularfeedbackcontrol(Ledzewicz, Schattler, 2007, 2011):

using(t) =1

0.02

(5.85 − 0.00873 q2/3(t)

q1/3(t)

+ 3 (0.084)5.85 + 0.00873 q2/3(t)

5.85 − 0.00873 q2/3(t)− 0.02

).


The optimal controlturns out to bebang–singular–bang, inparticular, to befull dose–partial dose–no dose.

The switching times found accurately by means ofarcparameterization: (Kaya & Noakes 2003, Maurer et al. 2005)

weights\switching times t1 t2 tf0.792 ≤ w ≤ 1 1.341 5.063 9.378

w = 0.7 1.341 3.141 8.1210 ≤ w ≤ 0.628 1.341 1.605 7.189

Themeaningful intervalis w ∈ [ 0.628 , 0.792 ] .


2000 2500 3000 3500 4000 4500

7400

7600

7800

8000

8200

8400

8600

p(tf ) [mm3]

p(t

f)+

140

y(t

f)

0.792 ≤ w ≤ 1

ϕ

0 2 4 6 8 10−10

0

10

20

30

40

50

60

switc

hing

func

tion,

σu

time t [days]

0.792 ≤ w ≤ 1

0 2000 4000 6000 8000 100000

1000

2000

3000

4000

5000

6000

7000

8000

9000

endothelial support q

tum

or v

olum

e p

0.792 ≤ w ≤ 1

0 2 4 6 8 10

0

2

4

6

8

10

12

14

16

time t [days]

angi

ogen

ic c

ontr

ol u

0.792 ≤ w ≤ 1


2000 2500 3000 3500 4000 4500

7400

7600

7800

8000

8200

8400

8600

p(tf ) [mm3]

p(t

f)+

140

y(t

f)

0.792 ≤ w ≤ 10 ≤ w ≤ 0.628

ϕ

0 2 4 6 8 10−10

0

10

20

30

40

50

60

switc

hing

func

tion,

σu

time t [days]

0.792 ≤ w ≤ 1

0 ≤ w ≤ 0.628

0 2000 4000 6000 8000 100000

1000

2000

3000

4000

5000

6000

7000

8000

9000


tum

or v

olum

e p

0.792 ≤ w ≤ 1

0 ≤ w ≤ 0.628

0 2 4 6 8 10

0

2

4

6

8

10

12

14

16

time t [days]

angi

ogen

ic c

ontr

ol u

0.792 ≤ w ≤ 1

0 ≤ w ≤ 0.628


2000 2500 3000 3500 4000 4500

7400

7600

7800

8000

8200

8400

8600

p(tf ) [mm3]

p(t

f)+

140

y(t

f)

0.792 ≤ w ≤ 1w = 0.70 ≤ w ≤ 0.628

ϕ

0 2 4 6 8 10−10

0

10

20

30

40

50

60

switc

hing

func

tion,

σu

time t [days]

0.792 ≤ w ≤ 1

w = 0.7

0 ≤ w ≤ 0.628

0 2000 4000 6000 8000 100000

1000

2000

3000

4000

5000

6000

7000

8000

9000


tum

or v

olum

e p

0.792 ≤ w ≤ 1

w = 0.7

0 ≤ w ≤ 0.628

0 2 4 6 8 10

0

2

4

6

8

10

12

14

16

time t [days]

angi

ogen

ic c

ontr

ol u

0.792 ≤ w ≤ 1

w = 0.7

0 ≤ w ≤ 0.628

Example 2 - Fed-batch Bioreactor

(Logist, Houska, Diehl, van Impe, 2010)

Thestate and controlvariables:

x1 : biomass [g]

x2 : substrate [g]

x3 : product, lysine [g]

x4 : fermenter volume [g]

u : volumetric rate of the feed stream [lt/h]

Example 2 - Fed-batch BioreactorTwo competing objective functionalsto maximize:

• The ratio of the product formed and the process duration,i.e., theproductivity:

ϕ1(x(tf), tf) =x3(tf)

tf.

• The ratio of the product formed and the mass of the substrateadded, i.e., theyield :

ϕ2(x(tf), tf) =x3(tf)

2.8 (x4(tf) − 5).

The duration of the processtf is free.


(E2)

min

(−

x3(tf)

tf, −

x3(tf)

2.8 (x4(tf) − 5)

)

s.t. x1 =

(0.125

x2

x4

)x1 , x1(0) = 0.1 ,

x2 = −

(0.125

0.135

x2

x4

)x1 + 2.8u , x2(0) = 14 ,

x3 = −384

(0.125

x2

x4

)2

+ 134

(0.125

x2

x4

), x3(0) = 0 ,

x4 = u , x4(0) = 5 ,

0 ≤ u(t) ≤ 2 , 5 ≤ x4(t) ≤ 20

0 ≤ tf ≤ 40 , 20 ≤ 2.8 (x4(tf) − 5) ≤ 42 .

Expectbang-bang, singular and boundary arcs.


Scalarization of Problem (E2):For a fixed w ∈ [0, 1] , solve

(E2w)

min α

s.t. dynamical and terminal constraints,

control and state constraints,

−w

(x3(tf)

tf+ β∗

1

)≤ α ,

−(1 − w)

(x3(tf)

2.8 (x4(tf) − 5)+ β∗

2

)≤ α ,

whereβ∗ ≤ 0 is a suitableutopia point.


Thestate constraint5 ≤ x4(t) ≤ 20 here is oforder one.

Theswitching functionis

σu(t) = 2.8λ2(t) q(t) + λ4(t) .

Thesingular arcs are of order one, but the singular control cannot be obtained in feedback form.


−25 −20 −15 −10

−22

−20

−18

−16

−14

−12

0 ≤ w ≤ 0.217

−(y

ield

)

−(productivity)0 10 20 30 40

0

0.2

0.4

0.6

0.8

1

σ u

t [hour]

0 ≤ w ≤ 0.217

0 10 20 30 400

5

10

15

20

t [hour]

x 3/100

, x 4 (

thic

ker

curv

e)

0 10 20 30 400

0.5

1

1.5

2

t [hour]

feed

ing

rate

u [l

/h]

0 ≤ w ≤ 0.217


−25 −20 −15 −10

−22

−20

−18

−16

−14

−12

0 ≤ w ≤ 0.217w = 0.3

−(y

ield

)


0

0.2

0.4

0.6

0.8

1

σ u

t [hour]

0 ≤ w ≤ 0.217

w = 0.3

0 10 20 30 400

5

10

15

20

t [hour]

x 3/100

, x 4 (

thic

ker

curv

e)

0 10 20 30 400

0.5

1

1.5

2

t [hour]

feed

ing

rate

u [l

/h]

0 ≤ w ≤ 0.217

w = 0.3


−25 −20 −15 −10

−22

−20

−18

−16

−14

−12

0 ≤ w ≤ 0.217w = 0.3w = 0.4

−(y

ield

)


0

0.2

0.4

0.6

0.8

1

σ u

t [hour]

0 ≤ w ≤ 0.217

w = 0.3

w = 0.4

0 10 20 30 400

5

10

15

20

t [hour]

x 3/100

, x 4 (

thic

ker

curv

e)

0 10 20 30 400

0.5

1

1.5

2

t [hour]

feed

ing

rate

u [l

/h]

0 ≤ w ≤ 0.217

w = 0.3w = 0.4


−25 −20 −15 −10

−22

−20

−18

−16

−14

−12

0 ≤ w ≤ 0.217

w = 0.3

w = 0.4

0.517 ≤ w ≤ 1

−(y

ield

)


0

0.2

0.4

0.6

0.8

1

σ u

t [hour]

0 ≤ w ≤ 0.217

w = 0.3

w = 0.4

0.517 ≤ w ≤ 1

0 10 20 30 400

5

10

15

20

t [hour]

x 3/100

, x 4 (

thic

ker

curv

e)

0 10 20 30 400

0.5

1

1.5

2

t [hour]

feed

ing

rate

u [l

/h]

0 ≤ w ≤ 0.217

w = 0.3w = 0.40.517 ≤ w ≤ 1


Weighted-sum scalarizationcannot generate “nonconvexparts” of the Pareto front:

−25 −20 −15 −10

−22

−20

−18

−16

−14

−12

productivity

yiel

d

References

References[1] H. BONNEL AND Y. C. KAYA , Optimization over the efficient set of convex multi-objective optimal control problems. J.

Optimization Theory and Applications147, 93–112 (2010).

[2] J. JAHN, Vector Optimization: Theory, Applications, and Extensions, Springer-Verlag, Berlin, Heidelberg, 2004.

[3] Y. K AYA AND H. MAURER, A numerical method for generating the Pareto front of nonconvex multi-objective optimalcontrol problems, to appear in Computational Optimization and Applications.

[4] U. L EDZEWICZ, H. MAURER AND H. SCHATTLER, Optimal and suboptimal protocols for a mathematical model for tumoranti-angiogenesis in combination with chemotherapy, Mathematical Bioscieneces and Engineering,8, 307–323, 2011.

[5] F. LOGIST, B. HOUSKA, M. DIEHL AND J. VAN IMPE, Fast Pareto set generation for nonlinear optimal control problemswith multiple objectives, Struct. Multidisciplinary Optimization,42, 591–603, 2010.

Future Work

1. Multi-objective control problems withr ≥ 3 objectives.

2. Optimization over the Pareto front using sensitivity analysis.

3. Multi-objective control problems for elliptic and parabolicequations.

Conclusion

We hope you’ll find your way now !

PSfrag

Thank you for your attention!

Numerical Methods for Multi-Objective Optimal Control Problems

Documents

Transcript of Numerical Methods for Multi-Objective Optimal Control Problems