Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 ·...

Transcript of Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 ·...

Weak Dynamic Programming for ViscositySolutions

B. Bouchard

Ceremade, Univ. Paris-Dauphine, and, Crest, Ensae

May 2009

Joint work with Nizar Touzi, CMAP, Ecole Polytechnique
Motivations

• Consider the control problem in standard form

V (t, x) := supν∈U

J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

• To derive the related HJB equation, one uses the DPP

′′V (t, x) = supν∈U

E [V (τ,X ντ )|X νt = x ]′′

• Usually not that easy to prove

a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

• Our aim is to provide a weak version, much easier to prove.
Motivations






E [V (τ,X ντ )|X νt = x ]′′




Motivations






E [V (τ,X ντ )|X νt = x ]′′




Motivations






E [V (τ,X ντ )|X νt = x ]′′




Motivations






E [V (τ,X ντ )|X νt = x ]′′




Motivations






E [V (τ,X ντ )|X νt = x ]′′




Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :

U0, a collection of Rd -valued progressively measurable processes.

• Controlled process :

(τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0rcll(Rd )

with [0,T ]× Rd ⊂ S ⊂{

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.
Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :




with [0,T ]× Rd ⊂ S ⊂{

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.
Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :




with [0,T ]× Rd ⊂ S ⊂{

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.
Reward and Value functions

• Reward function

J(t, x ; ν) := E[f(X νt,x(T )

)]defined for controls ν in

U :={ν ∈ U0 : E

[|f (X νt,x(T ))|

]

• Reward function

J(t, x ; ν) := E[f(X νt,x(T )


U :={ν ∈ U0 : E

[|f (X νt,x(T ))|

]

• Reward function

J(t, x ; ν) := E[f(X νt,x(T )


U :={ν ∈ U0 : E

[|f (X νt,x(T ))|

]
AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.






E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)


E[f(X ν̄t,x(T )

)|Fθ]







E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)


E[f(X ν̄t,x(T )

)|Fθ]







E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)


E[f(X ν̄t,x(T )

)|Fθ]





A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)


E[f(X ν̄t,x(T )

)|Fθ]





A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)


E[f(X ν̄t,x(T )

)|Fθ]

The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X νt,x(τ))

]′′

• Easy inequality : V (t, x) ≤ supν∈Ut E[V (τ,X νt,x(τ))

]

Proof :

V (t, x) = supν∈Ut

E[E[f (X νt,x(T ))|Fτ

]]where for some ν̃ω ∈ Uτ(ω)

E[f (X νt,x(T ))|Fτ

](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))



]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

[V (τ,X νt,x(τ))

]

Proof :





](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))




[V (τ,X νt,x(τ))

]Proof :



]]

where for some ν̃ω ∈ Uτ(ω)


](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))




[V (τ,X νt,x(τ))

]Proof :





](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))




[V (τ,X νt,x(τ))

]Proof :





](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))
The case where J(·; ν) is l.s.c. and V is continuous

• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)



]′′• Easy inequality :

V (t, x) ≤ supν∈Ut


]

• More difficult one :

V (t, x) ≥ supν∈Ut


]

• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)



]′′• Easy inequality :

V (t, x) ≤ supν∈Ut


]• More difficult one :



]

Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

1Ai (τ,Xνt,x(τ))ν

i .



on (ti − ri , ti ]× Bri (xi )

and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1


i .



on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .

Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1


i .



on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1


i .

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1

J(τ,X νt,x(τ); νi )1Ai

(τ,X νt,x(τ)

)

≥∑i≥1

(V (τ,X νt,x(τ))− 3ε

)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]

≥ E[V(τ,X νt,x(τ)

)]− 3ε .

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1


(τ,X νt,x(τ)

)≥

∑i≥1


)1Ai(τ,X νt,x(τ)

)

= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]


)]− 3ε .

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1


(τ,X νt,x(τ)

)≥

∑i≥1


)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]


)]− 3ε .

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1


(τ,X νt,x(τ)

)≥

∑i≥1


)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]


)]− 3ε .

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1


(τ,X νt,x(τ)

)≥

∑i≥1


)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]


)]− 3ε .

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1


(τ,X νt,x(τ)

)≥

∑i≥1


)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]


)]− 3ε .
Problems

• The lower-semicontinuity of J(·; ν) is very important in thisproof

: It is in general not difficult to obtain.

• The upper-semicontinuity of V is also very important

: It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).
Problems

• The lower-semicontinuity of J(·; ν) is very important in thisproof : It is in general not difficult to obtain.


Problems



Problems


• The upper-semicontinuity of V is also very important : It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).
Observation

To derive the PDE in the viscosity sense, try to obtain :


E[V (τ,X νt,x(τ)

]

but one only needs :


E[ϕ(τ,X νt,x(τ)

]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.ϕ being smooth it should be much easier to prove ! !
Observation



E[V (τ,X νt,x(τ)

]but one only needs :


E[ϕ(τ,X νt,x(τ)

]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.

ϕ being smooth it should be much easier to prove ! !
Observation



E[V (τ,X νt,x(τ)

]but one only needs :


E[ϕ(τ,X νt,x(τ)

]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.ϕ being smooth it should be much easier to prove ! !
The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

lim inf(t′,x ′)→(t,x), t′≤t

J(t ′, x ′; ν) ≥ J(t, x ; ν).

Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.

Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

V (t, x) ≥ supν∈Uϕt

E[ϕ(θν ,X νt,x(θ

ν))],

where Uϕt ={ν ∈ Ut : E

[ϕ(θν ,X νt,x(θ

ν))+]


J(t ′, x ′; ν) ≥ J(t, x ; ν).

Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have



ν))],



ν))+]


J(t ′, x ′; ν) ≥ J(t, x ; ν).




ν))],



ν))+]


J(t ′, x ′; ν) ≥ J(t, x ; ν).




ν))],



ν))+]
The weak DPPProof.

For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

1Ai (θν ,X νt,x(θ

ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1

J(θν ,X νt,x(θν); ν i )1Ai

(θν ,X νt,x(θ

ν))

≥∑i≥1

(ϕ(θν ,X νt,x(θ

ν))− 3ε)1Ai(θν ,X νt,x(θ

ν))

= ϕ(θν ,X νt,x(θν))− 3ε
The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that


on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .

Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1


ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X νt,x(θ

ν))

≥∑i≥1



ν))




νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1


ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X νt,x(θ

ν))

≥∑i≥1



ν))




νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1


ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X νt,x(θ

ν))

≥∑i≥1



ν))




νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1


ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X νt,x(θ

ν))

≥∑i≥1



ν))




νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1


ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X νt,x(θ

ν))

≥∑i≥1



ν))

The weak DPP

Proof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that onAi := (ti − ri , ti ]× Bi , disjoint sets that cover [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1


ν))ν i .

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fνθ]]

≥ E[ϕ(θν ,X νt,x(θ

ν))]− 3ε .
The weak DPP

Using test functions makes the proof straightforward :

supν∈Uϕt


ν))]≤ V (t, x)

≤ supν∈Ut

E[V ∗(θν ,X νt,x(θ

ν))]

Remark : If {X νt,x(θν), ν ∈ Ut} is bounded in L∞, one canapproximate V∗ from below by smooth functions and obtain :

supν∈Ut

E[V∗(θν ,X νt,x(θ

ν))]≤ V (t, x) ≤ sup

ν∈UtE[V ∗(θν ,X νt,x(θ

ν))]
The weak DPP


supν∈Uϕt


ν))]≤ V (t, x) ≤ sup


ν))]


supν∈Ut


ν))]≤ V (t, x) ≤ sup


ν))]
The weak DPP


supν∈Uϕt


ν))]≤ V (t, x) ≤ sup


ν))]


supν∈Ut


ν))]≤ V (t, x) ≤ sup


ν))]
Example : Framework

• Controlled process

dX (r) = µ (X (r), νr ) dr + σ (X (r), νr ) dWr

• U = square integrable progressively measurable processes withvalues in U ⊂ Rd

• f is l.s.c with f − with linear growth, µ and σ Lipschitzcontinuous.
Example : Framework




Example : Framework




Example : Verification of the assumptions

For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

⇒ lim inf E[f (X νt′,x ′(T ))

]≥ E

[f (X νt,x(T ))

].







]≥ E

[f (X νt,x(T ))

].







]≥ E

[f (X νt,x(T ))

].







]≥ E

[f (X νt,x(T ))

].



Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :


E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

Proof. Canonical space : W (ω) = ω. Set Ts(ω) := (ωr − ωs)r≥s andωs := (ωr∧s)r≥0.

Eˆf`X νt,x(T )

´|Fθ˜(ω) =

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω))«

dP(Tθ(ω)(ω)) .

=

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω̃))«

dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃ω)

where, ν̃ω(ω̃) := ν(ωθ(ω) + Tθ(ω)(ω̃)) ∈ Uθ(ω).


E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)


Eˆf`X νt,x(T )

´|Fθ˜(ω) =

Zf„


(T )(Tθ(ω)(ω))«

dP(Tθ(ω)(ω)) .

=

Zf„


(T )(Tθ(ω)(ω̃))«

dP(ω̃)




E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)


Eˆf`X νt,x(T )

´|Fθ˜(ω) =

Zf„


(T )(Tθ(ω)(ω))«

dP(Tθ(ω)(ω)) .

=

Zf„


(T )(Tθ(ω)(ω̃))«

dP(ω̃)




E[f(X ν̄t,x(T )

)|Fθ]


Proof.

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf„

Xν̃(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω̃))«

dP(ω̃),

and therefore

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf“X ν̃(Ts (ω̃))θ(ω),Xνt,x (θ)(ω)(T )(Tθ(ω)(ω̃))

”dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃) .


E[f(X ν̄t,x(T )

)|Fθ]


Proof.

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf„


(T )(Tθ(ω)(ω̃))«

dP(ω̃),

and therefore

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =


”dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃) .


E[f(X ν̄t,x(T )

)|Fθ]


Proof.

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf„


(T )(Tθ(ω)(ω̃))«

dP(ω̃),

and therefore

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =


”dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃) .
Example : Super-solution property

• Want to prove that V∗ is a viscosity super-solution of

infu∈U

(−LuV∗) ≥ 0

• By usingV (t, x) ≥ sup

ν∈UϕtE[ϕ(θν ,X νt,x(θ

ν))]

• Want to prove that V∗ is a viscosity super-solution of

infu∈U

(−LuV∗) ≥ 0

• By usingV (t, x) ≥ sup

ν∈UϕtE[ϕ(θν ,X νt,x(θ

ν))]

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]

≤ E[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.



• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.




}so that


]


]− r4

for some ιn → 0


]− r4/2.


]by the weak DPP.



• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.




}so that


]


]− r4

for some ιn → 0


]− r4/2.


]by the weak DPP.



• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.




}so that


]


]− r4

for some ιn → 0


]− r4/2.


]by the weak DPP.



• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.




}so that


]


]− r4

for some ιn → 0


]− r4/2.


]by the weak DPP.



• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.




}so that


]


]− r4

for some ιn → 0


]− r4/2.


]by the weak DPP.



• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.




}so that


]≤ E

[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0


]− r4/2.


]by the weak DPP.



• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.




}so that


]≤ E


]− r4

for some ιn → 0, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.


]by the weak DPP.



• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.




}so that


]≤ E


]− r4

for some ιn → 0, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.


]by the weak DPP.
Extensions

• Optimal control with running gain

• Optimal stopping

• Mixed optimal control/stopping, impulse control, ...
Extensions



Extensions

Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 ·...

Documents

Transcript of Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 ·...

Kay - Solutions

Kittel and kroemer Solutions

Entropic Hypocoercivity and Hydrodynamic Limits · CEREMADE, Universit e Paris-Dauphine Supported by ERC MALADY Heraklyon, July 21, 2015 S. Olla - CEREMADE Entropic Hypocoercivity.

Solutions Manual - files.book4me.xyzfiles.book4me.xyz/sample/Solution Manual for Introduction to Aircraft Structural... · Solutions Manual Solutions to Chapter 1 Problems S.1.1 The

Solutions 1

Week5HW S15 Solutions - nanohub.orgWeek5HW_S15_Solutions.pdf · HW5)Solutions(continued):) % 3d)% Provideasketchofthesolution,andexplainitinwords. % % Solution:) % % Theconcentrationis

Solutions fox

Solutions - Stanford University

Reactions in aqeous solutions

ISYE3039 HW2 Solutions

FURNITURE SOLUTIONS - LATTAS

Chapter 16: Solutions

Solutions for cell culture growth and analysis Solutions for cell

Solutions to Additional Problems - Department of Engineeringeelinux.ee.usm.maine.edu/courses/ele314/solutions/ch02.pdf · Solutions to Additional Problems 2.32. A discrete-time LTI

Exam%2%Solutions% - University of Florida · PHY2049Spring2013$ $ Exam2$solutions$ Exam%2%Solutions% $ Notethat$thereare$several$variations$of$some$problems,indicatedbychoicesin ...

SOLUTIONS FOR SIEVING

WBJEE 2010 Solutions

Practice Exam2 - Solutions

Chimie Des Solutions Mpsi

Physics exam solutions