of 84 /84
Weak Dynamic Programming for Viscosity Solutions B. Bouchard Ceremade, Univ. Paris-Dauphine, and, Crest, Ensae May 2009 Joint work with Nizar Touzi, CMAP, Ecole Polytechnique
• Author

others
• Category

## Documents

• view

5

0

Embed Size (px)

### Transcript of Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 ·...

• Weak Dynamic Programming for ViscositySolutions

B. Bouchard

Ceremade, Univ. Paris-Dauphine, and, Crest, Ensae

May 2009

Joint work with Nizar Touzi, CMAP, Ecole Polytechnique

• Motivations

• Consider the control problem in standard form

V (t, x) := supν∈U

J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

• To derive the related HJB equation, one uses the DPP

′′V (t, x) = supν∈U

E [V (τ,X ντ )|X νt = x ]′′

• Usually not that easy to prove

a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

• Our aim is to provide a weak version, much easier to prove.

• Motivations

• Consider the control problem in standard form

V (t, x) := supν∈U

J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

• To derive the related HJB equation, one uses the DPP

′′V (t, x) = supν∈U

E [V (τ,X ντ )|X νt = x ]′′

• Usually not that easy to prove

a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

• Our aim is to provide a weak version, much easier to prove.

• Motivations

• Consider the control problem in standard form

V (t, x) := supν∈U

J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

• To derive the related HJB equation, one uses the DPP

′′V (t, x) = supν∈U

E [V (τ,X ντ )|X νt = x ]′′

• Usually not that easy to prove

a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

• Our aim is to provide a weak version, much easier to prove.

• Motivations

• Consider the control problem in standard form

V (t, x) := supν∈U

J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

• To derive the related HJB equation, one uses the DPP

′′V (t, x) = supν∈U

E [V (τ,X ντ )|X νt = x ]′′

• Usually not that easy to prove

a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

• Our aim is to provide a weak version, much easier to prove.

• Motivations

• Consider the control problem in standard form

V (t, x) := supν∈U

J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

• To derive the related HJB equation, one uses the DPP

′′V (t, x) = supν∈U

E [V (τ,X ντ )|X νt = x ]′′

• Usually not that easy to prove

a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

• Our aim is to provide a weak version, much easier to prove.

• Motivations

• Consider the control problem in standard form

V (t, x) := supν∈U

J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

• To derive the related HJB equation, one uses the DPP

′′V (t, x) = supν∈U

E [V (τ,X ντ )|X νt = x ]′′

• Usually not that easy to prove

a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

• Our aim is to provide a weak version, much easier to prove.

• Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :

U0, a collection of Rd -valued progressively measurable processes.

• Controlled process :

(τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0rcll(Rd )

with [0,T ]× Rd ⊂ S ⊂{

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.

• Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :

U0, a collection of Rd -valued progressively measurable processes.

• Controlled process :

(τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0rcll(Rd )

with [0,T ]× Rd ⊂ S ⊂{

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.

• Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :

U0, a collection of Rd -valued progressively measurable processes.

• Controlled process :

(τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0rcll(Rd )

with [0,T ]× Rd ⊂ S ⊂{

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.

• Reward and Value functions

• Reward function

J(t, x ; ν) := E[f(X νt,x(T )

)]defined for controls ν in

U :={ν ∈ U0 : E

[|f (X νt,x(T ))|

]

• Reward and Value functions

• Reward function

J(t, x ; ν) := E[f(X νt,x(T )

)]defined for controls ν in

U :={ν ∈ U0 : E

[|f (X νt,x(T ))|

]

• Reward and Value functions

• Reward function

J(t, x ; ν) := E[f(X νt,x(T )

)]defined for controls ν in

U :={ν ∈ U0 : E

[|f (X νt,x(T ))|

]

• AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

• AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

• AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

• AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

• AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

• AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

• The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X νt,x(τ))

]′′

• Easy inequality : V (t, x) ≤ supν∈Ut E[V (τ,X νt,x(τ))

]

Proof :

V (t, x) = supν∈Ut

E[E[f (X νt,x(T ))|Fτ

]]where for some ν̃ω ∈ Uτ(ω)

E[f (X νt,x(T ))|Fτ

](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))

• The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X νt,x(τ))

]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

[V (τ,X νt,x(τ))

]

Proof :

V (t, x) = supν∈Ut

E[E[f (X νt,x(T ))|Fτ

]]where for some ν̃ω ∈ Uτ(ω)

E[f (X νt,x(T ))|Fτ

](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))

• The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X νt,x(τ))

]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

[V (τ,X νt,x(τ))

]Proof :

V (t, x) = supν∈Ut

E[E[f (X νt,x(T ))|Fτ

]]

where for some ν̃ω ∈ Uτ(ω)

E[f (X νt,x(T ))|Fτ

](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))

• The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X νt,x(τ))

]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

[V (τ,X νt,x(τ))

]Proof :

V (t, x) = supν∈Ut

E[E[f (X νt,x(T ))|Fτ

]]where for some ν̃ω ∈ Uτ(ω)

E[f (X νt,x(T ))|Fτ

](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))

• The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X νt,x(τ))

]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

[V (τ,X νt,x(τ))

]Proof :

V (t, x) = supν∈Ut

E[E[f (X νt,x(T ))|Fτ

]]where for some ν̃ω ∈ Uτ(ω)

E[f (X νt,x(T ))|Fτ

](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

≤ V (τ(ω),X νt,x(τ)(ω))

• The case where J(·; ν) is l.s.c. and V is continuous

• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X νt,x(τ))

]′′• Easy inequality :

V (t, x) ≤ supν∈Ut

E[V (τ,X νt,x(τ))

]

• More difficult one :

V (t, x) ≥ supν∈Ut

E[V (τ,X νt,x(τ))

]

• The case where J(·; ν) is l.s.c. and V is continuous

• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X νt,x(τ))

]′′• Easy inequality :

V (t, x) ≤ supν∈Ut

E[V (τ,X νt,x(τ))

]• More difficult one :

V (t, x) ≥ supν∈Ut

E[V (τ,X νt,x(τ))

]

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

1Ai (τ,Xνt,x(τ))ν

i .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

on (ti − ri , ti ]× Bri (xi )

and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

1Ai (τ,Xνt,x(τ))ν

i .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .

Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

1Ai (τ,Xνt,x(τ))ν

i .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

1Ai (τ,Xνt,x(τ))ν

i .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1

J(τ,X νt,x(τ); νi )1Ai

(τ,X νt,x(τ)

)

≥∑i≥1

(V (τ,X νt,x(τ))− 3ε

)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]

≥ E[V(τ,X νt,x(τ)

)]− 3ε .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1

J(τ,X νt,x(τ); νi )1Ai

(τ,X νt,x(τ)

)≥

∑i≥1

(V (τ,X νt,x(τ))− 3ε

)1Ai(τ,X νt,x(τ)

)

= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]

≥ E[V(τ,X νt,x(τ)

)]− 3ε .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1

J(τ,X νt,x(τ); νi )1Ai

(τ,X νt,x(τ)

)≥

∑i≥1

(V (τ,X νt,x(τ))− 3ε

)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]

≥ E[V(τ,X νt,x(τ)

)]− 3ε .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1

J(τ,X νt,x(τ); νi )1Ai

(τ,X νt,x(τ)

)≥

∑i≥1

(V (τ,X νt,x(τ))− 3ε

)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]

≥ E[V(τ,X νt,x(τ)

)]− 3ε .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1

J(τ,X νt,x(τ); νi )1Ai

(τ,X νt,x(τ)

)≥

∑i≥1

(V (τ,X νt,x(τ))− 3ε

)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]

≥ E[V(τ,X νt,x(τ)

)]− 3ε .

• The case where J(·; ν) is l.s.c. and V is continuous

Proof : Then,

E[f(X ν

ε

t,x(T ))|Fτ]

=∑i≥1

J(τ,X νt,x(τ); νi )1Ai

(τ,X νt,x(τ)

)≥

∑i≥1

(V (τ,X νt,x(τ))− 3ε

)1Ai(τ,X νt,x(τ)

)= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fτ]]

≥ E[V(τ,X νt,x(τ)

)]− 3ε .

• Problems

• The lower-semicontinuity of J(·; ν) is very important in thisproof

: It is in general not difficult to obtain.

• The upper-semicontinuity of V is also very important

: It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

• Problems

• The lower-semicontinuity of J(·; ν) is very important in thisproof : It is in general not difficult to obtain.

• The upper-semicontinuity of V is also very important

: It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

• Problems

• The lower-semicontinuity of J(·; ν) is very important in thisproof : It is in general not difficult to obtain.

• The upper-semicontinuity of V is also very important

: It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

• Problems

• The lower-semicontinuity of J(·; ν) is very important in thisproof : It is in general not difficult to obtain.

• The upper-semicontinuity of V is also very important : It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

• Observation

To derive the PDE in the viscosity sense, try to obtain :

V (t, x) ≥ supν∈Ut

E[V (τ,X νt,x(τ)

]

but one only needs :

V (t, x) ≥ supν∈Ut

E[ϕ(τ,X νt,x(τ)

]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.ϕ being smooth it should be much easier to prove ! !

• Observation

To derive the PDE in the viscosity sense, try to obtain :

V (t, x) ≥ supν∈Ut

E[V (τ,X νt,x(τ)

]but one only needs :

V (t, x) ≥ supν∈Ut

E[ϕ(τ,X νt,x(τ)

]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.

ϕ being smooth it should be much easier to prove ! !

• Observation

To derive the PDE in the viscosity sense, try to obtain :

V (t, x) ≥ supν∈Ut

E[V (τ,X νt,x(τ)

]but one only needs :

V (t, x) ≥ supν∈Ut

E[ϕ(τ,X νt,x(τ)

]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.ϕ being smooth it should be much easier to prove ! !

• The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

lim inf(t′,x ′)→(t,x), t′≤t

J(t ′, x ′; ν) ≥ J(t, x ; ν).

Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.

Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

V (t, x) ≥ supν∈Uϕt

E[ϕ(θν ,X νt,x(θ

ν))],

where Uϕt ={ν ∈ Ut : E

[ϕ(θν ,X νt,x(θ

ν))+]

• The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

lim inf(t′,x ′)→(t,x), t′≤t

J(t ′, x ′; ν) ≥ J(t, x ; ν).

Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

V (t, x) ≥ supν∈Uϕt

E[ϕ(θν ,X νt,x(θ

ν))],

where Uϕt ={ν ∈ Ut : E

[ϕ(θν ,X νt,x(θ

ν))+]

• The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

lim inf(t′,x ′)→(t,x), t′≤t

J(t ′, x ′; ν) ≥ J(t, x ; ν).

Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

V (t, x) ≥ supν∈Uϕt

E[ϕ(θν ,X νt,x(θ

ν))],

where Uϕt ={ν ∈ Ut : E

[ϕ(θν ,X νt,x(θ

ν))+]

• The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

lim inf(t′,x ′)→(t,x), t′≤t

J(t ′, x ′; ν) ≥ J(t, x ; ν).

Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

V (t, x) ≥ supν∈Uϕt

E[ϕ(θν ,X νt,x(θ

ν))],

where Uϕt ={ν ∈ Ut : E

[ϕ(θν ,X νt,x(θ

ν))+]

• The weak DPPProof.

For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

1Ai (θν ,X νt,x(θ

ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1

J(θν ,X νt,x(θν); ν i )1Ai

(θν ,X νt,x(θ

ν))

≥∑i≥1

(ϕ(θν ,X νt,x(θ

ν))− 3ε)1Ai(θν ,X νt,x(θ

ν))

= ϕ(θν ,X νt,x(θν))− 3ε

• The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .

Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

1Ai (θν ,X νt,x(θ

ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1

J(θν ,X νt,x(θν); ν i )1Ai

(θν ,X νt,x(θ

ν))

≥∑i≥1

(ϕ(θν ,X νt,x(θ

ν))− 3ε)1Ai(θν ,X νt,x(θ

ν))

= ϕ(θν ,X νt,x(θν))− 3ε

• The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

1Ai (θν ,X νt,x(θ

ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1

J(θν ,X νt,x(θν); ν i )1Ai

(θν ,X νt,x(θ

ν))

≥∑i≥1

(ϕ(θν ,X νt,x(θ

ν))− 3ε)1Ai(θν ,X νt,x(θ

ν))

= ϕ(θν ,X νt,x(θν))− 3ε

• The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

1Ai (θν ,X νt,x(θ

ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1

J(θν ,X νt,x(θν); ν i )1Ai

(θν ,X νt,x(θ

ν))

≥∑i≥1

(ϕ(θν ,X νt,x(θ

ν))− 3ε)1Ai(θν ,X νt,x(θ

ν))

= ϕ(θν ,X νt,x(θν))− 3ε

• The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

1Ai (θν ,X νt,x(θ

ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1

J(θν ,X νt,x(θν); ν i )1Ai

(θν ,X νt,x(θ

ν))

≥∑i≥1

(ϕ(θν ,X νt,x(θ

ν))− 3ε)1Ai(θν ,X νt,x(θ

ν))

= ϕ(θν ,X νt,x(θν))− 3ε

• The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

1Ai (θν ,X νt,x(θ

ν))ν i .

Then,

E[f(X ν

ε

t,x(T ))|Fνθ]

=∑i≥1

J(θν ,X νt,x(θν); ν i )1Ai

(θν ,X νt,x(θ

ν))

≥∑i≥1

(ϕ(θν ,X νt,x(θ

ν))− 3ε)1Ai(θν ,X νt,x(θ

ν))

= ϕ(θν ,X νt,x(θν))− 3ε

• The weak DPP

Proof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that onAi := (ti − ri , ti ]× Bi , disjoint sets that cover [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

1Ai (θν ,X νt,x(θ

ν))ν i .

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X ν

ε

t,x(T ))|Fνθ]]

≥ E[ϕ(θν ,X νt,x(θ

ν))]− 3ε .

• The weak DPP

Using test functions makes the proof straightforward :

supν∈Uϕt

E[ϕ(θν ,X νt,x(θ

ν))]≤ V (t, x)

≤ supν∈Ut

E[V ∗(θν ,X νt,x(θ

ν))]

Remark : If {X νt,x(θν), ν ∈ Ut} is bounded in L∞, one canapproximate V∗ from below by smooth functions and obtain :

supν∈Ut

E[V∗(θν ,X νt,x(θ

ν))]≤ V (t, x) ≤ sup

ν∈UtE[V ∗(θν ,X νt,x(θ

ν))]

• The weak DPP

Using test functions makes the proof straightforward :

supν∈Uϕt

E[ϕ(θν ,X νt,x(θ

ν))]≤ V (t, x) ≤ sup

ν∈UtE[V ∗(θν ,X νt,x(θ

ν))]

Remark : If {X νt,x(θν), ν ∈ Ut} is bounded in L∞, one canapproximate V∗ from below by smooth functions and obtain :

supν∈Ut

E[V∗(θν ,X νt,x(θ

ν))]≤ V (t, x) ≤ sup

ν∈UtE[V ∗(θν ,X νt,x(θ

ν))]

• The weak DPP

Using test functions makes the proof straightforward :

supν∈Uϕt

E[ϕ(θν ,X νt,x(θ

ν))]≤ V (t, x) ≤ sup

ν∈UtE[V ∗(θν ,X νt,x(θ

ν))]

Remark : If {X νt,x(θν), ν ∈ Ut} is bounded in L∞, one canapproximate V∗ from below by smooth functions and obtain :

supν∈Ut

E[V∗(θν ,X νt,x(θ

ν))]≤ V (t, x) ≤ sup

ν∈UtE[V ∗(θν ,X νt,x(θ

ν))]

• Example : Framework

• Controlled process

dX (r) = µ (X (r), νr ) dr + σ (X (r), νr ) dWr

• U = square integrable progressively measurable processes withvalues in U ⊂ Rd

• f is l.s.c with f − with linear growth, µ and σ Lipschitzcontinuous.

• Example : Framework

• Controlled process

dX (r) = µ (X (r), νr ) dr + σ (X (r), νr ) dWr

• U = square integrable progressively measurable processes withvalues in U ⊂ Rd

• f is l.s.c with f − with linear growth, µ and σ Lipschitzcontinuous.

• Example : Framework

• Controlled process

dX (r) = µ (X (r), νr ) dr + σ (X (r), νr ) dWr

• U = square integrable progressively measurable processes withvalues in U ⊂ Rd

• f is l.s.c with f − with linear growth, µ and σ Lipschitzcontinuous.

• Example : Verification of the assumptions

For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

⇒ lim inf E[f (X νt′,x ′(T ))

]≥ E

[f (X νt,x(T ))

].

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

• Example : Verification of the assumptions

For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

⇒ lim inf E[f (X νt′,x ′(T ))

]≥ E

[f (X νt,x(T ))

].

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

• Example : Verification of the assumptions

For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

⇒ lim inf E[f (X νt′,x ′(T ))

]≥ E

[f (X νt,x(T ))

].

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

• Example : Verification of the assumptions

For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

⇒ lim inf E[f (X νt′,x ′(T ))

]≥ E

[f (X νt,x(T ))

].

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

• Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

Proof. Canonical space : W (ω) = ω. Set Ts(ω) := (ωr − ωs)r≥s andωs := (ωr∧s)r≥0.

Eˆf`X νt,x(T )

´|Fθ˜(ω) =

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω))«

dP(Tθ(ω)(ω)) .

=

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω̃))«

dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃ω)

where, ν̃ω(ω̃) := ν(ωθ(ω) + Tθ(ω)(ω̃)) ∈ Uθ(ω).

• Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

Proof. Canonical space : W (ω) = ω. Set Ts(ω) := (ωr − ωs)r≥s andωs := (ωr∧s)r≥0.

Eˆf`X νt,x(T )

´|Fθ˜(ω) =

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω))«

dP(Tθ(ω)(ω)) .

=

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω̃))«

dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃ω)

where, ν̃ω(ω̃) := ν(ωθ(ω) + Tθ(ω)(ω̃)) ∈ Uθ(ω).

• Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

E[f(X νt,x(T )

)|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

Proof. Canonical space : W (ω) = ω. Set Ts(ω) := (ωr − ωs)r≥s andωs := (ωr∧s)r≥0.

Eˆf`X νt,x(T )

´|Fθ˜(ω) =

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω))«

dP(Tθ(ω)(ω)) .

=

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω̃))«

dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃ω)

where, ν̃ω(ω̃) := ν(ωθ(ω) + Tθ(ω)(ω̃)) ∈ Uθ(ω).

• Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

Proof.

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf„

Xν̃(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω̃))«

dP(ω̃),

and therefore

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf“X ν̃(Ts (ω̃))θ(ω),Xνt,x (θ)(ω)(T )(Tθ(ω)(ω̃))

”dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃) .

• Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

Proof.

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf„

Xν̃(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω̃))«

dP(ω̃),

and therefore

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf“X ν̃(Ts (ω̃))θ(ω),Xνt,x (θ)(ω)(T )(Tθ(ω)(ω̃))

”dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃) .

• Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

E[f(X ν̄t,x(T )

)|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

Proof.

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf„

Xν̃(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

(T )(Tθ(ω)(ω̃))«

dP(ω̃),

and therefore

Eˆf`X ν̄t,x(T )

´|Fθ˜(ω) =

Zf“X ν̃(Ts (ω̃))θ(ω),Xνt,x (θ)(ω)(T )(Tθ(ω)(ω̃))

”dP(ω̃)

= J(θ(ω),X νt,x(θ)(ω); ν̃) .

• Example : Super-solution property

• Want to prove that V∗ is a viscosity super-solution of

infu∈U

(−LuV∗) ≥ 0

• By usingV (t, x) ≥ sup

ν∈UϕtE[ϕ(θν ,X νt,x(θ

ν))]

• Example : Super-solution property

• Want to prove that V∗ is a viscosity super-solution of

infu∈U

(−LuV∗) ≥ 0

• By usingV (t, x) ≥ sup

ν∈UϕtE[ϕ(θν ,X νt,x(θ

ν))]

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]

≤ E[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]

≤ E[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]

≤ E[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]

≤ E[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]

≤ E[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]

≤ E[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]≤ E

[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]≤ E

[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Example : Super-solution property

• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

}so that

V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

]≤ E

[ϕ(θn,X utn,xn(θn))

]− r4

for some ιn → 0, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

]by the weak DPP.

• Extensions

• Optimal control with running gain

• Optimal stopping

• Mixed optimal control/stopping, impulse control, ...

• Extensions

• Optimal control with running gain

• Optimal stopping

• Mixed optimal control/stopping, impulse control, ...

• Extensions

• Optimal control with running gain

• Optimal stopping

• Mixed optimal control/stopping, impulse control, ...