Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 ·...

of 84 /84
Weak Dynamic Programming for Viscosity Solutions B. Bouchard Ceremade, Univ. Paris-Dauphine, and, Crest, Ensae May 2009 Joint work with Nizar Touzi, CMAP, Ecole Polytechnique

Embed Size (px)

Transcript of Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 ·...

  • Weak Dynamic Programming for ViscositySolutions

    B. Bouchard

    Ceremade, Univ. Paris-Dauphine, and, Crest, Ensae

    May 2009

    Joint work with Nizar Touzi, CMAP, Ecole Polytechnique

  • Motivations

    • Consider the control problem in standard form

    V (t, x) := supν∈U

    J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

    • To derive the related HJB equation, one uses the DPP

    ′′V (t, x) = supν∈U

    E [V (τ,X ντ )|X νt = x ]′′

    • Usually not that easy to prove

    a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

    b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

    • Our aim is to provide a weak version, much easier to prove.

  • Motivations

    • Consider the control problem in standard form

    V (t, x) := supν∈U

    J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

    • To derive the related HJB equation, one uses the DPP

    ′′V (t, x) = supν∈U

    E [V (τ,X ντ )|X νt = x ]′′

    • Usually not that easy to prove

    a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

    b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

    • Our aim is to provide a weak version, much easier to prove.

  • Motivations

    • Consider the control problem in standard form

    V (t, x) := supν∈U

    J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

    • To derive the related HJB equation, one uses the DPP

    ′′V (t, x) = supν∈U

    E [V (τ,X ντ )|X νt = x ]′′

    • Usually not that easy to prove

    a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

    b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

    • Our aim is to provide a weak version, much easier to prove.

  • Motivations

    • Consider the control problem in standard form

    V (t, x) := supν∈U

    J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

    • To derive the related HJB equation, one uses the DPP

    ′′V (t, x) = supν∈U

    E [V (τ,X ντ )|X νt = x ]′′

    • Usually not that easy to prove

    a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

    b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

    • Our aim is to provide a weak version, much easier to prove.

  • Motivations

    • Consider the control problem in standard form

    V (t, x) := supν∈U

    J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

    • To derive the related HJB equation, one uses the DPP

    ′′V (t, x) = supν∈U

    E [V (τ,X ντ )|X νt = x ]′′

    • Usually not that easy to prove

    a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

    b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

    • Our aim is to provide a weak version, much easier to prove.

  • Motivations

    • Consider the control problem in standard form

    V (t, x) := supν∈U

    J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X νt = x ]

    • To derive the related HJB equation, one uses the DPP

    ′′V (t, x) = supν∈U

    E [V (τ,X ντ )|X νt = x ]′′

    • Usually not that easy to prove

    a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

    b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

    • Our aim is to provide a weak version, much easier to prove.

  • Framework

    • (Ω,F ,F := (Ft)t≤T ,P), T > 0.

    • Controls :

    U0, a collection of Rd -valued progressively measurable processes.

    • Controlled process :

    (τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0rcll(Rd )

    with [0,T ]× Rd ⊂ S ⊂{

    (τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.

  • Framework

    • (Ω,F ,F := (Ft)t≤T ,P), T > 0.

    • Controls :

    U0, a collection of Rd -valued progressively measurable processes.

    • Controlled process :

    (τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0rcll(Rd )

    with [0,T ]× Rd ⊂ S ⊂{

    (τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.

  • Framework

    • (Ω,F ,F := (Ft)t≤T ,P), T > 0.

    • Controls :

    U0, a collection of Rd -valued progressively measurable processes.

    • Controlled process :

    (τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0rcll(Rd )

    with [0,T ]× Rd ⊂ S ⊂{

    (τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )}.

  • Reward and Value functions

    • Reward function

    J(t, x ; ν) := E[f(X νt,x(T )

    )]defined for controls ν in

    U :={ν ∈ U0 : E

    [|f (X νt,x(T ))|

    ]

  • Reward and Value functions

    • Reward function

    J(t, x ; ν) := E[f(X νt,x(T )

    )]defined for controls ν in

    U :={ν ∈ U0 : E

    [|f (X νt,x(T ))|

    ]

  • Reward and Value functions

    • Reward function

    J(t, x ; ν) := E[f(X νt,x(T )

    )]defined for controls ν in

    U :={ν ∈ U0 : E

    [|f (X νt,x(T ))|

    ]

  • AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

    A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

  • AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

    A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

  • AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

    A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

  • AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

    A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

  • AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

    A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

  • AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

    A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

  • The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

    ′′V (t, x) = supν∈Ut

    E[V (τ,X νt,x(τ))

    ]′′

    • Easy inequality : V (t, x) ≤ supν∈Ut E[V (τ,X νt,x(τ))

    ]

    Proof :

    V (t, x) = supν∈Ut

    E[E[f (X νt,x(T ))|Fτ

    ]]where for some ν̃ω ∈ Uτ(ω)

    E[f (X νt,x(T ))|Fτ

    ](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

    ≤ V (τ(ω),X νt,x(τ)(ω))

  • The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

    ′′V (t, x) = supν∈Ut

    E[V (τ,X νt,x(τ))

    ]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

    [V (τ,X νt,x(τ))

    ]

    Proof :

    V (t, x) = supν∈Ut

    E[E[f (X νt,x(T ))|Fτ

    ]]where for some ν̃ω ∈ Uτ(ω)

    E[f (X νt,x(T ))|Fτ

    ](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

    ≤ V (τ(ω),X νt,x(τ)(ω))

  • The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

    ′′V (t, x) = supν∈Ut

    E[V (τ,X νt,x(τ))

    ]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

    [V (τ,X νt,x(τ))

    ]Proof :

    V (t, x) = supν∈Ut

    E[E[f (X νt,x(T ))|Fτ

    ]]

    where for some ν̃ω ∈ Uτ(ω)

    E[f (X νt,x(T ))|Fτ

    ](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

    ≤ V (τ(ω),X νt,x(τ)(ω))

  • The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

    ′′V (t, x) = supν∈Ut

    E[V (τ,X νt,x(τ))

    ]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

    [V (τ,X νt,x(τ))

    ]Proof :

    V (t, x) = supν∈Ut

    E[E[f (X νt,x(T ))|Fτ

    ]]where for some ν̃ω ∈ Uτ(ω)

    E[f (X νt,x(T ))|Fτ

    ](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

    ≤ V (τ(ω),X νt,x(τ)(ω))

  • The case where J(·; ν) is l.s.c. and V is continuous• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

    ′′V (t, x) = supν∈Ut

    E[V (τ,X νt,x(τ))

    ]′′• Easy inequality : V (t, x) ≤ supν∈Ut E

    [V (τ,X νt,x(τ))

    ]Proof :

    V (t, x) = supν∈Ut

    E[E[f (X νt,x(T ))|Fτ

    ]]where for some ν̃ω ∈ Uτ(ω)

    E[f (X νt,x(T ))|Fτ

    ](ω) ≤ J(τ(ω),X νt,x(τ)(ω); ν̃ω)

    ≤ V (τ(ω),X νt,x(τ)(ω))

  • The case where J(·; ν) is l.s.c. and V is continuous

    • Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

    ′′V (t, x) = supν∈Ut

    E[V (τ,X νt,x(τ))

    ]′′• Easy inequality :

    V (t, x) ≤ supν∈Ut

    E[V (τ,X νt,x(τ))

    ]

    • More difficult one :

    V (t, x) ≥ supν∈Ut

    E[V (τ,X νt,x(τ))

    ]

  • The case where J(·; ν) is l.s.c. and V is continuous

    • Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

    ′′V (t, x) = supν∈Ut

    E[V (τ,X νt,x(τ))

    ]′′• Easy inequality :

    V (t, x) ≤ supν∈Ut

    E[V (τ,X νt,x(τ))

    ]• More difficult one :

    V (t, x) ≥ supν∈Ut

    E[V (τ,X νt,x(τ))

    ]

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

    J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

    on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

    1Ai (τ,Xνt,x(τ))ν

    i .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

    J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

    on (ti − ri , ti ]× Bri (xi )

    and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

    1Ai (τ,Xνt,x(τ))ν

    i .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

    J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

    on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .

    Given ν ∈ Ut , define

    νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

    1Ai (τ,Xνt,x(τ))ν

    i .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

    J(t, x ; ν i ) + ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

    on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,τ ]ν + 1(τ,T ]∑i≥1

    1Ai (τ,Xνt,x(τ))ν

    i .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Then,

    E[f(X ν

    ε

    t,x(T ))|Fτ]

    =∑i≥1

    J(τ,X νt,x(τ); νi )1Ai

    (τ,X νt,x(τ)

    )

    ≥∑i≥1

    (V (τ,X νt,x(τ))− 3ε

    )1Ai(τ,X νt,x(τ)

    )= V (τ,X νt,x(τ))− 3ε

    and

    V (t, x) ≥ J(t, x ; νε)

    = E[E[f(X ν

    ε

    t,x(T ))|Fτ]]

    ≥ E[V(τ,X νt,x(τ)

    )]− 3ε .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Then,

    E[f(X ν

    ε

    t,x(T ))|Fτ]

    =∑i≥1

    J(τ,X νt,x(τ); νi )1Ai

    (τ,X νt,x(τ)

    )≥

    ∑i≥1

    (V (τ,X νt,x(τ))− 3ε

    )1Ai(τ,X νt,x(τ)

    )

    = V (τ,X νt,x(τ))− 3ε

    and

    V (t, x) ≥ J(t, x ; νε)

    = E[E[f(X ν

    ε

    t,x(T ))|Fτ]]

    ≥ E[V(τ,X νt,x(τ)

    )]− 3ε .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Then,

    E[f(X ν

    ε

    t,x(T ))|Fτ]

    =∑i≥1

    J(τ,X νt,x(τ); νi )1Ai

    (τ,X νt,x(τ)

    )≥

    ∑i≥1

    (V (τ,X νt,x(τ))− 3ε

    )1Ai(τ,X νt,x(τ)

    )= V (τ,X νt,x(τ))− 3ε

    and

    V (t, x) ≥ J(t, x ; νε)

    = E[E[f(X ν

    ε

    t,x(T ))|Fτ]]

    ≥ E[V(τ,X νt,x(τ)

    )]− 3ε .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Then,

    E[f(X ν

    ε

    t,x(T ))|Fτ]

    =∑i≥1

    J(τ,X νt,x(τ); νi )1Ai

    (τ,X νt,x(τ)

    )≥

    ∑i≥1

    (V (τ,X νt,x(τ))− 3ε

    )1Ai(τ,X νt,x(τ)

    )= V (τ,X νt,x(τ))− 3ε

    and

    V (t, x) ≥ J(t, x ; νε)

    = E[E[f(X ν

    ε

    t,x(T ))|Fτ]]

    ≥ E[V(τ,X νt,x(τ)

    )]− 3ε .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Then,

    E[f(X ν

    ε

    t,x(T ))|Fτ]

    =∑i≥1

    J(τ,X νt,x(τ); νi )1Ai

    (τ,X νt,x(τ)

    )≥

    ∑i≥1

    (V (τ,X νt,x(τ))− 3ε

    )1Ai(τ,X νt,x(τ)

    )= V (τ,X νt,x(τ))− 3ε

    and

    V (t, x) ≥ J(t, x ; νε)

    = E[E[f(X ν

    ε

    t,x(T ))|Fτ]]

    ≥ E[V(τ,X νt,x(τ)

    )]− 3ε .

  • The case where J(·; ν) is l.s.c. and V is continuous

    Proof : Then,

    E[f(X ν

    ε

    t,x(T ))|Fτ]

    =∑i≥1

    J(τ,X νt,x(τ); νi )1Ai

    (τ,X νt,x(τ)

    )≥

    ∑i≥1

    (V (τ,X νt,x(τ))− 3ε

    )1Ai(τ,X νt,x(τ)

    )= V (τ,X νt,x(τ))− 3ε

    and

    V (t, x) ≥ J(t, x ; νε)

    = E[E[f(X ν

    ε

    t,x(T ))|Fτ]]

    ≥ E[V(τ,X νt,x(τ)

    )]− 3ε .

  • Problems

    • The lower-semicontinuity of J(·; ν) is very important in thisproof

    : It is in general not difficult to obtain.

    • The upper-semicontinuity of V is also very important

    : It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

  • Problems

    • The lower-semicontinuity of J(·; ν) is very important in thisproof : It is in general not difficult to obtain.

    • The upper-semicontinuity of V is also very important

    : It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

  • Problems

    • The lower-semicontinuity of J(·; ν) is very important in thisproof : It is in general not difficult to obtain.

    • The upper-semicontinuity of V is also very important

    : It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

  • Problems

    • The lower-semicontinuity of J(·; ν) is very important in thisproof : It is in general not difficult to obtain.

    • The upper-semicontinuity of V is also very important : It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

  • Observation

    To derive the PDE in the viscosity sense, try to obtain :

    V (t, x) ≥ supν∈Ut

    E[V (τ,X νt,x(τ)

    ]

    but one only needs :

    V (t, x) ≥ supν∈Ut

    E[ϕ(τ,X νt,x(τ)

    ]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.ϕ being smooth it should be much easier to prove ! !

  • Observation

    To derive the PDE in the viscosity sense, try to obtain :

    V (t, x) ≥ supν∈Ut

    E[V (τ,X νt,x(τ)

    ]but one only needs :

    V (t, x) ≥ supν∈Ut

    E[ϕ(τ,X νt,x(τ)

    ]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.

    ϕ being smooth it should be much easier to prove ! !

  • Observation

    To derive the PDE in the viscosity sense, try to obtain :

    V (t, x) ≥ supν∈Ut

    E[V (τ,X νt,x(τ)

    ]but one only needs :

    V (t, x) ≥ supν∈Ut

    E[ϕ(τ,X νt,x(τ)

    ]for all smooth function such that (t, x) achieves a minimum ofV − ϕ.ϕ being smooth it should be much easier to prove ! !

  • The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

    lim inf(t′,x ′)→(t,x), t′≤t

    J(t ′, x ′; ν) ≥ J(t, x ; ν).

    Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.

    Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

    V (t, x) ≥ supν∈Uϕt

    E[ϕ(θν ,X νt,x(θ

    ν))],

    where Uϕt ={ν ∈ Ut : E

    [ϕ(θν ,X νt,x(θ

    ν))+]

  • The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

    lim inf(t′,x ′)→(t,x), t′≤t

    J(t ′, x ′; ν) ≥ J(t, x ; ν).

    Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

    V (t, x) ≥ supν∈Uϕt

    E[ϕ(θν ,X νt,x(θ

    ν))],

    where Uϕt ={ν ∈ Ut : E

    [ϕ(θν ,X νt,x(θ

    ν))+]

  • The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

    lim inf(t′,x ′)→(t,x), t′≤t

    J(t ′, x ′; ν) ≥ J(t, x ; ν).

    Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

    V (t, x) ≥ supν∈Uϕt

    E[ϕ(θν ,X νt,x(θ

    ν))],

    where Uϕt ={ν ∈ Ut : E

    [ϕ(θν ,X νt,x(θ

    ν))+]

  • The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

    lim inf(t′,x ′)→(t,x), t′≤t

    J(t ′, x ′; ν) ≥ J(t, x ; ν).

    Theorem : Fix {θν , ν ∈ Ut} ⊂ T t[t,T ] a family of stopping times.Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

    V (t, x) ≥ supν∈Uϕt

    E[ϕ(θν ,X νt,x(θ

    ν))],

    where Uϕt ={ν ∈ Ut : E

    [ϕ(θν ,X νt,x(θ

    ν))+]

  • The weak DPPProof.

    For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

    J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

    on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

    1Ai (θν ,X νt,x(θ

    ν))ν i .

    Then,

    E[f(X ν

    ε

    t,x(T ))|Fνθ]

    =∑i≥1

    J(θν ,X νt,x(θν); ν i )1Ai

    (θν ,X νt,x(θ

    ν))

    ≥∑i≥1

    (ϕ(θν ,X νt,x(θ

    ν))− 3ε)1Ai(θν ,X νt,x(θ

    ν))

    = ϕ(θν ,X νt,x(θν))− 3ε

  • The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

    J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

    on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .

    Given ν ∈ Ut , define

    νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

    1Ai (θν ,X νt,x(θ

    ν))ν i .

    Then,

    E[f(X ν

    ε

    t,x(T ))|Fνθ]

    =∑i≥1

    J(θν ,X νt,x(θν); ν i )1Ai

    (θν ,X νt,x(θ

    ν))

    ≥∑i≥1

    (ϕ(θν ,X νt,x(θ

    ν))− 3ε)1Ai(θν ,X νt,x(θ

    ν))

    = ϕ(θν ,X νt,x(θν))− 3ε

  • The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

    J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

    on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

    1Ai (θν ,X νt,x(θ

    ν))ν i .

    Then,

    E[f(X ν

    ε

    t,x(T ))|Fνθ]

    =∑i≥1

    J(θν ,X νt,x(θν); ν i )1Ai

    (θν ,X νt,x(θ

    ν))

    ≥∑i≥1

    (ϕ(θν ,X νt,x(θ

    ν))− 3ε)1Ai(θν ,X νt,x(θ

    ν))

    = ϕ(θν ,X νt,x(θν))− 3ε

  • The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

    J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

    on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

    1Ai (θν ,X νt,x(θ

    ν))ν i .

    Then,

    E[f(X ν

    ε

    t,x(T ))|Fνθ]

    =∑i≥1

    J(θν ,X νt,x(θν); ν i )1Ai

    (θν ,X νt,x(θ

    ν))

    ≥∑i≥1

    (ϕ(θν ,X νt,x(θ

    ν))− 3ε)1Ai(θν ,X νt,x(θ

    ν))

    = ϕ(θν ,X νt,x(θν))− 3ε

  • The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

    J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

    on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

    1Ai (θν ,X νt,x(θ

    ν))ν i .

    Then,

    E[f(X ν

    ε

    t,x(T ))|Fνθ]

    =∑i≥1

    J(θν ,X νt,x(θν); ν i )1Ai

    (θν ,X νt,x(θ

    ν))

    ≥∑i≥1

    (ϕ(θν ,X νt,x(θ

    ν))− 3ε)1Ai(θν ,X νt,x(θ

    ν))

    = ϕ(θν ,X νt,x(θν))− 3ε

  • The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

    J(t, x ; ν i )+ε ≥ J(ti , xi ; ν i ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

    on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

    1Ai (θν ,X νt,x(θ

    ν))ν i .

    Then,

    E[f(X ν

    ε

    t,x(T ))|Fνθ]

    =∑i≥1

    J(θν ,X νt,x(θν); ν i )1Ai

    (θν ,X νt,x(θ

    ν))

    ≥∑i≥1

    (ϕ(θν ,X νt,x(θ

    ν))− 3ε)1Ai(θν ,X νt,x(θ

    ν))

    = ϕ(θν ,X νt,x(θν))− 3ε

  • The weak DPP

    Proof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that onAi := (ti − ri , ti ]× Bi , disjoint sets that cover [t,T ]× Rd .Given ν ∈ Ut , define

    νε := 1[t,θν ]ν + 1(θν ,T ]∑i≥1

    1Ai (θν ,X νt,x(θ

    ν))ν i .

    and

    V (t, x) ≥ J(t, x ; νε)

    = E[E[f(X ν

    ε

    t,x(T ))|Fνθ]]

    ≥ E[ϕ(θν ,X νt,x(θ

    ν))]− 3ε .

  • The weak DPP

    Using test functions makes the proof straightforward :

    supν∈Uϕt

    E[ϕ(θν ,X νt,x(θ

    ν))]≤ V (t, x)

    ≤ supν∈Ut

    E[V ∗(θν ,X νt,x(θ

    ν))]

    Remark : If {X νt,x(θν), ν ∈ Ut} is bounded in L∞, one canapproximate V∗ from below by smooth functions and obtain :

    supν∈Ut

    E[V∗(θν ,X νt,x(θ

    ν))]≤ V (t, x) ≤ sup

    ν∈UtE[V ∗(θν ,X νt,x(θ

    ν))]

  • The weak DPP

    Using test functions makes the proof straightforward :

    supν∈Uϕt

    E[ϕ(θν ,X νt,x(θ

    ν))]≤ V (t, x) ≤ sup

    ν∈UtE[V ∗(θν ,X νt,x(θ

    ν))]

    Remark : If {X νt,x(θν), ν ∈ Ut} is bounded in L∞, one canapproximate V∗ from below by smooth functions and obtain :

    supν∈Ut

    E[V∗(θν ,X νt,x(θ

    ν))]≤ V (t, x) ≤ sup

    ν∈UtE[V ∗(θν ,X νt,x(θ

    ν))]

  • The weak DPP

    Using test functions makes the proof straightforward :

    supν∈Uϕt

    E[ϕ(θν ,X νt,x(θ

    ν))]≤ V (t, x) ≤ sup

    ν∈UtE[V ∗(θν ,X νt,x(θ

    ν))]

    Remark : If {X νt,x(θν), ν ∈ Ut} is bounded in L∞, one canapproximate V∗ from below by smooth functions and obtain :

    supν∈Ut

    E[V∗(θν ,X νt,x(θ

    ν))]≤ V (t, x) ≤ sup

    ν∈UtE[V ∗(θν ,X νt,x(θ

    ν))]

  • Example : Framework

    • Controlled process

    dX (r) = µ (X (r), νr ) dr + σ (X (r), νr ) dWr

    • U = square integrable progressively measurable processes withvalues in U ⊂ Rd

    • f is l.s.c with f − with linear growth, µ and σ Lipschitzcontinuous.

  • Example : Framework

    • Controlled process

    dX (r) = µ (X (r), νr ) dr + σ (X (r), νr ) dWr

    • U = square integrable progressively measurable processes withvalues in U ⊂ Rd

    • f is l.s.c with f − with linear growth, µ and σ Lipschitzcontinuous.

  • Example : Framework

    • Controlled process

    dX (r) = µ (X (r), νr ) dr + σ (X (r), νr ) dWr

    • U = square integrable progressively measurable processes withvalues in U ⊂ Rd

    • f is l.s.c with f − with linear growth, µ and σ Lipschitzcontinuous.

  • Example : Verification of the assumptions

    For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

    ⇒ lim inf E[f (X νt′,x ′(T ))

    ]≥ E

    [f (X νt,x(T ))

    ].

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

  • Example : Verification of the assumptions

    For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

    ⇒ lim inf E[f (X νt′,x ′(T ))

    ]≥ E

    [f (X νt,x(T ))

    ].

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

  • Example : Verification of the assumptions

    For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

    ⇒ lim inf E[f (X νt′,x ′(T ))

    ]≥ E

    [f (X νt,x(T ))

    ].

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

  • Example : Verification of the assumptions

    For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

    L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X νt,x(T ) in L2

    ⇒ lim inf E[f (X νt′,x ′(T ))

    ]≥ E

    [f (X νt,x(T ))

    ].

    A1 (Independence) The process X νt,x is independent of Ft .

    A2 (Causality) ∀ ν̃ ∈ Ut : ν = ν̃ on A ⊂ F ⇒ X νt,x = X ν̃t,x on A.

    A3 (Stability under concatenation) ∀ ν̃ ∈ Ut , θ ∈ T t[t,T ] :ν1[0,θ] + ν̃1(θ,T ] ∈ Ut .

  • Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    Proof. Canonical space : W (ω) = ω. Set Ts(ω) := (ωr − ωs)r≥s andωs := (ωr∧s)r≥0.

    Eˆf`X νt,x(T )

    ´|Fθ˜(ω) =

    Zf„

    Xν(ωθ(ω)+Tθ(ω)(ω))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω))«

    dP(Tθ(ω)(ω)) .

    =

    Zf„

    Xν(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω̃))«

    dP(ω̃)

    = J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    where, ν̃ω(ω̃) := ν(ωθ(ω) + Tθ(ω)(ω̃)) ∈ Uθ(ω).

  • Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    Proof. Canonical space : W (ω) = ω. Set Ts(ω) := (ωr − ωs)r≥s andωs := (ωr∧s)r≥0.

    Eˆf`X νt,x(T )

    ´|Fθ˜(ω) =

    Zf„

    Xν(ωθ(ω)+Tθ(ω)(ω))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω))«

    dP(Tθ(ω)(ω)) .

    =

    Zf„

    Xν(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω̃))«

    dP(ω̃)

    = J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    where, ν̃ω(ω̃) := ν(ωθ(ω) + Tθ(ω)(ω̃)) ∈ Uθ(ω).

  • Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    a. For P-a.e ω ∈ Ω, ∃ ν̃ω ∈ Uθ(ω) s.t.

    E[f(X νt,x(T )

    )|Fθ]

    (ω) ≤ J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    Proof. Canonical space : W (ω) = ω. Set Ts(ω) := (ωr − ωs)r≥s andωs := (ωr∧s)r≥0.

    Eˆf`X νt,x(T )

    ´|Fθ˜(ω) =

    Zf„

    Xν(ωθ(ω)+Tθ(ω)(ω))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω))«

    dP(Tθ(ω)(ω)) .

    =

    Zf„

    Xν(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω̃))«

    dP(ω̃)

    = J(θ(ω),X νt,x(θ)(ω); ν̃ω)

    where, ν̃ω(ω̃) := ν(ωθ(ω) + Tθ(ω)(ω̃)) ∈ Uθ(ω).

  • Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

    Proof.

    Eˆf`X ν̄t,x(T )

    ´|Fθ˜(ω) =

    Zf„

    Xν̃(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω̃))«

    dP(ω̃),

    and therefore

    Eˆf`X ν̄t,x(T )

    ´|Fθ˜(ω) =

    Zf“X ν̃(Ts (ω̃))θ(ω),Xνt,x (θ)(ω)(T )(Tθ(ω)(ω̃))

    ”dP(ω̃)

    = J(θ(ω),X νt,x(θ)(ω); ν̃) .

  • Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

    Proof.

    Eˆf`X ν̄t,x(T )

    ´|Fθ˜(ω) =

    Zf„

    Xν̃(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω̃))«

    dP(ω̃),

    and therefore

    Eˆf`X ν̄t,x(T )

    ´|Fθ˜(ω) =

    Zf“X ν̃(Ts (ω̃))θ(ω),Xνt,x (θ)(ω)(T )(Tθ(ω)(ω̃))

    ”dP(ω̃)

    = J(θ(ω),X νt,x(θ)(ω); ν̃) .

  • Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

    b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν̃ ∈ Us , and ν̄ := ν1[0,θ] + ν̃1(θ,T ] :

    E[f(X ν̄t,x(T )

    )|Fθ]

    (ω) = J(θ(ω),X νt,x(θ)(ω); ν̃) for P− a.e. ω ∈ Ω.

    Proof.

    Eˆf`X ν̄t,x(T )

    ´|Fθ˜(ω) =

    Zf„

    Xν̃(ωθ(ω)+Tθ(ω)(ω̃))θ(ω),Xνt,x (θ)(ω)

    (T )(Tθ(ω)(ω̃))«

    dP(ω̃),

    and therefore

    Eˆf`X ν̄t,x(T )

    ´|Fθ˜(ω) =

    Zf“X ν̃(Ts (ω̃))θ(ω),Xνt,x (θ)(ω)(T )(Tθ(ω)(ω̃))

    ”dP(ω̃)

    = J(θ(ω),X νt,x(θ)(ω); ν̃) .

  • Example : Super-solution property

    • Want to prove that V∗ is a viscosity super-solution of

    infu∈U

    (−LuV∗) ≥ 0

    • By usingV (t, x) ≥ sup

    ν∈UϕtE[ϕ(θν ,X νt,x(θ

    ν))]

  • Example : Super-solution property

    • Want to prove that V∗ is a viscosity super-solution of

    infu∈U

    (−LuV∗) ≥ 0

    • By usingV (t, x) ≥ sup

    ν∈UϕtE[ϕ(θν ,X νt,x(θ

    ν))]

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]

    ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0

    , i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]

    ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0

    , i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]

    ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0

    , i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]

    ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0

    , i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]

    ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0

    , i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]

    ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0

    , i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]≤ E

    [ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0

    , i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]≤ E

    [ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Example : Super-solution property

    • (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

    • Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

    • Set ϕ̃(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

    • Then, −Luϕ̃(t, x) ≤ 0 on Br := {|t − t0| ≤ r , |x − x0| ≤ r}.

    • Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

    • Set θn := inf{s ≥ tn : (s,X utn,xn(s)) /∈ Br

    }so that

    V (tn, xn) + ιn = ϕ̃(tn, xn) ≤ E[ϕ̃(θn,X utn,xn(θn))

    ]≤ E

    [ϕ(θn,X utn,xn(θn))

    ]− r4

    for some ιn → 0, i.e. V (tn, xn) ≤ E[ϕ(θn,X utn,xn(θn))

    ]− r4/2.

    • While V (tn, xn) ≥ E[ϕ(θn,X utn,xn(θn))

    ]by the weak DPP.

  • Extensions

    • Optimal control with running gain

    • Optimal stopping

    • Mixed optimal control/stopping, impulse control, ...

  • Extensions

    • Optimal control with running gain

    • Optimal stopping

    • Mixed optimal control/stopping, impulse control, ...

  • Extensions

    • Optimal control with running gain

    • Optimal stopping

    • Mixed optimal control/stopping, impulse control, ...