Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a...

31
Mathematical Economics Discrete time: calculus of variations problem Paulo Brito 1 [email protected] University of Lisbon November 23, 2018

Transcript of Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a...

Page 1: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Mathematical EconomicsDiscrete time: calculus of variations problem

Paulo Brito

[email protected] of Lisbon

November 23, 2018

Page 2: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Discrete time: intuitionAssume we have a cake of initial size W0 = ϕ and we want to eatit completely by time t = T, then WT = 0The size of the cake evolution is:

Wt+1 = Wt − Ct ⇒ Wt = W0 −t−1∑s=0

Cs

 By imposing the two constraints W0 = ϕ and WT = 0 we have

T−1∑s=0

Ct = ϕ

 There is an infinite  number of paths {C0, . . .CT−1} verifyingthis conditionExample: if we chose Ct = C̄ constant for all t we get C̄ = ϕ

T .Is this optimal ? Depends on the value functional

Page 3: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Discrete time dynamic optimization: Intuition

A dynamic model is defined over sequences x (or (x, u)) inwhich there is some form of intertemporal interaction:actions today have an impact in the future;Types of time interactions:- intratemporal: iinteraction within one period- intertemporal: interaction across periodsAdmissible sequences: the set X contains a large number(possibly infinite) of admissible paths verifying a givenintratemporal relation  together with some informationregarding initial and/or terminal data ;Optimal sequences: an intertemporal optimalitycriterium  allows for choosing the best admissible sequence.

Page 4: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

The discrete time calculus of variations problem (CV)

Calculus of variations problem: find x = {xt}Tt=0 ∈ X , i.e., a path

belonging to the set of admissible paths X , that maximises anintertemporal objective function (or value functional)

J(x) =T−1∑t=0

F(xt+1, xt, t), x ∈ X

  where F(t, xt, xt+1) specifies an intratemporal relation for the valueof the action upon the state variable, within period t (i.e., betweentimes t and t + 1).

Page 5: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

time

period

state

flow

0

x0

1

x1

2

x2

t

xt

t+1

xt+1

T−2

xT−2

T−1

xT−1

T

xT

0

F0

1

F1

t

Ft

T−2

FT−2

T−1

FT−1

The value for changing the state variable in period t  is

Ft = F(xt+1, xt, t).

  The value of the a given sequence of actions across T − 1periods  is

J(x) = F(0, x0, x1) + . . .+ F(t, xt, xt+1) + . . .+ F(T − 1, xT−1, xT)

  The optimal sequence  x∗ has value

J∗ = J(x∗) = maxx

{J(x) : x ∈ X}

 

Page 6: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

The discrete time optimal control problem (OC)

Optimal control problem: find the pair(x, u) = ({xt}T

t=0, {ut}T−1t=0 ) ∈ D = X × U where D is defined by

the sequence of intratemporal relations for periods 0 to T − 1

xt+1 = G(xt, ut, t), t = 0, 1, . . . , t, . . .T − 1

plus other conditions, that maximises an intertemporal objectivefunctional

J(x, u) =T−1∑t=0

F(t, xt, ut)

 Observation: J(x) has terminal time T − 1 because if we knowthe optimal x∗T−1 and u∗

T−1, we know the optimal x∗T, becausex∗T = G(x∗T−1, u∗

T−1,T − 1).

Page 7: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

time

period

state

controlvalue flow

0

x0

1

x1

2

x2

t

xt

t+1

xt+1

T−2

xT−2

T−1

xT−1

T

xT

0

u0

1

u1

t

ut

T−2

uT−2

T−1

uT−1

0

F0

1

F1

t

Ft

T−2

FT−2

T−1

FT−1

The value associated to choosing the control ut, given the statext,  throughout period t  is

Ft = F(xt, ut, t).  Choosing the control ut, generates a the value for the state variableat the end of period t, xt+1 = g(xt, ut, t)The value of a given sequence of controls across T − 1periods  is

J(u, x) = F(0, x0, u0) + . . .+ F(t, xt, ut) + . . .+ F(T − 1, xT−1, uT−1)

  The optimal sequence  u∗ has valueJ∗ = J(u∗) = max

u{J(x, u) : (x, u) ∈ D}

 

Page 8: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Calculus of variations problems

We consider the following problems:Simplest problem: X = {T, x0, and xT given}Free terminal state problem: X = {x0, and T given}, xT freeConstrained terminal state problem:X = {x0, and T, and h(xT) ≥ 0 given}Discounted infinite horizon problems: T = ∞ andlimt→∞ xt free or constrained

Page 9: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Calculus of variations: simplest problemThe problem : Find x∗ = {x∗t }T

t=0  that maximizes

J(x) =T−1∑t=0

F(xt+1, xt, t), s. t. x0 = ϕ0, xT = ϕT (1)

where T, ϕ0 and ϕT are known.First order necessary conditions

Propositionif x∗ ≡ {x∗0, x∗1, . . . , x∗T} is a solution of problem (1) , it verifies theEuler-Lagrange equation and the admissibility conditions

∂F∂xt

(x∗t , x∗t−1, t − 1) + ∂F∂xt

(x∗t+1, x∗t , t) = 0, t = 1, 2, . . . ,T − 1x∗0 = ϕ0, t = 0x∗T = ϕT, t = T

Proof .

Page 10: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Application: cake eating problemThe problem

maxC

{ T∑t=0

βt ln(Ct) : Wt+1 = Wt − Ct, W0 = ϕ, WT = 0}

 Assumptions: 0 < β < 1, T ≥ 1, ϕ > 0Intuition:

at time t = 0, we have a cake of initial size ϕ, and we want toconsume it completely until time T (known)by choosing a sequence of bites such that:(1) we value independently each bite (the value functional is asum);(2) the instantaneous pleasure of each bite is increasing with thesize of the bite, but at a decreasing rate (the utility functionu(Ct) is concave);(3) we are impatient: when we plan to eat the cake we value morethe immediate bites rather than future bites ( there is a discountfactor βt which decreases with time).How should we eat the cake ?

Page 11: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Application: cake eating as a CV problem

We can transform the previous problem into a calculus ofvariations problem

maxW

{ T∑t=0

βt ln(Wt − Wt+1) : W0 = ϕ, WT = 0}

 

Page 12: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Application: cake eating problem - solutionThe optimal cake size is W∗ = {ϕ, . . . ,W∗

t , . . . ,W∗T−1, 0} where

W∗t =

(βt − βT

1 − βT

)ϕ, t = 0, 1, . . .T

 

It slightly bends down from a linear path because of time discounting,as 0 < β < 1 Proof .

Page 13: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Calculus of variations: free terminal state problemFind x∗ = {x∗t , }T

t=0  that maximizes

J(x) =T−1∑t=0

F(xt+1, xt, t) s.t. x0 = ϕ0 (2)

where ϕ0 and T is given and xT is free.First order necessary conditions:

Propositionif x∗ is a solution of problem (2) , it verifies the Euler-Lagrangeequation the admissibility condition and the transversality condition

∂F∂xt

(x∗t , x∗t−1, t − 1) + ∂F∂xt

(x∗t+1, x∗t , t) = 0, t = 1, 2, . . . ,T − 1

x∗0 = ϕ0, t = 0∂F∂xT

(x∗T, x∗T−1,T − 1) = 0, t = T

Proof .

Page 14: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Cake eating problem: free terminal state

Problem

max{C}

T∑t=0

βt ln(Ct), subject to Wt+1 = Wt − Ct, W0 = ϕ, WTfree

 The problem is ill-posed: there is no solution with economicmeaning Proof .Reason: the transversality condition only holds if C0 = ∞this is intuitive: if we had no restriction on the terminal size ofthe case (or if could borrow freely) we would overeat.We have to redefine the problem to obtain a reasonable solution.

Page 15: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Calculus of variations: constrained terminal stateFind x∗ = {x∗t }T

t=0  that maximizes

J(x) =T−1∑t=0

F(xt+1, xt, t) s.t. x0 = ϕ0, xT ≥ ϕT (3)

where T, ϕ0 and ϕT are given.First order necessary conditions:

Propositionif x∗ is a solution of problem (3), it verifies

∂F∂xt

(x∗t , x∗t−1, t − 1) + ∂F∂xt

(x∗t+1, x∗t , t) = 0, t = 1, 2, . . . ,T − 1

x∗0 = ϕ0, t = 0∂F∂xT

(x∗T, x∗T−1,T − 1) · (ϕT − x∗T) = 0, t = T

 Proof .

Page 16: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Cake eating problem: constrained terminal state

Problem

max{C}

T∑t=0

βt ln(Ct), subject to Wt+1 = Wt − Ct, W0 = ϕ, WT ≥ 0

F.o.c W∗

t+2 = (1 + β)W∗t+1 − βW∗

t , t = 0, 1, . . . ,T − 2W∗

0 = ϕ, t = 0βT−1

W∗T − W∗

T−1W∗

T = 0, t = T

 has the same formal solution as the simplest problem: W∗

T = 0 isdetermined endogenously not by assumption. Proof .

Page 17: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Calculus of variations: discounted infinite horizon 1

The problem: find the infinite sequence x∗ = {x∗t }∞t=0

maxx

∞∑t=0

βtf(xt+1, xt), s.t. x0 = ϕ0 (4)

where, 0 < β < 1, and ϕ0 are given and the terminal state is free(i.e., limt→∞ xt is free).The first order conditions are:

∂f∂xt

(x∗t , x∗t−1) + β∂f∂xt

(xt+1, xt) = 0, t = 0, 1, . . .

x∗0 = x0, t = 0limt→∞ βt−1 ∂f(x∗

t ,x∗t−1)

∂xt= 0, t → ∞

 

Page 18: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Calculus of variations: discounted infinite horizon 2

The problem: find the infinite sequence x∗ = {x∗t }∞t=0

maxx

∞∑t=0

βtf(xt+1, xt), s.t. x0 = ϕ0, limt→∞

xt ≥ 0 (5)

given β and ϕ0

F.o.c. ∂f∂xt

(x∗t , x∗t−1) + β∂f∂xt

(xt+1, xt) = 0, t = 0, 1, . . .

x∗0 = x0,

limt→∞ βt−1 ∂f∂xt

(x∗t , x∗t−1) · x∗t = 0,

 

Page 19: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Cake eating problem: infinite horizonProve that if the terminal state is free there is no finite solutionProve that with the terminal condition limt→∞ Wt ≥ 0 thesolution is generated by

W∗t = ϕ0β

t, t = 0, 1, . . . ,∞

 

Page 20: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Terminal conditions

Problem Given Optimality contitionsT xT T∗ x∗T

(CV1) fixed fixed T xT

(CV2) fixed free T∂F(x∗T, x∗T−1,T − 1)

∂xT= 0

(CV3) fixed xT ≥ ϕT T∂F(x∗T, x∗T−1,T − 1)

∂xT(ϕT − x∗T) = 0

(CV4) ∞ free ∞ limt→∞ βt−1 ∂F(x∗t , x∗t−1)

∂xt= 0

(CV5) ∞ x∞ ≥ 0 ∞ limt→∞ βt−1 ∂F(x∗t , x∗t−1)

∂xtx∗t = 0

Page 21: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Proofs

Page 22: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Proof of proposition 1Let x∗ = {x∗t }T

t=0 be an optimal path. Then the optimal value is

J(x∗) =T−1∑t=0

F(x∗t+1, x∗t , t)

  Introduce an admissible perturbation xt = x∗t + εt where ε0 = εT = 0, andεt ̸= 0 for any t ∈ {1, . . . ,T − 1}. The value for the perturbed path is

J(x) =T−1∑t=0

F(x∗t+1 + εt+1, x∗t + εt, t)

  Using a Taylor expansion we get first variation in the value functional

J(x)− J(x∗) = ∂F(x∗1, x∗0, 0)∂x0

ε0 +

T−1∑t=1

(∂F(x∗t , x∗t−1, t − 1)

∂xt+

∂F(x∗t+1, x∗t , t)∂xt

)εt+

+∂F(x∗T, x∗T−1,T − 1)

∂xTεT

=

T−1∑t=1

(∂F(x∗t , x∗t−1, t − 1)

∂xt+

∂F(x∗t+1, x∗t , t)∂xt

)εt

  The path x is optimal if J(x)− J(x∗) = 0. Then the EL is obtained.Return .

Page 23: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Solution of the cake eating problem 1The Euler-Lagrange equation

Observe that in this problem the utility function, evaluated at theoptimum, is

F∗t = F(W∗

t+1,W∗t , t) = βt ln(W∗

t − W∗t+1)

  and the Euler-Lagrange condition is∂F∗

t−1∂Wt

+∂F∗

t∂Wt

= 0

 applying to our utility function we have

∂Wt

(βt−1 ln(W∗

t−1 − W∗t ))

+∂

∂Wt

(βt ln(W∗

t − W∗t+1)

)=

= − βt−1

W∗t−1 − W∗

t+

βt

W∗t − W∗

t+1= 0 (6)

 is a linear second-order difference equation

W∗t+1 − W∗

t − β(W∗t − W∗

t−1), t = −1, . . . ,T − 3

Page 24: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Solution of cake eating problem 1

If W∗ is a solution of the problem then it verifies the first orderconditions

W∗t+2 = (1 + β)W∗

t+1 − βW∗t , t = 0, . . .T − 2

W∗0 = ϕ

W∗T = 0

 To find the solution either we transform to a planar system or are ableto transform to a simpler problem (we follow the second strategy)

Page 25: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Solution of cake eating problem 1Solving the problem: recursive approach

Because Wt+1 − Wt = −Ct, the EL equation is equivalent to

Ct+1 = βCt

 This is a first-order DE which has solution

Ct = C0βt, where C0 unknown

 Therefore

Wt+1 = Wt − C0βt, for t = 0, 1, . . . ,T − 1

 This is a unit-root equation with solution

Wt = ϕ− C0

t−1∑s=0

βs = ϕ− C0

(1 − βt

1 − β

)  because W0 = ϕ.

Page 26: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Solution of cake eating problem 1

However, to be optimal, Wt should be admissible, which meanssatisfying {

W∗0 = ϕ

W∗T = 0

  But Wt|t=0 = ϕ = ϕ

Wt|t=T = ϕ− C0

(1 − βT

1 − β

Setting Wt|t=T = 0 yields C∗0 = ϕ

(1 − β

1 − βT

)Then the solution to the cake eating problem is generated by

W∗t = ϕ

(1 − (1 − β)

(1 − βT)

(1 − βt)

(1 − β)

)= ϕ

(βt − βT

1 − βT

Return .

Page 27: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Proof of proposition 2

Using the same method of the proof of Proposition 1 we have

J(x)− J(x∗) = ∂F(x∗1, x∗0, 0)∂x0

ε0 +

T−1∑t=1

(∂F(x∗t , x∗t−1, t − 1)

∂xt+

∂F(x∗t+1, x∗t , t)∂xt

)εt+

+∂F(x∗T, x∗T−1,T − 1)

∂xTεT

=

T−1∑t=1

(∂F(x∗t , x∗t−1, t − 1)

∂xt+

∂F(x∗t+1, x∗t , t)∂xt

)εt+

+∂F(x∗T, x∗T−1,T − 1)

∂xTεT

  because the admissibility condition only implies that ε0 = 0. Return .

Page 28: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Cake eating problem: free terminal stateThe first-order conditions are

W∗t+2 = (1 + β)W∗

t+1 − βW∗t , t = 0, 1, . . . ,T − 1

W0 = ϕ, t = 0βT−1

W∗T−W∗

T−1= 0. t = T

Using the same transformation as in the fixed-terminal state problemwe have {

Ct+1 = βCt, t = 0, 1, . . . ,T − 1βT−1

CT−1= 0. t = T

But we already know that the solution to the first equation isCt = C0β

t. This implies that the transversality constraint becomes

βT−1

CT−1=

βT−1

C0βT−1 =1

C0

  which can only be zero if C0 = ∞. This means that WT → −∞which does not make sense. The problem is ill-posed.

Return .

Page 29: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Proof of proposition 3

Using the same method of the proof of Proposition 3 but introducing theterminal constraint

J(x) =T−1∑t=0

F(x∗t+1 + εt+1, x∗t + εt, t) + λ(ϕT − x∗T − εT)

  where λ is a Lagrange multiplier. Then the variation of the value of theproblem is

J(x)− J(x∗) =T−1∑t=1

(∂F(x∗t , x∗t−1, t − 1)

∂xt+

∂F(x∗t+1, x∗t , t)∂xt

)εt+

+∂F(x∗T, x∗T−1,T − 1)

∂xTεT + λ(ϕT − x∗T − εT)

 

Page 30: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Proof of proposition 3

Then J(x) = J(x∗) if the EL equation holds,(∂F(x∗T, x∗T−1,T − 1)

∂xT− λ

)εT = 0

  where εT is arbitrary, and the Karush-Kuhn-Tucker conditions hold

λ(ϕT − x∗t ) = 0, λ ≥ 0, x∗t ≥ ϕT.

  Therefore ∂F∗T−1

∂xT− λ = 0 but

∂F∗T−1

∂xT− λ =

(∂F∗

T−1∂xT

− λ

)(ϕT − x∗t ) =

∂F∗T−1

∂xT(ϕT − x∗t ) = 0

  which is the transversality condition. Return .

Page 31: Mathematical Economics Discrete time: calculus of ... · Discrete time: intuition Assume we have a cake of initial size W0 = ϕ and we want to eat it completely by time t = T, then

Solution to the cake eating problem with constrainedterminal state

Now the transversality condition is

−WTβT−1

WT − WT−1= 0

  Using our previous transformation we have

−WTβT−1

WT − WT−1= −WT

βT−1

CT−1= −WT

βT−1

βT−1C0= −WT

C0= 0

  only if WT = 0 for any finite and positive C0. Return .