Numerical Optimization; Dynamic Programming; Precautionary Saving

Christopher D. Carroll

English

Research Papers in Economics

The Method of Endogenous Gridpoints
for Solving Dynamic Stochastic Optimization Problems
Christopher D. Carroll
ccarroll@jhu.edu
July 28, 2005
Abstract
This paper introduces a method for solving numerical dynamic stochastic optimization
problems that avoids rootﬁnding operations. The idea is applicable to many microe-
conomic and macroeconomic problems, including life cycle, buﬀer-stock, and stochastic
growth problems. Software is provided.
Keywords: Dynamic optimization, precautionary saving, stochastic growth model, en-
dogenous gridpoints, liquidity constraints
JEL: C6, D9, E2
I am grateful to Nicola Fuchs-Sch¨ undeln, Wouter den Haan, Michael Haliassos, Ken Judd,
Albert Marcet, Dimitri Mavridis, Victor Rios-Rull, John Rust, Eric Young, and partic-
ipants in the 2002 meetings of the Society for Computational Economics for discussions
of the literature and existing solution techniques. Excellent research assistance has been
provided by Zhou Lu and Marc Chan. I am especially grateful to Michael Reiter for
improvements to the Matlab code. All errors are my own.
Department of Economics, The Johns Hopkins University, Baltimore MD
21218-2685; http://econ.jhu.edu/people/ccarroll. Software archive at
http://econ.jhu.edu/people/ccarroll/EndogenousArchive.zip.1 The Problem
Consider a consumer whose goal is to maximize discounted utility from consumption
max
T  
s=t
β
t−su(Cs) (1)
for a CRRA utility function u(C) = C1−ρ/(1 − ρ).1
The consumer’s problem will be specialized below to two cases: A standard microe-
conomic problem with uninsurable idiosyncratic shocks to labor income, and a standard
representative agent problem with shocks to aggregate productivity (the ‘micro’ and the
‘macro’ models).2
The consumer’s initial condition is deﬁned by two state variables: Mt is ‘market
resources’ (macro interpretation: capital plus current output) or ‘cash-on-hand’ (micro
interpretation: net worth plus current income), while Pt is permanent labor productivity
in both interpretations.
The transition process for Mt is broken up, for convenience of analysis, into three
steps. Assets at the end of the period are market resources minus consumption, equal
to
At = Mt − Ct, (2)
and capital at the beginning of the next period is what remains after a depreciation factor
k is applied,
Kt+1 = Atk, (3)
where k = (1 − δ) in the usual macro notation and k = 1 in the micro interpretation.
The ﬁnal step can be thought of as the transition from the beginning of period t + 1,
when capital Kt+1 but has not yet been used to produce output, and the middle of that
period, when output has been produced and incorporated into resources:
Mt+1 =
≡Lt+1       
et+1Θt+1Pt+1 Wt+1 + Kt+1Rt+1 (4)
where Wt+1 is the wage rate; Θt+1 is an iid transitory shock (e.g., unemployment) nor-
malized to satisfy Et[Θt+n] = 1 ∀ n > 0 (usually Θt = 1 ∀ t in the macro interpretation);
and et indicates labor eﬀort (or labor supply), which for purposes of this paper is ﬁxed at
et = 1, but in general could be allowed to vary. The disarticulation of the ﬂow of income
into labor and capital components is useful in thinking separately about the eﬀects of
productivity growth (captured by ΘP) and capital accumulation (K).
1Putting leisure in the utility function is straightforward but would distract from the paper’s point.
2Diﬀerent aspects of the setup of the problem will strike micro and macroeconomists as peculiar; with
patience, it should become clear how the problem as speciﬁed can be transformed into more familiar
forms.Permanent labor productivity (in either interpretation) evolves according to
Pt+1 = Gt+1PtΨt+1 (5)
for a permanent shock that satisﬁes Et[Ψt+n] = 1 ∀ n > 0 and Gt is exogenous and
perfectly predictable (see below for varying interpretations of G).
Deﬁning lower case variables as the upper-case variable scaled by the level of perma-
nent labor productivity, e.g. at = At/Pt, we have
at = mt − ct (6)
while with a bit of algebra the state transition becomes
mt+1 = etΘt+1       
≡lt+1
Wt+1 + (atk/Gt+1Ψt+1)
      
=kt+1
Rt+1. (7)
The interest and wage factors are assumed not to depend on anything other than
capital and productive labor input; together with the iid assumption about the struc-
ture of the shocks, this implies that the problem has a Bellman equation representation
(henceforth boldface indicates functions)
Vt(Mt,Pt) = max
Ct
{u(Ct) + βEt[Vt+1(Mt+1,Pt+1)]} (8)
subject to the transition equations.
Deﬁning Λt+1 ≡ Gt+1Ψt+1, consider the related problem
vt(mt) = max
ct

 
 
u(ct) + βEt


Λ
1−ρ
t+1vt+1(
=mt+1       
Wt+1lt+1 + Rt+1 atk/Λt+1       
kt+1
)




 
 
. (9)
Assume that there is some last period T in which
VT(MT,PT) = P
1−ρ
T vT(MT/PT) (10)
for some well-behaved vT (we will be more speciﬁc about the terminal value function
below). In this case it is easy to show that the solution to the ‘normalized’ problem
deﬁned by (9) yields the solution to the original problem via Vt = P
1−ρ
t vt for any t < T.3
Now deﬁne an end-of-period value function ‘Gothic v’ as
vt(at) = βEt[Λ
1−ρ
t+1vt+1(Wt+1lt+1 + k Rt+1at /Λt+1)] (11)
with derivative
v
a
t(at) = βEt
 
Λ
1−ρ
t+1v
m
t+1(Wt+1lt+1 + Rt+1atk/Λt+1)Rt+1k/Λt+1
 
(12)
= kβEt
 
Λ
−ρ
t+1v
m
t+1(Wt+1lt+1 + Rt+1atk/Λt+1)Rt+1
 
3See Carroll (2004) for a proof.
2and (11) and (6) imply that (9) can be rewritten using vt as
vt(mt) = max
{at}
{u(mt − at) + vt(at)}, (13)
and the envelope theorem can be applied
v
m
t (mt) = u
′(ct) (14)
while the ﬁrst order condition yields the Euler equation
u
′(ct) = v
a
t(at) (15)
= kβEt[u
′(Λt+1ct+1)Rt+1].
2 Recursion
Generically, problems like this can be solved by specifying a ﬁnal-period decision rule cT
and a procedure for recursion (obtaining ct from ct+1). Here we specify the recursion;
below we specify choices for the terminal decision rule.
2.1 A Standard Solution Method
The absence of a closed-form solution means that optimal decision functions (e.g. the
consumption function) must be constructed by calculating their values at a ﬁnite grid
of possible values of the state variables. Call some ordered set of such values  i ∈     ≡
{ 1, 2,..., I}.
With ct+1 in hand, the usual solution procedure is to specify a     and, for each element
 i, to use a numerical rootﬁnding routine to ﬁnd the χi that satisﬁes (15),
u
′(χi) = v
a
t( i − χi). (16)
The points { i,χi} are then used to construct an interpolating approximation to ct.
(Choice of interpolation method is separable from the point of this paper; see Judd (1998)
for a discussion of choices). Given the interpolated ct function the solution for earlier
periods is found by recursion.
One of the most computationally burdensome steps in this approach is the numer-
ical solution of (16) for each speciﬁed state gridpoint. Even if eﬃcient methods are
used for constructing the expectations (cf. the parameterized expectations method of
den Haan and Marcet (1990)) and shrewd choices are made for the points to include in
   , for each gridpoint a numerical rootﬁnding operation still must evaluate a substantial
number of candidate values for the control variable before ﬁnding values that satisfy (16)
to an acceptable degree of precision.
32.2 Endogenous Gridpoints Solution Method
This paper’s key contribution is to introduce an alternative approach that does not require
numerical rootﬁnding. The trick is to begin with end-of-period assets at and to use
the end-of-period marginal value function va
t, the ﬁrst order condition, and the budget
constraint to construct the unique values of middle-of-period mt generated by those at
values.
Speciﬁcally, deﬁne an exogenous, time-invariant ordered set of values of at collected
in αi ∈   α ≡ {α1,α2,...,αI}. For each end-of-period state αi the marginal value va
t(αi)
is easy to calculate; inverting the consumption ﬁrst order condition, the α’s generate
χi = u
′−1(v
a
t(αi)). (17)
Note that the budget constraint implies that
 i = αi + χi. (18)
We now have a collection of { i,χi} pairs in hand and can interpolate as before to
generate an approximation to ct. This completes the recursion.
The key distinction between this approach and the standard one is that the gridpoints
for the policy functions are not predetermined; instead they are endogenously generated
from a predetermined grid of values of end-of-period assets (hence the method’s name).
One reason the method is eﬃcient is that expectations are never computed for any grid-
point not used in the ﬁnal interpolating function; the standard method may compute
expectations for many unused gridpoints.
3 Macro Specialization
We ﬁrst specialize to a macroeconomic stochastic growth model. Assuming aggregate
production is Cobb-Douglas in capital and labor F(K,P) = KεP 1−ε, after normalizing
by productivity P (and assuming a constant value G for the labor productivity growth
factor), under the usual assumptions of perfect competition etc. if there is no aggregate
transitory shock (Θt+1 = 1) we have
Rt+1 = 1 + εk
ε−1
t+1 (19)
Wt+1 = (1 − ε)k
ε
t+1 (20)
and market resources are the sum of capital and production,
mt+1 = kt+1Rt+1 + Wt+1 (21)
= kt+1 + k
ε
t+1. (22)
We specify the terminal consumption function as
cT(m) = m, (23)
4which is very far from the converged inﬁnite horizon consumption rule, but easy to verify
as satisfying the assumption (10) imposed earlier. More eﬃcient choices are available,
but for our purposes simplicity trumps eﬃciency.
An arbitrary speciﬁcation of the process for permanent productivity shocks is a
three point distribution deﬁned by   Ψ = {0.9,1.0,1.1} with probabilities Pr(  Ψ) =
{0.25,0.50,0.25}.4
The top panel of ﬁgure 1 plots the converged consumption function that emerges
from this solution method for the benchmark set of parameter values speciﬁed in Table 1,
along with the consumption function for the standard perfect foresight version of the
model (  Ψ = Pr(  Ψ) = {1}).
4 Micro Specialization
In the microeconomic literature, the usual approach is to take aggregate interest and
wage rates as exogenous, and to focus on transitory (Θ) and permanent (Ψ) shocks to
idiosyncratic labor productivity. We again start the recursion with cT(m) = m, and the
permanent shocks are retained exactly as speciﬁed for the macro problem.5
4.1 Life Cycle Models
Life cycle models specify a stereotypical pattern of lifetime income growth deﬁned by
Gt where t is age rather than time and T is the maximum possible lifespan;6 mortality
uncertainty can be accommodated by age-varying values of β.
4.2 Buﬀer Stock Models
If R,W,G and β are constant, k = 1, and the impatience condition
RβE[(GΨ)
−ρ] < 1 (24)
holds, Deaton (1991) and Carroll (2004) show that the problem deﬁnes a contraction
mapping so that the consumption functions deﬁned by the problem converge from any
well-behaved initial starting function cT(m); the converged function is deﬁned as
c(m) = lim
n→∞
cT−n(m). (25)
We solve for the converged consumption function for two versions.
4With careful choice of points and weights, small-dimensional discrete representations like this do a
good job of approximating commonly-used continuous distributions like a lognormal, cf. Judd (1998).
An empirically realistic choice would have a much lower variance than the speciﬁcation here.
5An empirically realistic calibration for micro data would exhibit a permanent variance perhaps 100
times greater than an appropriate macro calibration; but appropriate calibration is not the point of this
paper.
6This is the context in which the assumption that cT(m) = m actually makes economic sense, as
distinct from merely providing a convenient starting point for recursion.
54.2.1 Version With Unemployment
Assume that in future periods there is a small probability ℘ that income will be zero
(corresponding to a substantial spell of unemployment):
Θt+1 =
 
0 with probability ℘ > 0
Ξt+1/(1 − ℘) with probability (1 − ℘)
(26)
where   Ξ = {0.9,1.0,1.1} and Pr(  Ξ|Θ > 0) = {0.25,0.50,0.25} (the same structure of
non-unemployment transitory shocks as for the permanent shocks).
Carroll (2004) shows that in this model,
lim
mt→0
ct(mt) = 0. (27)
This implies that the minimum value in   α should be α1 = 0, which will generate { 1,χ1} =
{0.,0.} as the ﬁrst point in the set of interpolating points. The resulting converged c(m)
is shown as the thin solid locus in the bottom panel of ﬁgure 1; see the software for details
of how the remaining values in   α were chosen.
4.2.2 Version With Liquidity Constraints
Microeconomic models often include a liquidity constraint in addition to the usual tran-
sition equations, and capturing the constraint often induces much additional code.
Dealing with a liquidity constraint using the method of endogenous gridpoints is
simple. The key observation is that when the constraint is on the cusp of binding, the
marginal value of consumption is equal to the marginal value of saving exactly zero
(assuming the constraint is of the form that requires a to be nonnegative; generalization
to more elaborate kinds of constraints is straightforward). If the ﬁrst value in the ordered
set   α is α1 = 0, then the method will produce
χ1 =  1 = u
′−1(v
a
t(0)), (28)
and if we deﬁne ˆ ct(m) as the function produced by interpolation among the points gen-
erated by   α, the consumption function imposing the constraint will be
ct(m) = min(m,ˆ ct(m)). (29)
If the consumption function is deﬁned as a piecewise linear spline interpolation among
the { ,χ} points, the constraint can be handled simply by adding the point { 0,χ0} =
{0,0} to the set of points that constitute the interpolation data.
The converged solution is shown as the bold locus in the bottom panel of ﬁgure 1.
5 Conclusion
The method of endogenous gridpoints can be extended to problems with multiple state
variables and multiple controls, e.g. a micro consumer with a portfolio choice problem, or
a labor supply decision; or a macro consumer with a utility function that exhibits habit
formation (see Carroll (2000) for examples). The method is useful both because it is
simpler than the standard method and because it reduces computational demands.
6References
Carroll, Christopher D., 2000, Lecture notes on solving microeconomic dynamic stochas-
tic optimization problems, Manuscript, Johns Hopkins University. Available at
http://econ.jhu.edu/people/ccarroll/solvingmicrodsops.pdf.
Carroll, Christopher D., 2004, Theoretical foundations of
buﬀer stock saving, NBER Working Paper Number 10867.
http://econ.jhu.edu/people/ccarroll/BufferStockProofsNew.pdf.
Deaton, Angus S., 1991, Saving and liquidity constraints, Econometrica 59, 1221–1248.
denHaan, Wouter J, and Albert Marcet, 1990, Solving the stochastic growth model by
parameterizing expectations, Journal of Business and Economic Statistics 8(1), 31–34.
available at http://ideas.repec.org/a/bes/jnlbes/v8y1990i1p31-34.html.
Judd, Kenneth L., 1998, Numerical Methods in Economics (The MIT Press, Cambridge,
Massachusetts).
7Table 1: Parameter Values
Parameters Common to All Models
ρ 2 Relative Risk Aversion
β 0.96 Annual Discount Factor
ℓ 1 Labor supply (ﬁxed)
  Ψ {0.90,1.00,1.10} Permanent Shock Realizations
Pr(  Ψ) {0.25,0.50,0.25} Permanent Shock Probabilities
Macro Model Parameters
k 0.90 Depreciation Factor
G 1.01 Exogenous Aggregate Productivity Growth Factor
ε 0.36 Capital Share in Production
Micro Model Parameters
k 1 Depreciation Factor
G 1.03 Trend Individual Wage Growth Factor
R 1.04 Real Interest Rate
W 1.00 Wage Rate
  Ξ {0.90,1.00,1.10} Transitory Shock Realizations for Employed
Pr(  Ξ|Θ > 0) {0.25,0.50,0.25} Transitory Shock Probabilities for Employed
Parameter Unique to Unemployment Model
℘ 0.005 Probability of Unemployment Spell
8Figure 1: Macro and Micro Consumption Functions
1. 2. 3. 4. 5.
m
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
c
Perfect Foresight Rep Agent ™
˜ With Perm Shocks
1. 2. 3. 4. 5.
m
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
c
˜ Unemp Micro cHmL
Liq Constr Micro cHmL ™
c=m ™
9Appendices: Mathematica Code
This appendix contains the core code used to generate the micro and macro model solu-
tions graphed in the ﬁgures. Common.nb contains the parameters and code that are shared
for both micro and macro solutions; Micro.nb and Macro.nb contain the speciﬁc param-
eterizations and specializations for the respective speciﬁc problems. The commands to
execute the solutions and graph them are not of general interest and are not included,
but are part of a downloadable package available on the author’s website. Downloadable
MATLAB code is also available on the author’s webpage; Michael Haliassos and Dimitri
Mavrides have written MATLAB and C++ code that solves a closely related problem; con-
tact Haliassos for more information.
Common.nb
uP c_     If c > 0,  then   cˆ   Ρ,   else      
nP z_     zˆ    1/Ρ  
vP at_     ¾ Β Sum 
 tp1    Vec   Loop   
 tp1   G  tp1 
 tp1    Vec   Loop   
ktp1   ¾ at/ tp1 
ltp1    tp1 eEffort 
mtp1   If MacroModel && ktp1    0,0,ktp1 R ktp1    ltp1 W ktp1   
 VecProb   Loop    VecProb   Loop   R ktp1  uP  tp1 Last cInterpFunc  mtp1  
,  Loop,Length  Vec  
,  Loop,Length  Vec  
      End Sum   
cInterpFunc    Interpolation   0.,0. , 1000.,1000.  ,InterpolationOrder  > 1   
SolveAnotherPeriod    Block   ,
AppendTo cInterpFunc,
Interpolation 
Union 
Chop 
Prepend 
Table 
Α   ΑVec  ΑLoop   
Χ   nP vP Α   
Μ   Α   Χ 
 Μ,Χ 
, ΑLoop,Length ΑVec   
, 0.,0.      Prepending 0,0  handles potentialliquidity constraint  
     Chop cuts off numericallyinsignificantdigits  
     Union removes duplicateentries   
,InterpolationOrder  > 1     Piecewise linear interpolation  
      End of AppendTo   
      End of SolveAnotherPeriod  
 Β,Ρ,n,eEffort,PeriodsToAdd     0.96,2,20,1,99  
10Micro.nb
 ¾,G,p     1,1.03,0.005  
    G 
MacroModel   False 
<< Common.nb 
   Triple exponentialgrowth to a   10 picks a good set of values for Α   
ΑVec   Table Exp Exp Exp ΑLoop    1    1    1 ”//N,
 ΑLoop,0,Log Log Log 10   1    1    1 ,Log Log Log 10   1    1    1 / n   1    
 Vec    Vec    0.9,1.,1.1  
 VecProb    VecProb    0.25,0.5,0.25  
 Vec   Prepend  Vec /  1   p ,0.  
 VecProb   Prepend  VecProb  1   p ,p   
R k_     1.04 
W k_     1. 
Macro.nb
 ¾,G,¶     0.9,1.01,0.36  
MacroModel   True 
<< Common.nb 
   PF SS k    kSS       GˆΡ / Β¾     1 /¶ ˆ 1/ ¶  1   
   PF SS a    aSS   kSS G/¾ 
ΑVec   Table Exp ΑLoop    1, ΑLoop,0,Log 3 aSS , Log 3 aSS / n   1    
 Vec    0.9,1.,1.1  
 VecProb    0.25,0.5,0.25  
 Vec    1.  
 VecProb    1.  
R k_     If k > 0,  then  1   ¶ kˆ ¶   1 ,  else     
W k_      1   ¶ kˆ¶  
11

Solving Dynamic Stochastic Optimization Problems Using the Method of Endogenous Gridpoints

http://www.econ.jhu.edu/people/ccarroll/EndogenousGridpoints.pdf

Solving Dynamic Stochastic Optimization Problems Using the Method of Endogenous Gridpoints

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics