"Itô's Lemma" and the Bellman equation: An applied view

Rare and randomly occurring events are important features of the economic world. In continuous time they can easily be modeled by Poisson processes. Analyzing optimal behavior in such a setup requires the appropriate version of the change of variables formula and the Hamilton-Jacobi-Bellman equation. This paper provides examples for the application of both tools in economic modeling. It accompanies the proofs in Sennewald (2005), who shows, under milder conditions than before, that the Hamilton-Jacobi-Bellman equation is both a necessary and sufficient criterion for optimality. The main example here consists of a consumption-investment problem with labor income. It is shown how the Hamilton-Jacobi-Bellman equation can be used to derive both a Keynes-Ramsey rule and a closed form solution. We also provide a new result. --Stochastic differential equation,Poisson process,Bellman equation,Portfolio optimization,Consump

"Ito's Lemma" and the Bellman equation for Poisson processes: An applied view

Rare and randomly occurring events are important features of the economic world. In continuous time they can easily be modeled by Poisson processes. Analyzing optimal behavior in such a setup requires the appropriate version of the change of variables formula and the Hamilton-Jacobi-Bellman equation. This paper provides examples for the application of both tools in economic modeling. It accompanies the proofs in Sennewald (2005), who shows, under milder conditions than before, that the Hamilton-Jacobi-Bellman equation is both a necessary and sufficient criterion for optimality. The main example here consists of a consumption-investment problem with labor income. It is shown how the Hamilton-Jacobi-Bellman equation can be used to derive both a Keynes-Ramsey rule and a closed form solution. We also provide a new result. --Stochastic differential equation,Poisson process,Bellman equation,Portfolio optimization,Consumption optimization

On the Fragility of the Basis on the Hamilton-Jacobi-Bellman Equation in Economic Dynamics

In this paper, we provide an example of the optimal growth model in which there exists infinitely many solutions to the Hamilton-Jacobi-Bellman equation but the value function does not satisfy this equation. We consider the cause of this phenomenon, and find that the lack of a solution to the original problem is crucial. We show that under several conditions, there exists a solution to the original problem if and only if the value function solves the Hamilton-Jacobi-Bellman equation. Moreover, in this case, the value function is the unique nondecreasing concave solution to the Hamilton-Jacobi-Bellman equation. We also show that without our conditions, this uniqueness result does not hold

“Itô’s Lemma“ and the Bellman Equation for Poisson Processes: An Applied View

Using the Hamilton-Jacobi-Bellman equation, we derive both a Keynes-Ramsey rule and a closed form solution for an optimal consumption-investment problem with labor income. The utility function is unbounded and uncertainty stems from a Poisson process. Our results can be derived because of the proofs presented in the accompanying paper by Sennewald (2006). Additional examples are given which highlight the correct use of the Hamilton-Jacobi-Bellman equation and the change-of-variables formula (sometimes referred to as “Ito’s-Lemma”) under Poisson uncertainty.stochastic differential equation, Poisson process, Bellman equation, portfolio optimization, consumption optimization

Optimal consumption of the finite time horizon Ramsey problem

AbstractIn this paper, we study the stochastic Ramsey problem related to an economic growth model with the CES production function in a finite time horizon. By changing variables, the Hamilton–Jacobi–Bellman equation associated with this optimization problem is transformed. By the viscosity solution technique, we show the existence of a classical solution of the transformed Hamilton–Jacobi–Bellman equation, and then give an optimal consumption policy of the original problem

Utility maximization with current utility on the wealth: regularity of solutions to the HJB equation

We consider a utility maximization problem for an investment-consumption portfolio when the current utility depends also on the wealth process. Such kind of problems arise, e.g., in portfolio optimization with random horizon or with random trading times. To overcome the difficulties of the problem we use the dual approach. We define a dual problem and treat it by means of dynamic programming, showing that the viscosity solutions of the associated Hamilton-Jacobi-Bellman equation belong to a suitable class of smooth functions. This allows to define a smooth solution of the primal Hamilton-Jacobi-Bellman equation, proving that this solution is indeed unique in a suitable class and coincides with the value function of the primal problem. Some financial applications of the results are provided

A Model for Optimal Human Navigation with Stochastic Effects

We present a method for optimal path planning of human walking paths in mountainous terrain, using a control theoretic formulation and a Hamilton-Jacobi-Bellman equation. Previous models for human navigation were entirely deterministic, assuming perfect knowledge of the ambient elevation data and human walking velocity as a function of local slope of the terrain. Our model includes a stochastic component which can account for uncertainty in the problem, and thus includes a Hamilton-Jacobi-Bellman equation with viscosity. We discuss the model in the presence and absence of stochastic effects, and suggest numerical methods for simulating the model. We discuss two different notions of an optimal path when there is uncertainty in the problem. Finally, we compare the optimal paths suggested by the model at different levels of uncertainty, and observe that as the size of the uncertainty tends to zero (and thus the viscosity in the equation tends to zero), the optimal path tends toward the deterministic optimal path