Controlled diffusion processes

This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.Comment: Published at http://dx.doi.org/10.1214/154957805100000131 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org

Equilibrium Exhaustible Resource Price Dynamics

We develop equilibrium models of an exhaustible resource market where both prices and extraction choices are determined endogenously. Our analysis highlights a role for adjustment costs in generating price dynamics that are consistent with observed oil and gas forward prices as well as with the two-factor prices processes that were calibrated in Schwartz and Smith (2000). Stochastic volatility aries in our two-factor model as a natural consequence of production for oil and natural gas prices. Differences between the endogenous price processes considered in earlier papers can generate significant differences in both financial and real option values.

Optimal control of a Brownian storage system

AbstractConsider a storage system (such as an inventory or bank account) whose content fluctuates as a Brownian Motion X = {X(t), t ⩾ 0} in the absence of any control. Let Y = {Y(t), t ⩾ 0} and Z = {Z(t), t ⩾ 0} be non-decreasing, non-anticipating functionals representing the cumulative input to the system and cumulative output from the system respectively. Theproblem is to choose Y and Z so as to minimize expected discounted cost subject to the requirement that X(t) + Y(t) - Z(t) ⩾ 0 for all t ⩾ 0 almost surely. In our first formulation, we assume a proportional input cost, a linear holding cost, and a proportional output reward (or cost). We explicitly compute an optimal policy involving a single critical number. In our second formulation, the cumulative input Y is required to be a step function, and an additional fixed charge is incurred each time that an input jump occurs. We explicitly compute an optimal policy involving two critical numbers. Applications to inventory/production control and stochastic cash management are discussed

Applications of Stochastic Control in Energy Real Options and Market Illiquidity

We present three interesting applications of stochastic control in finance. The first is a real option model that considers the optimal entry into and subsequent operation of a biofuel production facility. We derive the associated Hamilton Jacobi Bellman (HJB) equation for the entry and operating decisions along with the econometric analysis of the stochastic price inputs. We follow with a Monte Carlo analysis of the risk profile for the facility. The second application expands on the analysis of the biofuel facility to account for the associated regulatory and taxation uncertainty experienced by players in the renewables and energy industries. A federal biofuel production subsidy per gallon has been available to producers for many years but the subsidy price level has changed repeatedly. We model this uncertain price as a jump process. We present and solve the HJB equations for the associated multidimensional jump diffusion problem which also addresses the model uncertainty pervasive in real option problems such as these. The novel real option framework we present has many applications for industry practitioners and policy makers dealing with country risk or regulatory uncertainty---which is a very real problem in our current global environment. Our final application (which, although apparently different from the first two applications, uses the same tools) addresses the problem of producing reliable bid-ask spreads for derivatives in illiquid markets. We focus on the hedging of over the counter (OTC) equity derivatives where the underlying assets realistically have transaction costs and possible illiquidity which standard finance models such as Black-Scholes neglect. We present a model for hedging under market impact (such as bid-ask spreads, order book depth, liquidity) using temporary and permanent equity price impact functions and derive the associated HJB equations for the problem. This model transitions from continuous to impulse trading (control) with the introduction of fixed trading costs. We then price and hedge via the economically sound framework of utility indifference pricing. The problem of hedging under liquidity impact is an on-going concern of market makers following the Global Financial Crisis

Optimal Dynamic Procurement Policies for a Storable Commodity with L\'evy Prices and Convex Holding Costs

In this paper we study a continuous time stochastic inventory model for a commodity traded in the spot market and whose supply purchase is affected by price and demand uncertainty. A firm aims at meeting a random demand of the commodity at a random time by maximizing total expected profits. We model the firm's optimal procurement problem as a singular stochastic control problem in which controls are nondecreasing processes and represent the cumulative investment made by the firm in the spot market (a so-called stochastic "monotone follower problem"). We assume a general exponential L\'evy process for the commodity's spot price, rather than the commonly used geometric Brownian motion, and general convex holding costs. We obtain necessary and sufficient first order conditions for optimality and we provide the optimal procurement policy in terms of a "base inventory" process; that is, a minimal time-dependent desirable inventory level that the firm's manager must reach at any time. In particular, in the case of linear holding costs and exponentially distributed demand, we are also able to obtain the explicit analytic form of the optimal policy and a probabilistic representation of the optimal revenue. The paper is completed by some computer drawings of the optimal inventory when spot prices are given by a geometric Brownian motion and by an exponential jump-diffusion process. In the first case we also make a numerical comparison between the value function and the revenue associated to the classical static "newsvendor" strategy.Comment: 28 pages, 3 figures; improved presentation, added new results and section

Long-run average cost minimization of a stochastic processing system

A long-run average cost problem in stochastic control theory is addressed. This problem is related to the optimal control of a production-inventory system which is subjected to random fluctuation. The approach taken here is based on finding a smooth solution to the corresponding Hamilton-Jacobi-Bellman equation. This solution in turn is used to derive an optimal process for the above long-run average cost problem. Using the invariant distributions for positive recurrent diffusion processes, another derivation for the optimal long-run average cost is provided here