Optimal control formulation of transition path problems for Markov Jump Processes

Among various rare events, the effective computation of transition paths connecting metastable states in a stochastic model is an important problem. This paper proposes a stochastic optimal control formulation for transition path problems in an infinite time horizon for Markov jump processes on polish space. An unbounded terminal cost at a stopping time and a controlled transition rate for the jump process regulate the transition from one metastable state to another. The running cost is taken as an entropy form of the control velocity, in contrast to the quadratic form for diffusion processes. Using the Girsanov transformation for Markov jump processes, the optimal control problem in both finite time and infinite time horizon with stopping time fit into one framework: the optimal change of measures in the C\`adl\`ag path space via minimizing their relative entropy. We prove that the committor function, solved from the backward equation with appropriate boundary conditions, yields an explicit formula for the optimal path measure and the associated optimal control for the transition path problem. The unbounded terminal cost leads to a singular transition rate (unbounded control velocity), for which, the Gamma convergence technique is applied to pass the limit for a regularized optimal path measure. The limiting path measure is proved to solve a Martingale problem with an optimally controlled transition rate and the associated optimal control is given by Doob-h transformation. The resulting optimally controlled process can realize the transitions almost surely.Comment: 31 page

금융 수학에서의 자유 경계 문제

학위논문 (박사) -- 서울대학교 대학원 : 자연과학대학 수리과학부, 2021. 2. 강명주.This thesis focuses on theoretically investigating the free boundary problems which arise from two classical problems in mathematical finance: portfolio selection and optimal contract of principal-agent theory. In these two problems, the mathematical structure is not represented as a solely optimal stopping time problem but is formulated in a mixture form in which optimal stopping time is combined with stochastic control or singular control. In this structure, the characterization of the free boundary for optimal stopping time is subtle and the verification theorem of the optimality for the candidate control and stopping time is difficult to prove. To overcome these difficulties, we utilize the dual/martingale method, and then we analyze the variational inequality arising from the dual problem and formally provide the derivation of the solution to the variational inequality. In the thesis, we make a technical contribution by fully characterizing the free boundary and providing the duality and verification theorems. Based on the analytic results, we seek to gain further insight into two classical problems under several realistic model setups.본 학위 논문은 금융수학의 대표적인 두 가지 문제인 포트폴리오 이론, 최적 계약 이론에서 일어나는 자유 경계 문제 (free boundary problem) 에 대한 해석적 분석을 다룬다. 두 문제에서의 수학적 구조는 단순히 최적 정지 시간 (optimal stopping time) 문제로 나타나는 것이 아니라 확률적 제어 (stochastic control) 혹은 단일 제어 (singular control) 와 함께 결합된 형태로 귀착된다. 이러한 구조는 최적 정지 시간에 대한 자유 경계의 분석이 미묘하여 확률적 제어 및 정지 시간에 대한 최적성 (optimality) 증명이 어렵다. 본 논문은 이중/마팅게일 (dual/martingale) 이론을 활용하여 이중 문제 (dual problem) 에서 발생하는 변분 부등식 (variational inequality) 의 성질을 분석하고 그에 대한 해의 도출과정을 제시한다. 이를 통해 자유 경계를 완전히 특징화하고 이중성 및 최적성 증명을 함께 제시한다. 이러한 해석적 결과에 기반 하여 여러 가지 현실적인 모델에서의 포트폴리오 및 최적 계약 이론 대한 경제학적 의미를 제공한다.Contents Abstract i 1 Introduction 1 2 Optimal stopping in portfolio selection 7 2.1 Retirement decision in consumption-leisure and investment problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.1 Model and optimization problem . . . . . . . . . . . . 7 2.1.2 Main results: optimal retirement time and duality theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.1.3 Variational inequality: ODE analysis . . . . . . . . . . 15 2.1.4 Proof of Proposition 2.1.7 . . . . . . . . . . . . . . . . 17 2.1.5 Proof of Theorem 2.1.8 . . . . . . . . . . . . . . . . . 21 2.2 Risk preference change in consumption and investment problem 24 2.2.1 Model and optimization problem . . . . . . . . . . . . 24 2.2.2 Main results: optimal preference change time and duality theorem . . . . . . . . . . . . . . . . . . . . . . . 25 2.2.3 Variational inequality: PDE analysis . . . . . . . . . . 30 2.2.4 Proof of Theorem 2.2.4 . . . . . . . . . . . . . . . . . 32 2.2.5 Proof of Theorem 2.2.5 . . . . . . . . . . . . . . . . . 35 3 Singular control in portfolio selection 38 3.1 Consumption Ratcheting . . . . . . . . . . . . . . . . . . . . . 38 3.1.1 Model and optimization problem . . . . . . . . . . . . 39 3.1.2 Singular control and optimal stopping . . . . . . . . . 41 3.1.3 Main results: optimal consumption with ratcheting and retirement decision . . . . . . . . . . . . . . . . . 48 3.1.4 Proof of lemmas . . . . . . . . . . . . . . . . . . . . . 50 3.1.5 Proof of Theorem 3.1.10 . . . . . . . . . . . . . . . . . 56 3.1.6 Proof of Theorem 3.1.18 . . . . . . . . . . . . . . . . . 59 3.2 Drawdown Constraint on Consumption . . . . . . . . . . . . . 64 3.2.1 Model and optimization problem . . . . . . . . . . . . 65 3.2.2 Two dimensional singular control problem . . . . . . . 67 3.2.3 Main Results: optimal maximum process and optimal strategies . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.2.4 Proof of lemmas . . . . . . . . . . . . . . . . . . . . . 77 3.2.5 Proof of Proposition 3.2.15 . . . . . . . . . . . . . . . 87 3.2.6 Proof of Theorem 3.2.17 . . . . . . . . . . . . . . . . . 90 3.2.7 Proof of Theorem 3.2.18 . . . . . . . . . . . . . . . . . 96 4 Singular control in contract theory 99 4.1 Limited Commitment in Finite time horizon . . . . . . . . . . 99 4.1.1 Model and optimization problem . . . . . . . . . . . . 99 4.1.2 Singular control in nite time-horizon . . . . . . . . . 103 4.1.3 Main results: optimal contract with limited commitment 114 4.1.4 Proof of lemmas . . . . . . . . . . . . . . . . . . . . . 118 4.1.5 Proof of Proposition 4.1.16 . . . . . . . . . . . . . . . 124 4.1.6 Proof of Proposition 4.1.18 . . . . . . . . . . . . . . . 125 4.1.7 Proof of Theorem 4.1.19 . . . . . . . . . . . . . . . . . 126 4.1.8 Proof of Proposition 4.1.21 . . . . . . . . . . . . . . . 131 4.1.9 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . 133 Abstract (in Korean) 141 Acknowledgement (in Korean) 142Docto

On an optimal extraction problem with regime switching

Ferrari G, Yang S. On an optimal extraction problem with regime switching. Center for Mathematical Economics Working Papers. Vol 562. Bielefeld: Center for Mathematical Economics; 2016.This paper studies an optimal irreversible extraction problem of an exhaustible commodity in presence of regime shifts. A company extracts a natural resource from a reserve with finite capacity, and sells it in the market at a spot price that evolves according to a Brownian motion with volatility modulated by a two state Markov chain. In this setting, the company aims at finding the extraction rule that maximizes its expected, discounted net cash flow. The problem is set up as a finite-fuel two-dimensional degenerate singular stochastic control problem over an infinite time-horizon. We provide explicit expressions both for the value function and for the optimal control. We show that the latter prescribes a Skorokhod reflection of the optimally controlled state process at a certain state and price dependent threshold. This curve is given in terms of the optimal stopping boundary of an auxiliary family of perpetual optimal selling problems with regime switching. The techniques are those of stochastic calculus and stochastic optimal control theory

A unifying view on the irreversible investment exercise boundary in a stochastic, time-inhomogeneous capacity expansion problem

This paper studies the investment exercise boundary erasing in a stochastic, continuous time capacity expansion problem with irreversible investment on the finite time interval $[0, T]$ and a state dependent scrap value associated with the production facility at the finite horizon $T$. The capacity process is a time-inhomogeneous diffusion in which a monotone nondecreasing, possibly singular, process representing the cumulative investment enters additively. The levels of capacity, employment and operating capital contribute to the firm's production and are optimally chosen in order to maximize the expected total discounted profits. Two different approaches are employed to study and characterize the boundary. From one side, some first order condition are solved by using the Bank and El Karoui Representation Theorem, and that sheds further light on the connection between the threshold which the optimal policy of the singular stochastic control problem activates at and the optional solution of Representation Theorem. Its application in the presence of the scrap value is new. It is accomplished by a suitable devise to overcome the difficulties due to the presence of a non integral term in the maximizing functional. The optimal investment process is shown to become active at the so-called "base capacity" level, given as the unique solution of an integral equation. On the other hand, when the coefficients of the uncontrolled capacity process are deterministic, the optimal stopping problem classically associated to the original capacity problem is resumed and some essential properties of the investment exercise boundary are obtained. The optimal investment process is proved to be continuous. Unifying approaches and views, the exercise boundary is shown to coincide with the base capacity, hence it is characterized by an integral equation not requiring any a priori regularity

Controlled diffusion processes

This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.Comment: Published at http://dx.doi.org/10.1214/154957805100000131 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org

Irreversible Investment under L\'evy Uncertainty: an Equation for the Optimal Boundary

We derive a new equation for the optimal investment boundary of a general irreversible investment problem under exponential L\'evy uncertainty. The problem is set as an infinite time-horizon, two-dimensional degenerate singular stochastic control problem. In line with the results recently obtained in a diffusive setting, we show that the optimal boundary is intimately linked to the unique optional solution of an appropriate Bank-El Karoui representation problem. Such a relation and the Wiener Hopf factorization allow us to derive an integral equation for the optimal investment boundary. In case the underlying L\'evy process hits any real point with positive probability we show that the integral equation for the investment boundary is uniquely satisfied by the unique solution of another equation which is easier to handle. As a remarkable by-product we prove the continuity of the optimal investment boundary. The paper is concluded with explicit results for profit functions of (i) Cobb-Douglas type and (ii) CES type. In the first case the function is separable and in the second case non-separable.Comment: 19 page