Finite horizon optimal stopping of time-discontinuous functionals with applications to impulse control with delay

We study finite horizon optimal stopping problems for continuous-time Feller–Markov processes. The functional depends on time, state, and external parameters and may exhibit discontinuities with respect to the time variable. Both left- and right-hand discontinuities are considered. We investigate the dependence of the value function on the parameters, on the initial state of the process, and on the stopping horizon. We construct $\varepsilon$-optimal stopping times and provide conditions under which an optimal stopping time exists. We demonstrate how to approximate this optimal stopping time by solutions to discrete-time problems. Our results are applied to the study of impulse control problems with finite time horizon, decision lag, and execution delay

Approximate Dynamic Programming: Health Care Applications

This dissertation considers different approximate solutions to Markov decision problems formulated within the dynamic programming framework in two health care applications. Dynamic formulations are appropriate for problems which require optimization over time and a variety of settings for different scenarios and policies. This is similar to the situation in a lot of health care applications for which because of the curses of dimensionality, exact solutions do not always exist. Thus, approximate analysis to find near optimal solutions are motivated. To check the quality of approximation, additional evidence such as boundaries, consistency analysis, or asymptotic behavior evaluation are required. Emergency vehicle management and dose-finding clinical trials are the two heath care applications considered here in order to investigate dynamic formulations, approximate solutions, and solution quality assessments. The dynamic programming formulation for real-time ambulance dispatching and relocation policies, response-adaptive dose-finding clinical trial, and optimal stopping of adaptive clinical trials is presented. Approximate solutions are derived by multiple methods such as basis function regression, one-step look-ahead policy, simulation-based gridding algorithm, and diffusion approximation. Finally, some boundaries to assess the optimality gap and a proof of consistency for approximate solutions are presented to ensure the quality of approximation

VISCOSITY SOLUTIONS OF OPTIMAL STOPPING PROBLEMS FOR FELLER PROCESSES AND THEIR APPLICATIONS

This thesis constitutes a research work on deriving viscosity solutions to optimal stopping problems for Feller processes. We present conditions on the process under which the value function is the unique viscosity solution to a Hamilton-Jacobi-Bellman equation associated with a particular operator. More speci cally, assuming that the underlying controlled process is a Feller process, we prove the uniqueness of the viscosity solution. We also apply our results to study several examples of Feller processes. On the other hand, we try to extend our results by iterative optimal stopping methods in the rest of the work. This approach gives a numerical method to approximate the value function and suggest a way of nding the unique viscosity solution associated to the optimal stopping problem. We use it to study several relevant control problems which can reduce to corresponding optimal stopping problems. e.g., an impulse control problem as well as an optimal stopping problem for jump di usions and regime switching processes. In the end, as a complementary, we are trying to construct optimal stopping problems with multiplicative functionals related to a non-conservative Feller semigroup. As a consequence, viscosity solutions were obtained for such kind of constructions

Computational methods for stochastic control problems with applications in finance

textStochastic control is a broad tool with applications in several areas of academic interest. The financial literature is full of examples of decisions made under uncertainty and stochastic control is a natural framework to deal with these problems. Problems such as optimal trading, option pricing and economic policy all fall under the purview of stochastic control. These problems often face nonlinearities that make analytical solutions infeasible and thus numerical methods must be employed to find approximate solutions. In this dissertation three types of stochastic control formulations are used to model applications in finance and numerical methods are developed to solve the resulting nonlinear problems. To begin with, optimal stopping is applied to option pricing. Next, impulse control is used to study the problem of interest rate control faced by a nation's central bank, and finally a new type of hybrid control is developed and applied to an investment decision faced by money managers.Information, Risk, and Operations Management (IROM

Risk-sensitive optimal stopping with unbounded terminal cost function

In this paper we consider an infinite time horizon risk-sensitive optimal stopping problem for a Feller--Markov process with an unbounded terminal cost function. We show that in the unbounded case an associated Bellman equation may have multiple solutions and we give a probabilistic interpretation for the minimal and the maximal one. Also, we show how to approximate them using finite time horizon problems. The analysis, covering both discrete and continuous time case, is supported with illustrative examples.Comment: 32 pages, added Example 7, several rearrangement

Regression and convex switching system methods for stochastic control problems with applications to multiple-exercise options

University of Technology Sydney. Faculty of Business.In this thesis, we develop several new simulation-based algorithms for solving some important classes of optimal stochastic control problems. In particular, these methods are aimed at providing good approximate solutions to problems that involve a high-dimensional underlying processes. These algorithms are of the primal-dual kind and therefore provide a gauge of the distance to optimality of the given approximate solutions to the optimal one. These methods will be used in the pricing of the multiple-exercise option. In Chapter 1, we conduct a review of the literature that is relevant to the pricing of the multiple-exercise option and the primal and dual methods that we will be developing in this thesis. In the next two chapters of the thesis, we focus on regression-based dual methods for optimal multiple stopping problems in probability theory. In particular, we concentrate on finding upper bounds on the price of the multiple-exercise option as it sits within this framework. In Chapter 2, we derive an additive dual for the multiple-exercise options using financial arguments, and see that this approach leads to the construction of an algorithm that has greater computational efficiency than other methods in the literature. In Chapter 3, we derive the first known dual of the multiplicative kind for the multiple-exercise option and devise a tractable algorithm to compute it. In the penultimate chapter of the thesis, we focus on a new class of algorithms that are based on what is known as convex switching system. These algorithms provide approximate solutions to the more general class of optimal stochastic switching problems. In Chapter 4, techniques based on combinations of rigorous theory and heuristics arguments are used to improve the efficiency and applicability of the method. We then devise algorithms of the primal-dual kind to assess the accuracy of this approach. Chapter 5 concludes

Consistent Second-Order Conic Integer Programming for Learning Bayesian Networks

Bayesian Networks (BNs) represent conditional probability relations among a set of random variables (nodes) in the form of a directed acyclic graph (DAG), and have found diverse applications in knowledge discovery. We study the problem of learning the sparse DAG structure of a BN from continuous observational data. The central problem can be modeled as a mixed-integer program with an objective function composed of a convex quadratic loss function and a regularization penalty subject to linear constraints. The optimal solution to this mathematical program is known to have desirable statistical properties under certain conditions. However, the state-of-the-art optimization solvers are not able to obtain provably optimal solutions to the existing mathematical formulations for medium-size problems within reasonable computational times. To address this difficulty, we tackle the problem from both computational and statistical perspectives. On the one hand, we propose a concrete early stopping criterion to terminate the branch-and-bound process in order to obtain a near-optimal solution to the mixed-integer program, and establish the consistency of this approximate solution. On the other hand, we improve the existing formulations by replacing the linear "big-$M$" constraints that represent the relationship between the continuous and binary indicator variables with second-order conic constraints. Our numerical results demonstrate the effectiveness of the proposed approaches

Order reduction methods for solving large-scale differential matrix Riccati equations

We consider the numerical solution of large-scale symmetric differential matrix Riccati equations. Under certain hypotheses on the data, reduced order methods have recently arisen as a promising class of solution strategies, by forming low-rank approximations to the sought after solution at selected timesteps. We show that great computational and memory savings are obtained by a reduction process onto rational Krylov subspaces, as opposed to current approaches. By specifically addressing the solution of the reduced differential equation and reliable stopping criteria, we are able to obtain accurate final approximations at low memory and computational requirements. This is obtained by employing a two-phase strategy that separately enhances the accuracy of the algebraic approximation and the time integration. The new method allows us to numerically solve much larger problems than in the current literature. Numerical experiments on benchmark problems illustrate the effectiveness of the procedure with respect to existing solvers

El método de aproximación por media muestral

The sample average approximation (SAA) method is an approach for solving stochastic optimization problems by using Monte Carlo simulation. The basic idea of such method is that we can approximate the expected objetive function by the corresponding sample average function using a random sample. We solve the obtained sample average approximating problem by deterministic optimization techniques, and the process is repeated several times with different samples to obtain candidate solutions along with statistical estimates of their optimality gaps until a stopping criterion is satisfied. In section 1 we describe the expected value and sample average approximation problems and give a few examples of real cases in which it can be useful. In section 2 we show many results related to convergence of estimators (objective value, optimal solution, etc) under certain assumptions. In section 3 we discuss convergence rates of objetive values. In section 4 we implement the method to study two problems (that involve different random variables) to illustrate the power of the method.Universidad de Sevilla. Grado en Matemática

Optimal stopping and hard terminal constraints applied to a missile guidance problem

This paper describes two new types of deterministic optimal stopping control problems: optimal stopping control with hard terminal constraints only and optimal stopping control with both minimum control effort And hard termind constraints. Both problems are initially formulated in continuous-time (a discretetime formulation is given towards the end of the paper) and soIutions given via dynamic programming. A numeric solution to the continuous-time dynamic programming equations is then briefly discussed. The optimal stopping with terminal constraints problem in continuous-time is a natural description of a particular type of missile guidance problem. This missile guidance appiication is introduced and the presented solutions used in missile engagements against targets