Controlled diffusion processes

This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.Comment: Published at http://dx.doi.org/10.1214/154957805100000131 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org

Value Iteration for Long-run Average Reward in Markov Decision Processes

Markov decision processes (MDPs) are standard models for probabilistic systems with non-deterministic behaviours. Long-run average rewards provide a mathematically elegant formalism for expressing long term performance. Value iteration (VI) is one of the simplest and most efficient algorithmic approaches to MDPs with other properties, such as reachability objectives. Unfortunately, a naive extension of VI does not work for MDPs with long-run average rewards, as there is no known stopping criterion. In this work our contributions are threefold. (1) We refute a conjecture related to stopping criteria for MDPs with long-run average rewards. (2) We present two practical algorithms for MDPs with long-run average rewards based on VI. First, we show that a combination of applying VI locally for each maximal end-component (MEC) and VI for reachability objectives can provide approximation guarantees. Second, extending the above approach with a simulation-guided on-demand variant of VI, we present an anytime algorithm that is able to deal with very large models. (3) Finally, we present experimental results showing that our methods significantly outperform the standard approaches on several benchmarks

The Canadian Business Cycle: A Comparison of Models

This paper examines the ability of linear and nonlinear models to replicate features of real Canadian GDP. We evaluate the models using various business-cycle metrics. From the 9 data generating processes designed, none can completely accommodate every business-cycle metric under consideration. Richness and complexity do not guarantee a close match with Canadian data. Our findings for Canada are consistent with Piger and Morley's (2005) study of the United States data and confirms the contradiction of their results with those reported by Engel, Haugh, and Pagan (2005): nonlinear models do provide an improvement in matching business-cycle features. Lastly, the empirical results suggest that investigating the merits of forecast combination would be worthwhile.Business fluctuations and cycles; Econometric and statistical methods

Federal Funds Rate Prediction

Recent research has reported that both the federal funds rate futures market and the federal funds target contain valuable information for explaining the behavior of the US effective federal funds rate. A parallel literature on interest rate modelling has recorded evidence that the dynamics of interest rates displays significant regime-switching behavior. In this paper we produce out of sample forecasts of the federal funds rate at horizons up to 8 weeks ahead using linear and nonlinear, regime-switching equilibrium correction models of the funds rate and employing both point and density measures of forecast accuracy. We cannot discriminate among the models considered in terms of point forecast accuracy. However, in terms of density forecast accuracy, we find that the term structure model of the federal funds futures rate is significantly better than the other models considered, and that regime-switching models provide a substantial forecasting improvement relative to their linear counterparts and relative to individual series of the futures rate.federal funds rate, term structure of interest rates, forecasting, nonlinearity

DYNAMIC PROGRAMMING: HAS ITS DAY ARRIVED?

Research Methods/ Statistical Methods,

Controllability Metrics on Networks with Linear Decision Process-type Interactions and Multiplicative Noise

This paper aims at the study of controllability properties and induced controllability metrics on complex networks governed by a class of (discrete time) linear decision processes with mul-tiplicative noise. The dynamics are given by a couple consisting of a Markov trend and a linear decision process for which both the "deterministic" and the noise components rely on trend-dependent matrices. We discuss approximate, approximate null and exact null-controllability. Several examples are given to illustrate the links between these concepts and to compare our results with their continuous-time counterpart (given in [16]). We introduce a class of backward stochastic Riccati difference schemes (BSRDS) and study their solvability for particular frameworks. These BSRDS allow one to introduce Gramian-like controllability metrics. As application of these metrics, we propose a minimal intervention-targeted reduction in the study of gene networks