Energy Efficient Execution of POMDP Policies

Recent advances in planning techniques for partially observable Markov decision processes have focused on online search techniques and offline point-based value iteration. While these techniques allow practitioners to obtain policies for fairly large problems, they assume that a non-negligible amount of computation can be done between each decision point. In contrast, the recent proliferation of mobile and embedded devices has lead to a surge of applications that could benefit from state of the art planning techniques if they can operate under severe constraints on computational resources. To that effect, we describe two techniques to compile policies into controllers that can be executed by a mere table lookup at each decision point. The first approach compiles policies induced by a set of alpha vectors (such as those obtained by point-based techniques) into approximately equivalent controllers, while the second approach performs a simulation to compile arbitrary policies into approximately equivalent controllers. We also describe an approach to compress controllers by removing redundant and dominated nodes, often yielding smaller and yet better controllers. Further compression and higher value can sometimes be obtained by considering stochastic controllers. The compilation and compression techniques are demonstrated on benchmark problems as well as a mobile application to help persons with Alzheimer's to way-find. The battery consumption of several POMDP policies is compared against finite-state controllers learned using methods introduced in this paper. Experiments performed on the Nexus 4 phone show that finite-state controllers are the least battery consuming POMDP policies

Robust Equilibrium Yield Curves

This paper studies the quantitative implications of the interaction between robust control and stochastic volatility for key asset pricing phenomena. We present an equilibrium term structure model in which output growth is conditionally heteroskedastic. The agent does not know the true model of the economy and chooses optimal policies that are robust to model misspecification. The choice of robust policies greatly amplifies the effect of conditional heteroskedasticity in consumption growth, improving the model's ability to explain asset prices. In a robust control framework, stochastic volatility in consumption growth generates both a state-dependent market price of model uncertainty and a stochastic market price of risk. We estimate the model using data from the bond and equity market, as well as consumtion data. We show that the model is consistent with key empirical regularities that characterize the bond and equity markets. We also characterize empirically the set of models the robust representative agent entertains, and show that this set is "small". In other words, it is statistically difficult to distinguish between models in this set.Yield curve, market price of uncertainty, robust control

Active model learning and diverse action sampling for task and motion planning

The objective of this work is to augment the basic abilities of a robot by learning to use new sensorimotor primitives to enable the solution of complex long-horizon problems. Solving long-horizon problems in complex domains requires flexible generative planning that can combine primitive abilities in novel combinations to solve problems as they arise in the world. In order to plan to combine primitive actions, we must have models of the preconditions and effects of those actions: under what circumstances will executing this primitive achieve some particular effect in the world? We use, and develop novel improvements on, state-of-the-art methods for active learning and sampling. We use Gaussian process methods for learning the conditions of operator effectiveness from small numbers of expensive training examples collected by experimentation on a robot. We develop adaptive sampling methods for generating diverse elements of continuous sets (such as robot configurations and object poses) during planning for solving a new task, so that planning is as efficient as possible. We demonstrate these methods in an integrated system, combining newly learned models with an efficient continuous-space robot task and motion planner to learn to solve long horizon problems more efficiently than was previously possible.Comment: Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain. https://www.youtube.com/playlist?list=PLoWhBFPMfSzDbc8CYelsbHZa1d3uz-W_

Robust Equilibrium Yield Curves

This paper studies the quantitative implications of the interaction between robust control and stochastic volatility for key asset pricing phenomena. We present an equilibrium term structure model with a representative agent and an output growth process that is conditionally heteroskedastic. The agent does not know the true model of the economy and chooses optimal policies that are robust to model misspecification. The choice of robust policies greatly amplifies the effect of conditional heteroskedasticity in consumption growth, improving the model’s ability to explain asset prices. In a robust control framework, stochastic volatility in consumption growth generates both a state-dependent market price of model uncertainty and a stochastic market price of risk. We estimate the model using data from the bond and equity markets, as well as consumption data. We show that the model is consistent with key empirical regularities that characterize the bond and equity markets. We also characterize empirically the set of models the robust representative agent entertains, and show that this set is ?small?. That is, it is statistically difficult to distinguish between models in this set.Yield curves, Market price of Uncertainty, Robust control.

ON CHOOSING A BASE COVERAGE LEVEL FOR MULTIPLE PERIL CROP INSURANCE CONTRACTS

For multiple peril crop insurance, the U.S. Department of Agriculture'Â’s Risk Management Agency estimates the premium rate for a base coverage level and then uses multiplicative adjustment factors to recover rates at other coverage levels. Given this methodology, accurate estimation of the base coverage level from 65% to 50%. The purpose of this analysis was to provide some insight into whether such a change should or should not be carried out. Not surprisingly, our findings indicate that the higher coverage level should be maintained as the base.Risk and Uncertainty,

A FLEXIBLE METHOD FOR EMPIRICALLY ESTIMATING PROBABILITY FUNCTIONS

This paper presents a hyperbolic trigonometric (HT) transformation procedure for empirically estimating a cumulative probability distribution function (cdf), from which the probability density function (pdf) can be obtained by differentiation. Maximum likelihood (ML) is the appropriate estimation technique, but a particularly appealing feature of the HT transformation as opposed to other zero-one transformations is that the transformed cdf can be fitted with ordinary least squares (OLS) regression. Although OLS estimates are biased and inconsistent, they are usually very close to ML estimates; thus use of OLS estimates as starting values greatly facilitates use of numerical search procedures to obtain ML estimates. ML estimates have desirable asymptotic properties. The procedure is no more difficult to use than unconstrained nonlinear regression. Advantages of the procedure as compared to alternative procedures for fitting probability functions are discussed in the manuscript. Use of the conditional method is illustrated by application to two sets of yield response data.Research Methods/ Statistical Methods,