Unscented Bayesian Optimization for Safe Robot Grasping

We address the robot grasp optimization problem of unknown objects considering uncertainty in the input space. Grasping unknown objects can be achieved by using a trial and error exploration strategy. Bayesian optimization is a sample efficient optimization algorithm that is especially suitable for this setups as it actively reduces the number of trials for learning about the function to optimize. In fact, this active object exploration is the same strategy that infants do to learn optimal grasps. One problem that arises while learning grasping policies is that some configurations of grasp parameters may be very sensitive to error in the relative pose between the object and robot end-effector. We call these configurations unsafe because small errors during grasp execution may turn good grasps into bad grasps. Therefore, to reduce the risk of grasp failure, grasps should be planned in safe areas. We propose a new algorithm, Unscented Bayesian optimization that is able to perform sample efficient optimization while taking into consideration input noise to find safe optima. The contribution of Unscented Bayesian optimization is twofold as if provides a new decision process that drives exploration to safe regions and a new selection procedure that chooses the optimal in terms of its safety without extra analysis or computational cost. Both contributions are rooted on the strong theory behind the unscented transformation, a popular nonlinear approximation method. We show its advantages with respect to the classical Bayesian optimization both in synthetic problems and in realistic robot grasp simulations. The results highlights that our method achieves optimal and robust grasping policies after few trials while the selected grasps remain in safe regions.Comment: conference pape

Inverse Optimal Stopping and Optimal Closure of Illiquid Positions

Many economic situations are modeled as stopping problems. Examples include job search, pricing of American options, timing of market entry and irreversible investment decisions. The first part of the thesis analyzes optimal stopping in a dynamic mechanism design framework. It deals with a principal-agent problem where the principal and the agent have different preferences over stopping rules. The agent privately observes a one-dimensional Markov process that influences her payoff. Based on her observation the agent decides when to stop. In order to induce the agent to employ a different stopping rule the principal commits to a transfer that depends only on the time the agent stopped. The goal is to characterize the set of stopping rules that can be implemented using such a transfer. To this end the well-known single crossing condition from static mechanism design is transferred to optimal stopping problems. In a discrete-time framework it is shown that under this condition a stopping rule is implementable if and only if it is of cut-off type. If time is continuous, a cut-off rule is implementable provided that the associated threshold satisfies certain regularity assumptions. The transfer admits a closed form representation based on the reflected version of the underlying Markov process. A uniqueness result for the transfer is provided. As a consequence one obtains a new nonlinear integral equation characterizing the optimal stopping boundary in one-dimensional stopping problems. The second part of this thesis analyzes the problem of how to close a large asset position in an illiquid market. The first goal is to characterize trading strategies that make very high liquidation costs unlikely. To this end a model that allows for a price-sensitive closure of the position is set-up. It provides a simple device for designing and controlling the distribution of the revenues from unwinding the position. The risk inherent in the open position is modeled by a functional that can be interpreted as the time-average of the squared value-at-risk of the open position. Market illiquidity is reflected by a linear, temporary price impact. The stochastic control problem consists of minimizing a weighted sum of the execution costs and the risk functional. By appealing to dynamic programming, semi-explicit formulas for the optimal execution strategies are derived in a discrete-time framework. Within the continuous-time version of the model the optimal trading rates can be characterized in terms of a partial differential equation (PDE) describing by how much they differ from the optimal risk-neutral trading rate. The PDE possesses a singularity and does not, in general, have a closed-form solution. A uniqueness result for solutions in the viscosity sense is provided, allowing in the following to identify the value function and optimal trading rates. It is shown that optimal strategies from the discrete model converge to the continuous-time optimal trading rates. In the next step this model is generalized by incorporating a stochastic price impact. The liquidation constraint is relaxed by introducing a set of scenarios where the position does not have to be closed. A purely probabilistic solution of this not necessarily Markovian control problem is provided by means of a backward stochastic differential equation (BSDE). The BSDE in this problem possesses a singular terminal condition. It is shown that a minimal supersolution of the BSDE exists. Special cases for which the control problem has explicit solutions are discussed. Finally, the impact of a cross-hedging opportunity on liquidation strategies is analyzed. Suppose there is an open position to be closed in an illiquid forward market (e.g. a commodity market) before delivery. The liquidity of the asset increases as the delivery date approaches. Therefore, an early closure eliminates the risk inherent in the open position but also omits the opportunity of reducing execution costs. Assume further that there is a proxy market where forwards of a correlated asset are traded. Liquidity in the proxy market is high and thus performing a cross-hedge reduces execution costs. However, since the prices are not perfectly correlated, this hedging strategy entails basis risk. Using techniques from singular stochastic control theory allows to obtain an optimal trade-off between execution costs and basis risk. Explicit optimal hedging strategies for simple liquidity dynamics are derived

Drift dependence of optimal trade execution strategies under transient price impact

We give a complete solution to the problem of minimizing the expected liquidity costs in presence of a general drift when the underlying market impact model has linear transient price impact with exponential resilience. It turns out that this problem is well-posed only if the drift is absolutely continuous. Optimal strategies often do not exist, and when they do, they depend strongly on the derivative of the drift. Our approach uses elements from singular stochastic control, even though the problem is essentially non-Markovian due to the transience of price impact and the lack in Markovian structure of the underlying price process. As a corollary, we give a complete solution to the minimization of a certain cost-risk criterion in our setting

Optimal execution strategy with an uncertain volume target

In the seminal paper on optimal execution of portfolio transactions, Almgren and Chriss (2001) define the optimal trading strategy to liquidate a fixed volume of a single security under price uncertainty. Yet there exist situations, such as in the power market, in which the volume to be traded can only be estimated and becomes more accurate when approaching a specified delivery time. During the course of execution, a trader should then constantly adapt their trading strategy to meet their fluctuating volume target. In this paper, we develop a model that accounts for volume uncertainty and we show that a risk-averse trader has benefit in delaying their trades. More precisely, we argue that the optimal strategy is a trade-off between early and late trades in order to balance risk associated with both price and volume. By incorporating a risk term related to the volume to trade, the static optimal strategies suggested by our model avoid the explosion in the algorithmic complexity usually associated with dynamic programming solutions, all the while yielding competitive performance

Theoretical and Numerical Analysis of an Optimal Execution Problem with Uncertain Market Impact

This paper is a continuation of Ishitani and Kato (2015), in which we derived a continuous-time value function corresponding to an optimal execution problem with uncertain market impact as the limit of a discrete-time value function. Here, we investigate some properties of the derived value function. In particular, we show that the function is continuous and has the semigroup property, which is strongly related to the Hamilton-Jacobi-Bellman quasi-variational inequality. Moreover, we show that noise in market impact causes risk-neutral assessment to underestimate the impact cost. We also study typical examples under a log-linear/quadratic market impact function with Gamma-distributed noise.Comment: 24 pages, 14 figures. Continuation of the paper arXiv:1301.648

Optimal Trading Strategies in a Limit Order Market with Imperfect Liquidity

We study the optimal execution strategy of selling a security. In a continuous time diffusion framework, a risk-averse trader faces the choice of selling the security promptly or placing a limit order and hence delaying the transaction in order to sell at a more favorable price. We introduce a random delay parameter, which defers limit order execution and characterizes market liquidity. The distribution of expected time-to-fill of limit orders conforms to the empirically observed exponential distribution of trading times, and its variance decreases with liquidity. We obtain a closed-form solution and demonstrate that the presence of the lag factor linearizes the impact of other market parameters on the optimal limit price. Finally, two more stylized facts are rationalized in our model: the equilibrium bid-ask spread decreases with liquidity, but increases with agents risk aversion