Leveraging Distributional Bias for Reactive Collision Avoidance under Uncertainty: A Kernel Embedding Approach

Many commodity sensors that measure the robot and dynamic obstacle's state have non-Gaussian noise characteristics. Yet, many current approaches treat the underlying-uncertainty in motion and perception as Gaussian, primarily to ensure computational tractability. On the other hand, existing planners working with non-Gaussian uncertainty do not shed light on leveraging distributional characteristics of motion and perception noise, such as bias for efficient collision avoidance. This paper fills this gap by interpreting reactive collision avoidance as a distribution matching problem between the collision constraint violations and Dirac Delta distribution. To ensure fast reactivity in the planner, we embed each distribution in Reproducing Kernel Hilbert Space and reformulate the distribution matching as minimizing the Maximum Mean Discrepancy (MMD) between the two distributions. We show that evaluating the MMD for a given control input boils down to just matrix-matrix products. We leverage this insight to develop a simple control sampling approach for reactive collision avoidance with dynamic and uncertain obstacles. We advance the state-of-the-art in two respects. First, we conduct an extensive empirical study to show that our planner can infer distributional bias from sample-level information. Consequently, it uses this insight to guide the robot to good homotopy. We also highlight how a Gaussian approximation of the underlying uncertainty can lose the bias estimate and guide the robot to unfavorable states with a high collision probability. Second, we show tangible comparative advantages of the proposed distribution matching approach for collision avoidance with previous non-parametric and Gaussian approximated methods of reactive collision avoidance

UrbanFly: Uncertainty-Aware Planning for Navigation Amongst High-Rises with Monocular Visual-Inertial SLAM Maps

We present UrbanFly: an uncertainty-aware real-time planning framework for quadrotor navigation in urban high-rise environments. A core aspect of UrbanFly is its ability to robustly plan directly on the sparse point clouds generated by a Monocular Visual Inertial SLAM (VINS) backend. It achieves this by using the sparse point clouds to build an uncertainty-integrated cuboid representation of the environment through a data-driven monocular plane segmentation network. Our chosen world model provides faster distance queries than the more common voxel-grid representation, and UrbanFly leverages this capability in two different ways leading to as many trajectory optimizers. The first optimizer uses a gradient-free cross-entropy method to compute trajectories that minimize collision probability and smoothness cost. Our second optimizer is a simplified version of the first and uses a sequential convex programming optimizer initialized based on probabilistic safety estimates on a set of randomly drawn trajectories. Both our trajectory optimizers are made computationally tractable and independent of the nature of underlying uncertainty by embedding the distribution of collision violations in Reproducing Kernel Hilbert Space. Empowered by the algorithmic innovation, UrbanFly outperforms competing baselines in metrics such as collision rate, trajectory length, etc., on a high fidelity AirSim simulator augmented with synthetic and real-world dataset scenes.Comment: Submitted to IROS 2022, Code available at https://github.com/sudarshan-s-harithas/UrbanFl

Function Embeddings for Multi-modal Bayesian Inference

Tractable Bayesian inference is a fundamental challenge in robotics and machine learning. Standard approaches such as Gaussian process regression and Kalman filtering make strong Gaussianity assumptions about the underlying distributions. Such assumptions, however, can quickly break down when dealing with complex systems such as the dynamics of a robot or multi-variate spatial models. In this thesis we aim to solve Bayesian regression and filtering problems without making assumptions about the underlying distributions. We develop techniques to produce rich posterior representations for complex, multi-modal phenomena. Our work extends kernel Bayes' rule (KBR), which uses empirical estimates of distributions derived from a set of training samples and embeds them into a high-dimensional reproducing kernel Hilbert space (RKHS). Bayes' rule itself occurs on elements of this space. Our first contribution is the development of an efficient method for estimating posterior density functions from kernel Bayes' rule, applied to both filtering and regression. By embedding fixed-mean mixtures of component distributions, we can efficiently find an approximate pre-image by optimising the mixture weights using a convex quadratic program. The result is a complex, multi-modal posterior representation. Our next contributions are methods for estimating cumulative distributions and quantile estimates from the posterior embedding of kernel Bayes' rule. We examine a number of novel methods, including those based on our density estimation techniques, as well as directly estimating the cumulative through use of the reproducing property of RKHSs. Finally, we develop a novel method for scaling kernel Bayes' rule inference to large datasets, using a reduced-set construction optimised using the posterior likelihood. This method retains the ability to perform multi-output inference, as well as our earlier contributions to represent explicitly non-Gaussian posteriors and quantile estimates

Non-parametric policy search with limited information loss

Learning complex control policies from non-linear and redundant sensory input is an important challenge for reinforcement learning algorithms. Non-parametric methods that approximate values functions or transition models can address this problem, by adapting to the complexity of the dataset. Yet, many current non-parametric approaches rely on unstable greedy maximization of approximate value functions, which might lead to poor convergence or oscillations in the policy update. A more robust policy update can be obtained by limiting the information loss between successive state-action distributions. In this paper, we develop a policy search algorithm with policy updates that are both robust and non-parametric. Our method can learn non-parametric control policies for infinite horizon continuous Markov decision processes with non-linear and redundant sensory representations. We investigate how we can use approximations of the kernel function to reduce the time requirements of the demanding non-parametric computations. In our experiments, we show the strong performance of the proposed method, and how it can be approximated efficiently. Finally, we show that our algorithm can learn a real-robot underpowered swing-up task directly from image data

Non-parametric policy search with limited information loss

Learning complex control policies from non-linear and redundant sensory input is an important challenge for reinforcement learning algorithms. Non-parametric methods that approximate values functions or transition models can address this problem, by adapting to the complexity of the data set. Yet, many current non-parametric approaches rely on unstable greedy maximization of approximate value functions, which might lead to poor convergence or oscillations in the policy update. A more robust policy update can be obtained by limiting the information loss between successive state-action distributions. In this paper, we develop a policy search algorithm with policy updates that are both robust and non-parametric. Our method can learn non- parametric control policies for infinite horizon continuous Markov decision processes with non-linear and redundant sensory representations. We investigate how we can use approximations of the kernel function to reduce the time requirements of the demanding non-parametric computations. In our experiments, we show the strong performance of the proposed method, and how it can be approximated efficiently. Finally, we show that our algorithm can learn a real-robot under-powered swing-up task directly from image data

PDE-Constrained Equilibrium Problems under Uncertainty: Existence, Optimality Conditions and Regularization

In dieser Arbeit werden PDE-beschränkte Gleichgewichtsprobleme unter Unsicherheiten analysiert. Im Detail diskutieren wir eine Klasse von risikoneutralen verallgemeinerten Nash-Gleichgewichtsproblemen sowie eine Klasse von risikoaversen Nash Gleichgewichtsproblemen. Sowohl für die risikoneutralen PDE-beschränkten Optimierungsprobleme mit punktweisen Zustandsschranken als auch für die risikoneutralen verallgemeinerten Nash Gleichgewichtsprobleme wird die Existenz von Lösungen beziehungsweise Nash Gleichgewichten bewiesen und Optimalitätsbedingungen hergeleitet. Die Betrachtung von Ungleichheitsbedingungen an den stochastischen Zustand führt in beiden Fällen zu Komplikationen bei der Herleitung der Lagrange-Multiplikatoren. Nur durch höhere Regularität des stochastischen Zustandes können wir auf die bestehende Optimalitätstheorie für konvexe Optimierungsprobleme zurückgreifen. Die niedrige Regularität des Lagrange-Multiplikators stellt auch für die numerische Lösbarkeit dieser Probleme ein große Herausforderung dar. Wir legen den Grundstein für eine erfolgreiche numerische Behandlung risikoneutraler Nash Gleichgewichtsproblem mittels Moreau-Yosida Regularisierung, indem wir zeigen, dass dieser Regularisierungsansatz konsistent ist. Die Moreau-Yosida Regularisierung liefert eine Folge von parameterabhängigen Nash Gleichgewichtsproblemen und der Grenzübergang im Glättungsparameter zeigt, dass die stationären Punkte des regularisierten Problems gegen ein verallgemeinertes Nash Gleichgewicht des ursprünglich Problems schwach konvergieren. Die Theorie legt also nahe, dass auf der Moreau-Yosida Regularisierung eine numerische Methode aufgebaut werden kann. Darauf aufbauend werden Algorithmen vorgeschlagen, die aufzeigen, wie risikoneutrale PDE-beschränkte Optimierungsprobleme mit punktweisen Zustandsschranken und risikoneutrale PDE-beschränkte verallgemeinerte Nash Gleichgewichtsprobleme gelöst werden können. Für die Modellierung der Risikopräferenz in der Klasse von risikoaversen Nash Gleichgewichtsprobleme verwenden wir kohärente Risikomaße. Da kohärente Risikomaße im Allgemeinen nicht glatt sind, ist das resultierende PDE-beschränkte Nash Gleichgewichtsproblem ebenfalls nicht glatt. Daher glätten wir die kohärenten Risikomaße mit Hilfe einer Epi-Regularisierungstechnik. Sowohl für das ursprüngliche Nash Gleichgewichtsproblem als auch für die geglätteten parameterabhängigen Nash Gleichgewichtsprobleme wird die Existenz von Nash Gleichgewichten gezeigt, sowie Optimalitätsbedingungen hergeleitet. Wir liefern wertvolle Resultate dafür, dass dieser Glättungsansatz sich für die Entwicklung eines numerischen Verfahren eignet, indem wir beweisen können, dass sowohl eine Folge von stationären Punkten als auch eine Folge von Nash Gleichgewichten des epi-regularisierten Problems eine schwach konvergente Teilfolge hat, deren Grenzwert ein Nash Gleichgewicht des ursprünglichen Problems ist.In this paper, we analyze PDE-constrained equilibrium problems under uncertainty. In detail, we discuss a class of risk-neutral generalized Nash equilibrium problems and a class of risk-averse Nash equilibrium problems. For both, the risk-neutral PDE-constrained optimization problems with pointwise state constraints and the risk-neutral generalized Nash equilibrium problems, the existence of solutions and Nash equilibria, respectively, is proved and optimality conditions are derived. The consideration of inequality conditions on the stochastic state leads in both cases to complications in the derivation of the Lagrange multipliers. Only by higher regularity of the stochastic state we can resort to the existing optimality theory for convex optimization problems. The low regularity of the Lagrange multiplier also poses a major challenge for the numerical solvability of these problems. We lay the foundation for a successful numerical treatment of risk-neutral Nash equilibrium problems using Moreau-Yosida regularization by showing that this regularization approach is consistent. The Moreau-Yosida regularization yields a sequence of parameter-dependent Nash equilibrium problems and the boundary transition in the smoothing parameter shows that the stationary points of the regularized problem converge weakly against a generalized Nash equilibrium of the original problem. Thus, the theory suggests that a numerical method can be built on the Moreau-Yosida regularization. Based on this, algorithms are proposed to show how to solve risk-neutral PDE-constrained optimization problems with pointwise state bounds and risk-neutral PDE-constrained generalized Nash equilibrium problems. I n order to model risk preference in the class of risk-averse Nash equilibrium problems, we use coherent risk measures. Since coherent risk measures are generally not smooth, the resulting PDE-constrained Nash equilibrium problem is also not smooth. Therefore, we smooth the coherent risk measures using an epi-regularization technique. For both the original Nash equilibrium problem and the smoothed parameter-dependent Nash equilibrium problems, we show the existence of Nash equilibria, and derive optimality conditions. We provide valuable results for making this smoothing approach suitable for the development of a numerical method by proving that both, a sequence of stationary points and a sequence of Nash equilibria of the epi-regularized problem, have a weakly convergent subsequence whose limit is a Nash equilibrium of the original problem