245 research outputs found
Combinatorial Bayesian Optimization using the Graph Cartesian Product
This paper focuses on Bayesian Optimization (BO) for objectives on
combinatorial search spaces, including ordinal and categorical variables.
Despite the abundance of potential applications of Combinatorial BO, including
chipset configuration search and neural architecture search, only a handful of
methods have been proposed. We introduce COMBO, a new Gaussian Process (GP) BO.
COMBO quantifies "smoothness" of functions on combinatorial search spaces by
utilizing a combinatorial graph. The vertex set of the combinatorial graph
consists of all possible joint assignments of the variables, while edges are
constructed using the graph Cartesian product of the sub-graphs that represent
the individual variables. On this combinatorial graph, we propose an ARD
diffusion kernel with which the GP is able to model high-order interactions
between variables leading to better performance. Moreover, using the Horseshoe
prior for the scale parameter in the ARD diffusion kernel results in an
effective variable selection procedure, making COMBO suitable for high
dimensional problems. Computationally, in COMBO the graph Cartesian product
allows the Graph Fourier Transform calculation to scale linearly instead of
exponentially. We validate COMBO in a wide array of realistic benchmarks,
including weighted maximum satisfiability problems and neural architecture
search. COMBO outperforms consistently the latest state-of-the-art while
maintaining computational and statistical efficiency.Comment: Accepted to NeurIPS 2019, code: https://github.com/QUVA-Lab/COMB
Belief-space Planning for Active Visual SLAM in Underwater Environments.
Autonomous mobile robots operating in a priori unknown environments must be able to integrate path planning with simultaneous localization and mapping (SLAM) in order to perform tasks like exploration, search and rescue, inspection, reconnaissance, target-tracking, and others. This level of autonomy is especially difficult in underwater environments, where GPS is unavailable, communication is limited, and environment features may be sparsely- distributed. In these situations, the path taken by the robot can drastically affect the performance of SLAM, so the robot must plan and act intelligently and efficiently to ensure successful task completion.
This document proposes novel research in belief-space planning for active visual SLAM in underwater environments. Our motivating application is ship hull inspection with an autonomous underwater robot. We design a Gaussian belief-space planning formulation that accounts for the randomness of the loop-closure measurements in visual SLAM and serves as the mathematical foundation for the research in this thesis. Combining this planning formulation with sampling-based techniques, we efficiently search for loop-closure actions throughout the environment and present a two-step approach for selecting revisit actions that results in an opportunistic active SLAM framework. The proposed active SLAM method is tested in hybrid simulations and real-world field trials of an underwater robot performing inspections of a physical modeling basin and a U.S. Coast Guard cutter.
To reduce computational load, we present research into efficient planning by compressing the representation and examining the structure of the underlying SLAM system. We propose the use of graph sparsification methods online to reduce complexity by planning with an approximate distribution that represents the original, full pose graph. We also propose the use of the Bayes tree data structure—first introduced for fast inference in SLAM—to perform efficient incremental updates when evaluating candidate plans that are similar. As a final contribution, we design risk-averse objective functions that account for the randomness within our planning formulation. We show that this aversion to uncertainty in the posterior belief leads to desirable and intuitive behavior within active SLAM.PhDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/133303/1/schaves_1.pd
Scalable Loss-calibrated Bayesian Decision Theory and Preference Learning
Bayesian decision theory provides a framework for optimal action
selection
under uncertainty given a utility function over actions and
world
states and a distribution over world states. The application of
Bayesian
decision theory in practice is often limited by two problems:
(1)
in application domains such as recommendation, the true utility
function
of a user is a priori unknown and must be learned from user
interactions; and (2) computing expected utilities under complex
state
distributions and (potentially uncertain) utility functions is
often
computationally expensive and requires tractable approximations.
In this thesis, we aim to address both of these problems. For
(1),
we take a Bayesian non-parametric approach to utility function
modeling
and learning. In our first contribution, we exploit community
structure
prevalent in collective user preferences using a Dirichlet
Process
mixture of Gaussian Processes (GPs). In our second contribution,
we
take the underlying GP preference model of the first
contribution
and show how to jointly address both (1) and (2) by sparsifying
the
GP model in order to preserve optimal decisions while ensuring
tractable
expected utility computations. In our third and final
contribution,
we directly address (2) in a Monte Carlo framework by deriving
an
optimal loss-calibrated importance sampling distribution and
show
how it can be extended to uncertain utility representations
developed
in the previous contributions.
Our empirical evaluations in various applications including
multiple preference learning problems using synthetic and real
user
data and robotics decision-making scenarios derived from actual
occupancy
grid maps demonstrate the effectiveness of the theoretical
foundations laid in this thesis and pave the way for future
advances
that address important practical problems at the intersection of
Bayesian
decision theory and scalable machine learning
Large-area visually augmented navigation for autonomous underwater vehicles
Submitted to the Joint Program in Applied Ocean Science & Engineering
in partial fulfillment of the requirements for the degree of Doctor of Philosophy
at the Massachusetts Institute of Technology
and the Woods Hole Oceanographic Institution
June 2005This thesis describes a vision-based, large-area, simultaneous localization and mapping (SLAM) algorithm that respects the low-overlap imagery constraints typical of autonomous underwater vehicles (AUVs) while exploiting the inertial sensor information that is routinely available on such platforms. We adopt a systems-level approach exploiting the complementary aspects of inertial sensing and visual perception from a calibrated pose-instrumented platform. This systems-level strategy yields a robust solution to underwater imaging that
overcomes many of the unique challenges of a marine environment (e.g., unstructured terrain, low-overlap imagery, moving light source). Our large-area SLAM algorithm recursively incorporates relative-pose constraints using a view-based representation that exploits exact sparsity in the Gaussian canonical form. This sparsity allows for efficient O(n) update complexity in the number of images composing the view-based map by utilizing recent multilevel relaxation techniques. We show that our algorithmic formulation is inherently sparse unlike other feature-based canonical SLAM algorithms, which impose sparseness via pruning approximations. In particular, we investigate
the sparsification methodology employed by sparse extended information filters (SEIFs)
and offer new insight as to why, and how, its approximation can lead to inconsistencies in
the estimated state errors. Lastly, we present a novel algorithm for efficiently extracting consistent marginal covariances useful for data association from the information matrix. In summary, this thesis advances the current state-of-the-art in underwater visual navigation by demonstrating end-to-end automatic processing of the largest visually navigated dataset to date using data collected from a survey of the RMS Titanic (path length over 3 km and 3100 m2 of mapped area). This accomplishment embodies the summed contributions of this thesis to several current SLAM research issues including scalability, 6 degree of
freedom motion, unstructured environments, and visual perception.This work was funded in part by the CenSSIS ERC of the National Science Foundation
under grant EEC-9986821, in part by the Woods Hole Oceanographic Institution through a
grant from the Penzance Foundation, and in part by a NDSEG Fellowship awarded through
the Department of Defense
Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
On top of machine learning models, uncertainty quantification (UQ) functions
as an essential layer of safety assurance that could lead to more principled
decision making by enabling sound risk assessment and management. The safety
and reliability improvement of ML models empowered by UQ has the potential to
significantly facilitate the broad adoption of ML solutions in high-stakes
decision settings, such as healthcare, manufacturing, and aviation, to name a
few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods
for ML models with a particular focus on neural networks and the applications
of these UQ methods in tackling engineering design as well as prognostics and
health management problems. Toward this goal, we start with a comprehensive
classification of uncertainty types, sources, and causes pertaining to UQ of ML
models. Next, we provide a tutorial-style description of several
state-of-the-art UQ methods: Gaussian process regression, Bayesian neural
network, neural network ensemble, and deterministic UQ methods focusing on
spectral-normalized neural Gaussian process. Established upon the mathematical
formulations, we subsequently examine the soundness of these UQ methods
quantitatively and qualitatively (by a toy regression example) to examine their
strengths and shortcomings from different dimensions. Then, we review
quantitative metrics commonly used to assess the quality of predictive
uncertainty in classification and regression problems. Afterward, we discuss
the increasingly important role of UQ of ML models in solving challenging
problems in engineering design and health prognostics. Two case studies with
source codes available on GitHub are used to demonstrate these UQ methods and
compare their performance in the life prediction of lithium-ion batteries at
the early stage and the remaining useful life prediction of turbofan engines
- …