Model Estimation Within Planning and Learning

Risk and reward are fundamental concepts in the cooperative control of unmanned systems. In this research, we focus on developing a constructive relationship between cooperative planning and learning algorithms to mitigate the learning risk, while boosting system (planner & learner) asymptotic performance and guaranteeing the safety of agent behavior. Our framework is an instance of the intelligent cooperative control architecture (iCCA) where the learner incrementally improves on the output of a baseline planner through interaction and constrained exploration. We extend previous work by extracting the embedded parameterized transition model from within the cooperative planner and making it adaptable and accessible to all iCCA modules. We empirically demonstrate the advantage of using an adaptive model over a static model and pure learning approaches in an example GridWorld problem and a UAV mission planning scenario with 200 million possibilities. Finally we discuss two extensions to our approach to handle cases where the true model can not be captured exactly through the presumed functional form.United States. Air Force Office of Scientific Research (FA9550-09-1-0522)Natural Sciences and Engineering Research Council of CanadaUSAF (FA9550-09-1-0522

Nonparametric Bayesian behavior modeling

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2008.Includes bibliographical references (p. 91-94).As autonomous robots are increasingly used in complex, dynamic environments, it is crucial that the dynamic elements are modeled accurately. However, it is often difficult to generate good models due to either a lack of domain understanding or the domain being intractably large. In many domains, even defining the size of the model can be a challenge. While methods exist to cluster data of dynamic agents into common motion patterns, or "behaviors," assumptions of the number of expected behaviors must be made. This assumption can cause clustering processes to under-fit or over-fit the training data. In a poorly understood domain, knowing the number of expected behaviors a priori is unrealistic and in an extremely large domain, correctly fitting the training data is difficult. To overcome these obstacles, this thesis takes a Bayesian approach and applies a Dirichlet process (DP) prior over behaviors, which uses experience to reduce the likelihood of over-fitting or under-fitting the model complexity. Additionally, the DP maintains a probability mass associated with a novel behavior and can address countably infinite behaviors. This learning technique is applied to modeling agents driving in an urban setting. The learned DP-based driver behavior model is first demonstrated on a simulated city. Building on successful simulation results, the methodology is applied to GPS data of taxis driving around Boston. Accurate prediction of future vehicle behavior from the model is shown in both domains.by Joshua Mason Joseph.S.M

A Bayesian nonparametric approach to modeling battery health

The batteries of many consumer products are both a substantial portion of the product's cost and commonly a first point of failure. Accurately predicting remaining battery life can lower costs by reducing unnecessary battery replacements. Unfortunately, battery dynamics are extremely complex, and we often lack the domain knowledge required to construct a model by hand. In this work, we take a data-driven approach and aim to learn a model of battery time-to-death from training data. Using a Dirichlet process prior over mixture weights, we learn an infinite mixture model for battery health. The Bayesian aspect of our model helps to avoid over-fitting while the nonparametric nature of the model allows the data to control the size of the model, preventing under-fitting. We demonstrate our model's effectiveness by making time-to-death predictions using real data from nickel-metal hydride battery packs.United States. Army Research Office (Nostra Project STTR W911NF-08-C-0066)iRobo

Mobile Agent Trajectory Prediction using Bayesian Nonparametric Reachability Trees

This paper presents an efficient trajectory prediction algorithm that has been developed to improve the performance of future collision avoidance and detection systems. The main idea is to embed the inferred intention information of surrounding agents into their estimated reachability sets to obtain a probabilistic description of their future paths. More specifically, the proposed approach combines the recently developed RRT-Reach algorithm and mixtures of Gaussian Processes. RRT-Reach was introduced by the authors as an extension of the closed-loop rapidly-exploring random tree (CL-RRT) algorithm to compute reachable sets of moving objects in real-time. A mixture of Gaussian processes (GP) is a flexible nonparametric Bayesian model used to represent a distribution over trajectories and have been previously demonstrated by the authors in a UAV interception and tracking of ground vehicles planning scheme. The mixture is trained using typical maneuvers learned from statistical data, and RRT-Reach utilizes samples from the GP to grow probabilistically weighted feasible paths of the surrounding vehicles. The resulting approach, denoted as RR-GP, has RRTReach's benefits of computing trajectories that are dynamically feasible by construction, therefore efficiently approximating the reachability set of surrounding vehicles following typical patterns. RRT-GP also features the GP mixture's benefits of providing a probabilistic weighting on the feasible trajectories produced by RRTReach, allowing our system to systematically weight trajectories by their likelihood. A demonstrative example on a car-like vehicle illustrates the advantages of the RR-GP approach by comparing it to two other GP-based algorithms. © 2011 by Professor Jonathan P. How, Massachusetts Institute of Technology. Published by the American Institute of Aeronautics and Astronautics, Inc

Reinforcement learning with misspecified model classes

Real-world robots commonly have to act in complex, poorly understood environments where the true world dynamics are unknown. To compensate for the unknown world dynamics, we often provide a class of models to a learner so it may select a model, typically using a minimum prediction error metric over a set of training data. Often in real-world domains the model class is unable to capture the true dynamics, due to either limited domain knowledge or a desire to use a small model. In these cases we call the model class misspecified, and an unfortunate consequence of misspecification is that even with unlimited data and computation there is no guarantee the model with minimum prediction error leads to the best performing policy. In this work, our approach improves upon the standard maximum likelihood model selection metric by explicitly selecting the model which achieves the highest expected reward, rather than the most likely model. We present an algorithm for which the highest performing model from the model class is guaranteed to be found given unlimited data and computation. Empirically, we demonstrate that our algorithm is often superior to the maximum likelihood learner in a batch learning setting for two common RL benchmark problems and a third real-world system, the hydrodynamic cart-pole, a domain whose complex dynamics cannot be known exactly.United States. Office of Naval Research. Multidisciplinary University Research Initiative (N00014-11-1-0688

Characterizing divergence through three adjacent Australian avian transition zones

Aim The diversification of the Australian avifauna has been greatly influenced by prominent historical and modern barriers to dispersal. The aims of this study were to characterize the patterns of divergence in population pairs of meliphagoid birds across adjacent transition zones and characterize how well morphometric divergence, habitat association and taxonomic or species ranking can predict genetic divergence. Location: Northern Queensland, Australia. Methods Genetic divergence between parental populations on either side of the three biogeographical barriers corresponding to three clusters of hybrid zones was characterized in 27 species complexes of meliphagoid birds using one mitochondrial, 23 autosomal and 12 Z chromosome loci collected from a sequence capture system. Within each species, we characterized morphometric divergence using wing, bill and tail measurements from museum samples. Lastly, we evaluated the predictive power of these morphometric measurements on genetic divergence. Results Population pairs on either side of a transition zone depict a wide range of genomic and morphometric divergence. For some systems, species exhibiting morphometric divergence show little to no genomic divergence, while, conversely, other species exhibiting little to no morphometric divergence may show clear genomic divergence. Species rank is shown to be the strongest predictor for genetic divergence, habitat is the next strongest predictor and morphometric divergence is the weakest predictor. Main conclusions The variation in divergence levels of population pairs affirms that transition zones are ideal natural experiments to study the speciation process. In particular, transition zones allow understanding of how genomic divergence accumulates during speciation. Additionally, standing species rank classifications mostly prove to be robust after genetic characterization. Lastly, the discordance between morphometric and genetic divergence suggests other non-morphometric phenotypic traits used to designate species rank, such as song or plumage, may play a more important role in predicting genetic divergence

The TESS-Keck Survey II: An Ultra-Short Period Rocky Planet and its Siblings Transiting the Galactic Thick-Disk Star TOI-561

We report the discovery of TOI-561, a multi-planet system in the galactic thick disk that contains a rocky, ultra-short period planet (USP). This bright ($V=10.2$) star hosts three small transiting planets identified in photometry from the NASA TESS mission: TOI-561 b (TOI-561.02, P=0.44 days, $R_b = 1.45\pm0.11\,R_\oplus$), c (TOI-561.01, P=10.8 days, $R_c=2.90\pm0.13\,R_\oplus$), and d (TOI-561.03, P=16.3 days, $R_d=2.32\pm0.16\,R_\oplus$). The star is chemically ([Fe/H]$=-0.41\pm0.05$, [$\alpha$/H]$=+0.23\pm0.05$) and kinematically consistent with the galactic thick disk population, making TOI-561 one of the oldest ($10\pm3\,$Gyr) and most metal-poor planetary systems discovered yet. We dynamically confirm planets b and c with radial velocities from the W. M. Keck Observatory High Resolution Echelle Spectrometer. Planet b has a mass and density of $3.2\pm0.8\,M_\oplus$ and $5.5^{+2.0}_{-1.6}\,$g$\,$cm$^{-3}$, consistent with a rocky composition. Its lower-than-average density is consistent with an iron-poor composition, although an Earth-like iron-to-silicates ratio is not ruled out. Planet c is $7.0\pm2.3\,M_\oplus$ and $1.6\pm0.6\,$g$\,$cm$^{-3}$, consistent with an interior rocky core overlaid with a low-mass volatile envelope. Several attributes of the photometry for planet d (which we did not detect dynamically) complicate the analysis, but we vet the planet with high-contrast imaging, ground-based photometric follow-up and radial velocities. TOI-561 b is the first rocky world around a galactic thick-disk star confirmed with radial velocities and one of the best rocky planets for thermal emission studies.Comment: Accepted at The Astronomical Journal; 25 pages, 10 figure