Autonomous 3D Exploration of Large Structures Using an UAV Equipped with a 2D LIDAR

This paper addressed the challenge of exploring large, unknown, and unstructured industrial environments with an unmanned aerial vehicle (UAV). The resulting system combined well-known components and techniques with a new manoeuvre to use a low-cost 2D laser to measure a 3D structure. Our approach combined frontier-based exploration, the Lazy Theta* path planner, and a flyby sampling manoeuvre to create a 3D map of large scenarios. One of the novelties of our system is that all the algorithms relied on the multi-resolution of the octomap for the world representation. We used a Hardware-in-the-Loop (HitL) simulation environment to collect accurate measurements of the capability of the open-source system to run online and on-board the UAV in real-time. Our approach is compared to different reference heuristics under this simulation environment showing better performance in regards to the amount of explored space. With the proposed approach, the UAV is able to explore 93% of the search space under 30 min, generating a path without repetition that adjusts to the occupied space covering indoor locations, irregular structures, and suspended obstaclesUnión Europea Marie Sklodowska-Curie 64215Unión Europea MULTIDRONE (H2020-ICT-731667)Uniión Europea HYFLIERS (H2020-ICT-779411

Understanding Model-Based Reinforcement Learning and its Application in Safe Reinforcement Learning

Model-based reinforcement learning algorithms have been shown to achieve successful results on various continuous control benchmarks, but the understanding of model-based methods is limited. We try to interpret how model-based method works through novel experiments on state-of-the-art algorithms with an emphasis on the model learning part. We evaluate the role of the model learning in policy optimization and propose methods to learn a more accurate model. With a better understanding of model-based reinforcement learning, we then apply model-based methods to solve safe reinforcement learning (RL) problems with near-zero violation of hard constraints throughout training. Drawing an analogy with how humans and animals learn to perform safe actions, we break down the safe RL problem into three stages. First, we train agents in a constraint-free environment to learn a performant policy for reaching high rewards, and simultaneously learn a model of the dynamics. Second, we use model-based methods to plan safe actions and train a safeguarding policy from these actions through imitation. Finally, we propose a factored framework to train an overall policy that mixes the performant policy and the safeguarding policy. This three-step curriculum ensures near-zero violation of safety constraints at all times. As an advantage of model-based method, the sample complexity required at the second and third steps of the process is significantly lower than model-free methods and can enable online safe learning. We demonstrate the effectiveness of our methods in various continuous control problems and analyze the advantages over state-of-the-art approaches

MHD Simulations of AGN Jets in a Dynamic Galaxy Cluster Medium

We present a pair of 3-d magnetohydrodynamical simulations of intermittent jets from a central active galactic nucleus (AGN) in a galaxy cluster extracted from a high resolution cosmological simulation. The selected cluster was chosen as an apparently relatively relaxed system, not having undergone a major merger in almost 7 Gyr. Despite this characterization and history, the intra-cluster medium (ICM) contains quite active "weather". We explore the effects of this ICM weather on the morphological evolution of the AGN jets and lobes. The orientation of the jets is different in the two simulations so that they probe different aspects of the ICM structure and dynamics. We find that even for this cluster that can be characterized as relaxed by an observational standard, the large-scale, bulk ICM motions can significantly distort the jets and lobes. Synthetic X-ray observations of the simulations show that the jets produce complex cavity systems, while synthetic radio observations reveal bending of the jets and lobes similar to wide-angle tail (WAT) radio sources. The jets are cycled on and off with a 26 Myr period using a 50% duty cycle. This leads to morphological features similar to those in "double-double" radio galaxies. While the jet and ICM magnetic fields are generally too weak in the simulations to play a major role in the dynamics, Maxwell stresses can still become locally significant.Comment: 20 pages, 14 figures, accepted for publication in the Astrophysical Journa

The Formation of the First Star in the Universe

We describe results from a fully self-consistent three dimensional hydrodynamical simulation of the formation of one of the first stars in the Universe. Dark matter dominated pre-galactic objects form because of gravitational instability from small initidal density perturbations. As they assemble via hierarchical merging, primordial gas cools through ro-vibrational lines of hydrogen molecules and sinks to the center of the dark matter potential well. The high redshift analog of a molecular cloud is formed. When the dense, central parts of the cold gas cloud become self-gravitating, a dense core of approximately 100 solar mass undergoes rapid contraction. At densities n>10^9 cm^-3 a one solar mass proto-stellar core becomes fully molecular due to three-body H_2 formation. Contrary to analytical expectations this process does not lead to renewed fragmentation and only one star is formed. The calculation is stopped when optical depth effects become important, leaving the final mass of the fully formed star somewhat uncertain. At this stage the protostar is acreting material very rapidly (~0.01 solar masses per year). Radiative feedback from the star will not only halt its growth but also inhibit the formation of other stars in the same pre-galactic object (at least until the first star ends its life, presumably as a supernova). We conclude that at most one massive (M >> 1 solar mass) metal free star forms per pre-galactic halo, consistent with recent abundance measurements of metal poor galactic halo stars.Comment: 8 pages, 5 figures. Science. Published in Science Express online on 11/15/2001. More information can be found at http://www.TomAbel.com/GB

Goal Set Inverse Optimal Control and Iterative Re-planning for Predicting Human Reaching Motions in Shared Workspaces

To enable safe and efficient human-robot collaboration in shared workspaces it is important for the robot to predict how a human will move when performing a task. While predicting human motion for tasks not known a priori is very challenging, we argue that single-arm reaching motions for known tasks in collaborative settings (which are especially relevant for manufacturing) are indeed predictable. Two hypotheses underlie our approach for predicting such motions: First, that the trajectory the human performs is optimal with respect to an unknown cost function, and second, that human adaptation to their partner's motion can be captured well through iterative re-planning with the above cost function. The key to our approach is thus to learn a cost function which "explains" the motion of the human. To do this, we gather example trajectories from pairs of participants performing a collaborative assembly task using motion capture. We then use Inverse Optimal Control to learn a cost function from these trajectories. Finally, we predict reaching motions from the human's current configuration to a task-space goal region by iteratively re-planning a trajectory using the learned cost function. Our planning algorithm is based on the trajectory optimizer STOMP, it plans for a 23 DoF human kinematic model and accounts for the presence of a moving collaborator and obstacles in the environment. Our results suggest that in most cases, our method outperforms baseline methods when predicting motions. We also show that our method outperforms baselines for predicting human motion when a human and a robot share the workspace.Comment: 12 pages, Accepted for publication IEEE Transaction on Robotics 201