Hierarchies of Planning and Reinforcement Learning for Robot Navigation

Solving robotic navigation tasks via reinforcement learning (RL) is challenging due to their sparse reward and long decision horizon nature. However, in many navigation tasks, high-level (HL) task representations, like a rough floor plan, are available. Previous work has demonstrated efficient learning by hierarchal approaches consisting of path planning in the HL representation and using sub-goals derived from the plan to guide the RL policy in the source task. However, these approaches usually neglect the complex dynamics and sub-optimal sub-goal-reaching capabilities of the robot during planning. This work overcomes these limitations by proposing a novel hierarchical framework that utilizes a trainable planning policy for the HL representation. Thereby robot capabilities and environment conditions can be learned utilizing collected rollout data. We specifically introduce a planning policy based on value iteration with a learned transition model (VI-RL). In simulated robotic navigation tasks, VI-RL results in consistent strong improvement over vanilla RL, is on par with vanilla hierarchal RL on single layouts but more broadly applicable to multiple layouts, and is on par with trainable HL path planning baselines except for a parking task with difficult non-holonomic dynamics where it shows marked improvements.Comment: 7 pages, 5 figures, 2021 IEEE International Conference on Robotics and Automation (ICRA), v2: DOI number adde

The onset of photoionization in Sakurai's Object (V4334 Sgr)

We investigate the reheating of the very late thermal pulse (VLTP) object V4334 Sgr (Sakurai's Object) using radio observations from the Very Large Array, and optical spectra obtained with the Very Large Telescope. We find a sudden rise of the radio flux at 5 and 8 GHz - from <= 90 micro-Jy and 80 +/- 30 micro-Jy in February 2005 to 320 micro-Jy and 280 micro-Jy in June 2006. Optical line emission is also evolving, but the emission lines are fading. The optical line emission and early radio flux are attributed to a fast shock (and not photoionization as was reported earlier) which occurred around 1998. The fading is due to post-shock cooling and recombination. The recent rapid increase in radio flux is evidence for the onset of photoionization of carbon starting around 2005. The current results indicate an increase in the stellar temperature to 12 kK in 2006. The mass ejected in the VLTP eruption is M_ej >= 1e-4 Msol, but could be as high as 1e-2 Msol, depending mainly on the distance and the clumping factor of the outflow. We derive a distance between 1.8 and 5 kpc. A high mass loss could expose the helium layer and yield abundances compatible with those of [WC] and PG1159 stars.Comment: 4 pages, 2 figures; accepted for publication in A&A letter

The very fast evolution of Sakurai's object

V4334 Sgr (a.k.a. Sakurai's object) is the central star of an old planetary nebula that underwent a very late thermal pulse a few years before its discovery in 1996. We have been monitoring the evolution of the optical emission line spectrum since 2001. The goal is to improve the evolutionary models by constraining them with the temporal evolution of the central star temperature. In addition the high resolution spectral observations obtained by X-shooter and ALMA show the temporal evolution of the different morphological components.Comment: 2 pages, 2 figures to appear in the Proceedings of the IAU Symp. 323: "Planetary nebulae: Multi-wavelength probes of stellar and galactic evolution". Eds. X.-W. Liu, L. Stanghellini and A. Karaka