Learning Temporally Extended Skills in Continuous Domains as Symbolic Actions for Planning

Problems which require both long-horizon planning and continuous control capabilities pose significant challenges to existing reinforcement learning agents. In this paper we introduce a novel hierarchical reinforcement learning agent which links temporally extended skills for continuous control with a forward model in a symbolic discrete abstraction of the environment's state for planning. We term our agent SEADS for Symbolic Effect-Aware Diverse Skills. We formulate an objective and corresponding algorithm which leads to unsupervised learning of a diverse set of skills through intrinsic motivation given a known state abstraction. The skills are jointly learned with the symbolic forward model which captures the effect of skill execution in the state abstraction. After training, we can leverage the skills as symbolic actions using the forward model for long-horizon planning and subsequently execute the plan using the learned continuous-action control skills. The proposed algorithm learns skills and forward models that can be used to solve complex tasks which require both continuous control and long-horizon planning capabilities with high success rate. It compares favorably with other flat and hierarchical reinforcement learning baseline agents and is successfully demonstrated with a real robot.Comment: Project website (including video) is available at https://seads.is.tue.mpg.de/. (v2) Accepted for publication at the 6th Conference on Robot Learning (CoRL) 2022, Auckland, New Zealand. (v3) Added details on checkpointing (S.8.1), with references on p.7, p.8, p.21 to clarify number of env. steps of reported result

Context-Conditional Navigation with a Learning-Based Terrain- and Robot-Aware Dynamics Model

In autonomous navigation settings, several quantities can be subject to variations. Terrain properties such as friction coefficients may vary over time depending on the location of the robot. Also, the dynamics of the robot may change due to, e.g., different payloads, changing the system's mass, or wear and tear, changing actuator gains or joint friction. An autonomous agent should thus be able to adapt to such variations. In this paper, we develop a novel probabilistic, terrain- and robot-aware forward dynamics model, termed TRADYN, which is able to adapt to the above-mentioned variations. It builds on recent advances in meta-learning forward dynamics models based on Neural Processes. We evaluate our method in a simulated 2D navigation setting with a unicycle-like robot and different terrain layouts with spatially varying friction coefficients. In our experiments, the proposed model exhibits lower prediction error for the task of long-horizon trajectory prediction, compared to non-adaptive ablation models. We also evaluate our model on the downstream task of navigation planning, which demonstrates improved performance in planning control-efficient paths by taking robot and terrain properties into account.Comment: \copyright 2023 IEEE. Accepted for publication in European Conference on Mobile Robots (ECMR), 2023. Updated copyright statemen

An approach to construct a three-dimensional isogeometric model from µ-CT scan data with an application to the bridge of a violin

We present an algorithm to build a ready to use isogeometric model from scan data gained by a µ-CT scan. Based on a three-dimensional multi-patch reference geometry, which includes the major topological features, we fit the outline, then the cross-section and finally the three-dimensional geometry. The key step is to fit the outline, where a non-linear least squares problem is solved with a Gauss-Newton approach presented by Borges and Pastva (2002). We extend this approach by a regularisation and a precise interpolation of selected data points. The resulting NURBS geometry is ready for applying isogeometric analysis tools for efficient numerical simulations. As a particular example we examine the scan data of a violin bridge and present the complete workflow from the µ-CT scan up to the numerical simulation based on isogeometric mortar methods. We illustrate the relevance of the constructed geometry with a vibro-acoustical application

Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts

In this paper, we present a method for table tennis ball trajectory filtering and prediction. Our gray-box approach builds on a physical model. At the same time, we use data to learn parameters of the dynamics model, of an extended Kalman filter, and of a neural model that infers the ball's initial condition. We demonstrate superior prediction performance of our approach over two black-box approaches, which are not supplied with physical prior knowledge. We demonstrate that initializing the spin from parameters of the ball launcher using a neural network drastically improves long-time prediction performance over estimating the spin purely from measured ball positions. An accurate prediction of the ball trajectory is crucial for successful returns. We therefore evaluate the return performance with a pneumatic artificial muscular robot and achieve a return rate of 29/30 (97.7%).Comment: Accepted for publication at the 5th Annual Conference on Learning for Dynamics and Control (L4DC) 2023. With supplementary materia

X-ray phase-contrast tomography with a compact laser-driven synchrotron source.

Between X-ray tubes and large-scale synchrotron sources, a large gap in performance exists with respect to the monochromaticity and brilliance of the X-ray beam. However, due to their size and cost, large-scale synchrotrons are not available for more routine applications in small and medium-sized academic or industrial laboratories. This gap could be closed by laser-driven compact synchrotron light sources (CLS), which use an infrared (IR) laser cavity in combination with a small electron storage ring. Hard X-rays are produced through the process of inverse Compton scattering upon the intersection of the electron bunch with the focused laser beam. The produced X-ray beam is intrinsically monochromatic and highly collimated. This makes a CLS well-suited for applications of more advanced--and more challenging--X-ray imaging approaches, such as X-ray multimodal tomography. Here we present, to our knowledge, the first results of a first successful demonstration experiment in which a monochromatic X-ray beam from a CLS was used for multimodal, i.e., phase-, dark-field, and attenuation-contrast, X-ray tomography. We show results from a fluid phantom with different liquids and a biomedical application example in the form of a multimodal CT scan of a small animal (mouse, ex vivo). The results highlight particularly that quantitative multimodal CT has become feasible with laser-driven CLS, and that the results outperform more conventional approaches

Heterogeneity of Graphite Lithiation in State‐of‐the‐Art Cylinder‐Type Li‐Ion Cells

The two‐dimensional lithium distribution in the graphite anode was non‐destructively probed by spatially resolved neutron diffraction for a batch consisting of 34 different cylinder‐type (18650) Li‐ion batteries in fully charged state. The uniformity of the lithium distribution was quantified and correlated to the cell specifications/electrochemistry and to intrinsic cell parameters like electrode thickness, position of current collectors, etc. which were obtained by X‐ray micro‐computed tomography. Non‐uniformities in the lithiation state of the anode from a constant plateau have been observed for the majority of the studied cells. Their location corresponds to the positions of current tabs connecting the electrode stripes and areas of incomplete electrode coating at the beginning and the end of the electrode stripes. Four commonly used schemes of current lid connection were identified. Each of them displays its own effect on the uniformity of the lithiation at the anode and, therefore, variation of the intrinsic state‐of‐charge distribution and, most probably, the ageing behavior of the electrodes

Experimental validation of image contrast correlation between ultra-small-angle X-ray scattering and grating-based dark-field imaging using a laser-driven compact X-ray source: Experimentelle Verifizierung des Zusammenhangs zwischen Röntgen-Kleinwinkelstreuung und gitter-basierter Röntgen-Dunkelfeldbildgebung unter Verwendung eines laser-getriebenen Kompaktsynchrotrons

X-ray phase and dark-field contrast have recently been the source of much attention in the field of X-ray imaging, as they both contribute new imaging signals based on physical principles that differ from conventional X-ray imaging. With a so-called Talbot grating interferometer, both phase-contrast and dark-field images are obtained simultaneously with the conventional attenuation-based X-ray image, providing three complementary image modalities that are intrinsically registered. Whereas the physical contrast mechanisms behind attenuation and phase contrast are well understood, a formalism to describe the dark-field signal is still in progress. In this article, we report on correlative experimental results obtained with a grating interferometer and with small-angle X-ray scattering. Furthermore, we use a proposed model to quantitatively describe the results, which could be of great importance for future clinical and biomedical applications of grating-based X-ray imagin