Modularity through Attention: Efficient Training and Transfer of Language-Conditioned Policies for Robot Manipulation

Language-conditioned policies allow robots to interpret and execute human instructions. Learning such policies requires a substantial investment with regards to time and compute resources. Still, the resulting controllers are highly device-specific and cannot easily be transferred to a robot with different morphology, capability, appearance or dynamics. In this paper, we propose a sample-efficient approach for training language-conditioned manipulation policies that allows for rapid transfer across different types of robots. By introducing a novel method, namely Hierarchical Modularity, and adopting supervised attention across multiple sub-modules, we bridge the divide between modular and end-to-end learning and enable the reuse of functional building blocks. In both simulated and real world robot manipulation experiments, we demonstrate that our method outperforms the current state-of-the-art methods and can transfer policies across 4 different robots in a sample-efficient manner. Finally, we show that the functionality of learned sub-modules is maintained beyond the training process and can be used to introspect the robot decision-making process. Code is available at https://github.com/ir-lab/ModAttn.Comment: 2022 Conference on Robot Learning (CoRL

Clone Swarms: Learning to Predict and Control Multi-Robot Systems by Imitation

In this paper, we propose SwarmNet -- a neural network architecture that can learn to predict and imitate the behavior of an observed swarm of agents in a centralized manner. Tested on artificially generated swarm motion data, the network achieves high levels of prediction accuracy and imitation authenticity. We compare our model to previous approaches for modelling interaction systems and show how modifying components of other models gradually approaches the performance of ours. Finally, we also discuss an extension of SwarmNet that can deal with nondeterministic, noisy, and uncertain environments, as often found in robotics applications

Group SELFIES: A Robust Fragment-Based Molecular String Representation

We introduce Group SELFIES, a molecular string representation that leverages group tokens to represent functional groups or entire substructures while maintaining chemical robustness guarantees. Molecular string representations, such as SMILES and SELFIES, serve as the basis for molecular generation and optimization in chemical language models, deep generative models, and evolutionary methods. While SMILES and SELFIES leverage atomic representations, Group SELFIES builds on top of the chemical robustness guarantees of SELFIES by enabling group tokens, thereby creating additional flexibility to the representation. Moreover, the group tokens in Group SELFIES can take advantage of inductive biases of molecular fragments that capture meaningful chemical motifs. The advantages of capturing chemical motifs and flexibility are demonstrated in our experiments, which show that Group SELFIES improves distribution learning of common molecular datasets. Further experiments also show that random sampling of Group SELFIES strings improves the quality of generated molecules compared to regular SELFIES strings. Our open-source implementation of Group SELFIES is available online, which we hope will aid future research in molecular generation and optimization.Comment: 11 pages + references and appendi

Texturas: simulacion visual de nubes

Con este trabajo pretendemos iniciarnos en el estudio de representación de texturas para obtener imágenes visualmente realísticas de fenómenos gaseosos en principio, para luego continuar con fenómenos sólidos. La representación de escenas que contienen nubes, humo, efectos de dispersión atmosférica, y otros fenómenos gaseosos ha recibido un extenso tratamiento en la literatura de computación gráfica. Muchos artículos lo resuelven principalpente con efectos de dispersión atmosférica, mientras otros cubren la iluminación de estos fenómenos gaseosos en detalle. Otra forma de representar estos fenómenos es modelar la geometría de estos gases. Kajiya usa un modelo basado en la física, Voss usa fractales, Max usa campos elevados, y Ebert and Parent muestra resultados reales que pueden obtenerse usando funciones basadas en flujo de turbulencia para modelar la densidad de una variedad de gases. Las nubes presentan serios problemas para las técnicas normales de generación de imagen mediante computadoras, porque no tienen superficies y contornos bien definidos. Además, las nubes contienen diversos grados de translucidez y su estructura amorfa puede cambiar con el tiempo. En este trabajo se intenta implementar el método de simulación de nubes presentado por Gardner usando superficies curvas y planas, cuyas superficies translúcidas y sombreadas son modeladas por funciones matemáticas de textura. Gardner desarrolló una función de textura naturalística basada en una serie de Fourier modificada.Eje: Ingeniería del software. Computación gráfica y visualizaciónRed de Universidades con Carreras en Informática (RedUNCI