86 research outputs found
Recommended from our members
Simultaneously encoding movement and sEMG-based stiffness for robotic skill learning
Transferring human stiffness regulation strategies to robots enables them to effectively and efficiently acquire adaptive impedance control policies to deal with uncertainties during the accomplishment of physical contact tasks in an unstructured environment. In this work, we develop such a physical human-robot interaction (pHRI) system which allows robots to learn variable impedance skills from human demonstrations. Specifically, the biological signals, i.e., surface electromyography (sEMG) are utilized for the extraction of human arm stiffness features during the task demonstration. The estimated human arm stiffness is then mapped into a robot impedance controller. The dynamics of both movement and stiffness are simultaneously modeled by using a model combining the hidden semi-Markov model (HSMM) and the Gaussian mixture regression (GMR). More importantly, the correlation between the movement information and the stiffness information is encoded in a systematic manner. This approach enables capturing uncertainties over time and space and allows the robot to satisfy both position and stiffness requirements in a task with modulation of the impedance controller. The experimental study validated the proposed approach
Audio Visual Language Maps for Robot Navigation
While interacting in the world is a multi-sensory experience, many robots
continue to predominantly rely on visual perception to map and navigate in
their environments. In this work, we propose Audio-Visual-Language Maps
(AVLMaps), a unified 3D spatial map representation for storing cross-modal
information from audio, visual, and language cues. AVLMaps integrate the
open-vocabulary capabilities of multimodal foundation models pre-trained on
Internet-scale data by fusing their features into a centralized 3D voxel grid.
In the context of navigation, we show that AVLMaps enable robot systems to
index goals in the map based on multimodal queries, e.g., textual descriptions,
images, or audio snippets of landmarks. In particular, the addition of audio
information enables robots to more reliably disambiguate goal locations.
Extensive experiments in simulation show that AVLMaps enable zero-shot
multimodal goal navigation from multimodal prompts and provide 50% better
recall in ambiguous scenarios. These capabilities extend to mobile robots in
the real world - navigating to landmarks referring to visual, audio, and
spatial concepts. Videos and code are available at: https://avlmaps.github.io.Comment: Project page: https://avlmaps.github.io
- …