Audio-based Roughness Sensing and Tactile Feedback for Haptic Perception in Telepresence

Haptic perception is highly important for immersive teleoperation of robots, especially for accomplishing manipulation tasks. We propose a low-cost haptic sensing and rendering system, which is capable of detecting and displaying surface roughness. As the robot fingertip moves across a surface of interest, two microphones capture sound coupled directly through the fingertip and through the air, respectively. A learning-based detector system analyzes the data in real time and gives roughness estimates with both high temporal resolution and low latency. Finally, an audio-based vibrational actuator displays the result to the human operator. We demonstrate the effectiveness of our system through lab experiments and our winning entry in the ANA Avatar XPRIZE competition finals, where briefly trained judges solved a roughness-based selection task even without additional vision feedback. We publish our dataset used for training and evaluation together with our trained models to enable reproducibility of results.Comment: IEEE International Conference on Systems, Man, and Cybernetics (SMC), Honolulu, Hawaii, USA, October 202

Robust Immersive Telepresence and Mobile Telemanipulation: NimbRo wins ANA Avatar XPRIZE Finals

Robotic avatar systems promise to bridge distances and reduce the need for travel. We present the updated NimbRo avatar system, winner of the $5M grand prize at the international ANA Avatar XPRIZE competition, which required participants to build intuitive and immersive robotic telepresence systems that could be operated by briefly trained operators. We describe key improvements for the finals, compared to the system used in the semifinals: To operate without a power- and communications tether, we integrated a battery and a robust redundant wireless communication system. Video and audio data are compressed using low-latency HEVC and Opus codecs. We propose a new locomotion control device with tunable resistance force. To increase flexibility, the robot's upper-body height can be adjusted by the operator. We describe essential monitoring and robustness tools which enabled the success at the competition. Finally, we analyze our performance at the competition finals and discuss lessons learned.Comment: M. Schwarz and C. Lenz contributed equall

Data augmentation for training a neural network for image reconstruction in MPI

Neural networks need to be trained with immense datasets for successful image reconstruction. Acquiring these datasets may be a difficult task, especially in medical imaging. Data augmentation techniques are used to enlarge an available dataset by synthesizing new data. In this work, it is proposed to use the single measurements of a system matrix measurement in magnetic particle imaging for training a neural network for image reconstruction. Before training, mixup augmentation is used to create linear combinations of the single measurements and thus, enlarging the training dataset. Image reconstruction results using neural networks trained with an augmented system matrix are compared to images that have been reconstructed using the conventional system-matrix-based approach