Agent as Cerebrum, Controller as Cerebellum: Implementing an Embodied LMM-based Agent on Drones

In this study, we present a novel paradigm for industrial robotic embodied agents, encapsulating an 'agent as cerebrum, controller as cerebellum' architecture. Our approach harnesses the power of Large Multimodal Models (LMMs) within an agent framework known as AeroAgent, tailored for drone technology in industrial settings. To facilitate seamless integration with robotic systems, we introduce ROSchain, a bespoke linkage framework connecting LMM-based agents to the Robot Operating System (ROS). We report findings from extensive empirical research, including simulated experiments on the Airgen and real-world case study, particularly in individual search and rescue operations. The results demonstrate AeroAgent's superior performance in comparison to existing Deep Reinforcement Learning (DRL)-based agents, highlighting the advantages of the embodied LMM in complex, real-world scenarios.Comment: 17 pages, 12 figure

Towards an Interactive Humanoid Companion with Visual Tracking Modalities

The idea of robots acting as human companions is not a particularly new or original one. Since the notion of “robot ” was created, the idea of robots replacing humans in dangerous, dirty and dull activities has been inseparably tied with the fantasy of human-like robots being friends and existing side by side with humans. In 1989, Engelberger (Engelberger

HapticSnakes: multi-haptic feedback wearable robots for immersive virtual reality

Haptic feedback plays a large role in enhancing immersion and presence in VR. However, previous research and commercial products have limitations in terms of variety and locations of delivered feedbacks. To address these challenges, we present HapticSnakes, which are snake-like waist-worn robots that can deliver multiple types of feedback in various body locations, including taps-, gestures-, airflow-, brushing- and gripper-based feedbacks. We developed two robots, one is lightweight and suitable for taps and gestures, while the other is capable of multiple types of feedback. We presented a design space based on our implementations and conducted two evaluations. Since taps are versatile, easy to deliver and largely unexplored, our first evaluation focused on distinguishability of tap strengths and locations on the front and back torso. Participants had highest accuracy in distinguishing feedback on the uppermost regions and had superior overall accuracy in distinguishing feedback strengths over locations. Our second user study investigated HapticSnakes' ability to deliver multiple feedback types within VR experiences, as well as users' impressions of wearing our robots and receiving novel feedback in VR. The results indicate that participants had distinct preferences for feedbacks and were in favor of using our robots throughout. Based on the results of our evaluations, we extract design considerations and discuss research challenges and opportunities for developing multi-haptic feedback robots. - 2019, The Author(s).Open Access funding provided by the Qatar National Library. The presented work is supported in part through Program for Leading Graduate Schools, “Graduate Program for Embodiment Informatics” by Japan’s Ministry of Education, Culture, Sports, Science and Technology. We would also like to thank Mr. Thomas Höglund for his contribution to the mechanical design and control software of the HapticSnakes system.Scopu