2,835 research outputs found
Early Turn-taking Prediction with Spiking Neural Networks for Human Robot Collaboration
Turn-taking is essential to the structure of human teamwork. Humans are
typically aware of team members' intention to keep or relinquish their turn
before a turn switch, where the responsibility of working on a shared task is
shifted. Future co-robots are also expected to provide such competence. To that
end, this paper proposes the Cognitive Turn-taking Model (CTTM), which
leverages cognitive models (i.e., Spiking Neural Network) to achieve early
turn-taking prediction. The CTTM framework can process multimodal human
communication cues (both implicit and explicit) and predict human turn-taking
intentions in an early stage. The proposed framework is tested on a simulated
surgical procedure, where a robotic scrub nurse predicts the surgeon's
turn-taking intention. It was found that the proposed CTTM framework
outperforms the state-of-the-art turn-taking prediction algorithms by a large
margin. It also outperforms humans when presented with partial observations of
communication cues (i.e., less than 40% of full actions). This early prediction
capability enables robots to initiate turn-taking actions at an early stage,
which facilitates collaboration and increases overall efficiency.Comment: Submitted to IEEE International Conference on Robotics and Automation
(ICRA) 201
Symbol Emergence in Robotics: A Survey
Humans can learn the use of language through physical interaction with their
environment and semiotic communication with other people. It is very important
to obtain a computational understanding of how humans can form a symbol system
and obtain semiotic skills through their autonomous mental development.
Recently, many studies have been conducted on the construction of robotic
systems and machine-learning methods that can learn the use of language through
embodied multimodal interaction with their environment and other systems.
Understanding human social interactions and developing a robot that can
smoothly communicate with human users in the long term, requires an
understanding of the dynamics of symbol systems and is crucially important. The
embodied cognition and social interaction of participants gradually change a
symbol system in a constructive manner. In this paper, we introduce a field of
research called symbol emergence in robotics (SER). SER is a constructive
approach towards an emergent symbol system. The emergent symbol system is
socially self-organized through both semiotic communications and physical
interactions with autonomous cognitive developmental agents, i.e., humans and
developmental robots. Specifically, we describe some state-of-art research
topics concerning SER, e.g., multimodal categorization, word discovery, and a
double articulation analysis, that enable a robot to obtain words and their
embodied meanings from raw sensory--motor information, including visual
information, haptic information, auditory information, and acoustic speech
signals, in a totally unsupervised manner. Finally, we suggest future
directions of research in SER.Comment: submitted to Advanced Robotic
Towards Assistive Feeding with a General-Purpose Mobile Manipulator
General-purpose mobile manipulators have the potential to serve as a
versatile form of assistive technology. However, their complexity creates
challenges, including the risk of being too difficult to use. We present a
proof-of-concept robotic system for assistive feeding that consists of a Willow
Garage PR2, a high-level web-based interface, and specialized autonomous
behaviors for scooping and feeding yogurt. As a step towards use by people with
disabilities, we evaluated our system with 5 able-bodied participants. All 5
successfully ate yogurt using the system and reported high rates of success for
the system's autonomous behaviors. Also, Henry Evans, a person with severe
quadriplegia, operated the system remotely to feed an able-bodied person. In
general, people who operated the system reported that it was easy to use,
including Henry. The feeding system also incorporates corrective actions
designed to be triggered either autonomously or by the user. In an offline
evaluation using data collected with the feeding system, a new version of our
multimodal anomaly detection system outperformed prior versions.Comment: This short 4-page paper was accepted and presented as a poster on
May. 16, 2016 in ICRA 2016 workshop on 'Human-Robot Interfaces for Enhanced
Physical Interactions' organized by Arash Ajoudani, Barkan Ugurlu, Panagiotis
Artemiadis, Jun Morimoto. It was peer reviewed by one reviewe
Diffusion Co-Policy for Synergistic Human-Robot Collaborative Tasks
Modeling multimodal human behavior has been a key barrier to increasing the
level of interaction between human and robot, particularly for collaborative
tasks. Our key insight is that an effective, learned robot policy used for
human-robot collaborative tasks must be able to express a high degree of
multimodality, predict actions in a temporally consistent manner, and recognize
a wide range of frequencies of human actions in order to seamlessly integrate
with a human in the control loop. We present Diffusion Co-policy, a method for
planning sequences of actions that synergize well with humans during test time.
The co-policy predicts joint human-robot action sequences via a
Transformer-based diffusion model, which is trained on a dataset of
collaborative human-human demonstrations, and directly executes the robot
actions in a receding horizon control framework. We demonstrate in both
simulation and real environments that the method outperforms other state-of-art
learning methods on the task of human-robot table-carrying with a human in the
loop. Moreover, we qualitatively highlight compelling robot behaviors that
demonstrate evidence of true human-robot collaboration, including mutual
adaptation, shared task understanding, leadership switching, and low levels of
wasteful interaction forces arising from dissent.Comment: IEEE Robotics and Automation Letters (RA-L), 2023. 8 pages, 7
figures, 3 tables. Supplementary material at
https://sites.google.com/view/diffusion-co-policy-hr
Generating Engagement Behaviors in Human-Robot Interaction
Based on a study of the engagement process between humans, I have developed models for four types of connection events involving gesture and speech: directed gaze, mutual facial gaze, adjacency pairs and backchannels. I have developed and validated a reusable Robot Operating System (ROS) module that supports engagement between a human and a humanoid robot by generating appropriate connection events. The module implements policies for adding gaze and pointing gestures to referring phrases (including deictic and anaphoric references), performing end-of-turn gazes, responding to human-initiated connection events and maintaining engagement. The module also provides an abstract interface for receiving information from a collaboration manager using the Behavior Markup Language (BML) and exchanges information with a previously developed engagement recognition module. This thesis also describes a Behavior Markup Language (BML) realizer that has been developed for use in robotic applications. Instead of the existing fixed-timing algorithms used with virtual agents, this realizer uses an event-driven architecture, based on Petri nets, to ensure each behavior is synchronized in the presence of unpredictable variability in robot motor systems. The implementation is robot independent, open-source and uses the Robot Operating System (ROS)
- …