Early Turn-taking Prediction with Spiking Neural Networks for Human Robot Collaboration

Turn-taking is essential to the structure of human teamwork. Humans are typically aware of team members' intention to keep or relinquish their turn before a turn switch, where the responsibility of working on a shared task is shifted. Future co-robots are also expected to provide such competence. To that end, this paper proposes the Cognitive Turn-taking Model (CTTM), which leverages cognitive models (i.e., Spiking Neural Network) to achieve early turn-taking prediction. The CTTM framework can process multimodal human communication cues (both implicit and explicit) and predict human turn-taking intentions in an early stage. The proposed framework is tested on a simulated surgical procedure, where a robotic scrub nurse predicts the surgeon's turn-taking intention. It was found that the proposed CTTM framework outperforms the state-of-the-art turn-taking prediction algorithms by a large margin. It also outperforms humans when presented with partial observations of communication cues (i.e., less than 40% of full actions). This early prediction capability enables robots to initiate turn-taking actions at an early stage, which facilitates collaboration and increases overall efficiency.Comment: Submitted to IEEE International Conference on Robotics and Automation (ICRA) 201

Probabilistic Human-Robot Information Fusion

This thesis is concerned with combining the perceptual abilities of mobile robots and human operators to execute tasks cooperatively. It is generally agreed that a synergy of human and robotic skills offers an opportunity to enhance the capabilities of today’s robotic systems, while also increasing their robustness and reliability. Systems which incorporate both human and robotic information sources have the potential to build complex world models, essential for both automated and human decision making. In this work, humans and robots are regarded as equal team members who interact and communicate on a peer-to-peer basis. Human-robot communication is addressed using probabilistic representations common in robotics. While communication can in general be bidirectional, this work focuses primarily on human-to-robot information flow. More specifically, the approach advocated in this thesis is to let robots fuse their sensor observations with observations obtained from human operators. While robotic perception is well-suited for lower level world descriptions such as geometric properties, humans are able to contribute perceptual information on higher abstraction levels. Human input is translated into the machine representation via Human Sensor Models. A common mathematical framework for humans and robots reinforces the notion of true peer-to-peer interaction. Human-robot information fusion is demonstrated in two application domains: (1) scalable information gathering, and (2) cooperative decision making. Scalable information gathering is experimentally demonstrated on a system comprised of a ground vehicle, an unmanned air vehicle, and two human operators in a natural environment. Information from humans and robots was fused in a fully decentralised manner to build a shared environment representation on multiple abstraction levels. Results are presented in the form of information exchange patterns, qualitatively demonstrating the benefits of human-robot information fusion. The second application domain adds decision making to the human-robot task. Rational decisions are made based on the robots’ current beliefs which are generated by fusing human and robotic observations. Since humans are considered a valuable resource in this context, operators are only queried for input when the expected benefit of an observation exceeds the cost of obtaining it. The system can be seen as adjusting its autonomy at run-time based on the uncertainty in the robots’ beliefs. A navigation task is used to demonstrate the adjustable autonomy system experimentally. Results from two experiments are reported: a quantitative evaluation of human-robot team effectiveness, and a user study to compare the system to classical teleoperation. Results show the superiority of the system with respect to performance, operator workload, and usability

Implementation of PPP as new GNSS Observation Type in the Geomonitoring System GOCA

[EN] Early detection of significant movements in both natural and artificial structures is crucial to prevent human, environmental and economic losses. For this reason, Geomonitoring in an active field. GNSS technics are also a filed in which lot of research and improvement have been made in recent years. Some studies have indicated the potential of GNSS technics in the field of Geomonitoring. The aim of this master thesis is developing a software that allows processing GNSS data with Precise Point Positioning technic in the context of the geomonitoring project GOCA. With this implementation, potential of PPP with low cost receiver (U-Blox ZED-F9P) using different products and settings is evaluated in this document. Based on a literature review, that includes the study of GOCA project and a summary of main PPP approaches, a C++ dialog-based software was design and developed, using RTKLIB and WaPPP as software engines. Besides that, two different observations were made (one 12 hours to post-processing and one real time) in order to test the developed software and evaluate the obtained results using different parameters or products. The obtained results reaffirm the potential of the PPP technique, even using low cost receiver. Even some differences between different software engines or IGS products were found, the results allow us to conclude that PPP is a technique with many advantages in the field of geomonitoring, since it avoids the use of several receivers and good accuracies are obtained. However, some aspects need further research in this context, as there is no common criterion for establishing convergence time and new methodologies and algorithms are being developed in the field of PPP processing.[ES] La detección temprana de movimientos significativos en estructuras naturales y artificiales es crucial para prevenir pérdidas humanas, ambientales y económicas. Por esta razón, Geomonitorización es un campo activo. Las técnicas de GNSS son también un campo en el que se han realizado muchas investigaciones y mejoras en los últimos años. Algunos estudios han indicado el potencial de las técnicas GNSS en el campo de la geomonitorización. El objetivo de esta tesis de máster es desarrollar un software que permita el procesamiento de datos GNSS con la técnica de posicionamiento de punto preciso en el contexto del proyecto de geomonitorización GOCA. Con esta implementación, el potencial del PPP con el receptor de bajo coste (U-Blox ZED-F9P) usando diversos productos y configuraciones se va a evalúar en este documento. Basado en una revisión de la literatura, que incluye el estudio del proyecto GOCA y un resumen de los principales enfoques PPP, se diseñó y desarrolló un software basado en diálogos C++, utilizando RTKLIB y WaPPP como motores de software. Además, se realizaron dos observaciones diferentes (una de 12 horas para el post-procesamiento y otra en tiempo real) con el fin de probar el software desarrollado y evaluar los resultados obtenidos utilizando diferentes parámetros o productos. Los resultados obtenidos reafirman el potencial de la técnica PPP, incluso utilizando un receptor de bajo coste. Incluso habiendo encontrado algunas diferencias entre diferentes motores de software o productos IGS, los resultados nos permiten concluir que PPP es una técnica con muchas ventajas en el campo de la geomonitorización, ya que evita el uso de varios receptores y se obtienen buenas precisiones. Sin embargo, algunos aspectos necesitan más investigación en este contexto, ya que no existe un criterio común para establecer el tiempo de convergencia y se están desarrollando nuevas metodologías y algoritmos en el campo del procesamiento PPP.Luján García Muñoz, R. (2020). Implementation of PPP as new GNSS Observation Type in the Geomonitoring System GOCA. http://hdl.handle.net/10251/139668TFG