Personal Guides: Heterogeneous Robots Sharing Personal Tours in Multi-Floor Environments

GidaBot is an application designed to setup and run a heterogeneous team of robots to act as tour guides in multi-floor buildings. Although the tours can go through several floors, the robots can only service a single floor, and thus, a guiding task may require collaboration among several robots. The designed system makes use of a robust inter-robot communication strategy to share goals and paths during the guiding tasks. Such tours work as personal services carried out by one or more robots. In this paper, a face re-identification/verification module based on state-of-the-art techniques is developed, evaluated offline, and integrated into GidaBot’s real daily activities, to avoid new visitors interfering with those attended. It is a complex problem because, as users are casual visitors, no long-term information is stored, and consequently, faces are unknown in the training step. Initially, re-identification and verification are evaluated offline considering different face detectors and computing distances in a face embedding representation. To fulfil the goal online, several face detectors are fused in parallel to avoid face alignment bias produced by face detectors under certain circumstances, and the decision is made based on a minimum distance criterion. This fused approach outperforms any individual method and highly improves the real system’s reliability, as the tests carried out using real robots at the Faculty of Informatics in San Sebastian show.This work has been partially funded by the Basque Government, Spain, grant number IT900-16, and the Spanish Ministry of Economy and Competitiveness (MINECO), grant number RTI2018-093337-B-I00

A real-time service for face re-identification based on deep learning and online clustering

Un servizio di re-identificazione facciale, una volta rilevata una faccia, si occupa di determinare se è già stata vista dal sistema e se è nuova. Poter distinguere persone nuove da quelle già viste, può essere utile nei robot sociali per poter adattare il loro comportamento a seconda di chi sta interagendo con loro. La re-identificazione è un compito particolarmente impegnativo poiché si parte senza avere alcun tipo di informazione su chi bisognerà identificare. Il database sui cui si basa il riconoscimento deve essere aggiornato in tempo reale senza alcun controllo sulle condizioni in cui sono state raccolte le facce e la loro qualità. Il sistema proposto si basa su una rete neurale per poter estrarre le caratteristiche di una faccia rilevata e su un sistema di clustering per raggruppare correttamente i volti appartenenti alla stessa persona. Poiché il riconoscimento si basa sui cluster creati dal sistema, è particolarmente importante aggiornarli con facce di buona qualità. Per fare ciò ogni volto è associato con quality score che risulta cruciale per decide che facce mantenere, quali eliminare e più in generale nelle operazioni di creazione ed aggiornamento dei cluster. Il sistema proposto ha raggiunto una precisione di 0.78 su facce rilevate da un robot mobile in un ambiente non controllato. Un False Acceptance Rate di 0.15 e un False Rejection Rate of 0.07, principalmente a causa di errori di classificazione dovuti a viste laterali dei volti.A face re-identification service aims to verify if detected faces have already been seen from the system. The ability of identify new and returning persons and distinguish them can be useful in social robots in order to adapt their behavior according to who they have in front. The reidentification problem is particular challenging since we start with no previous information about the persons to recognize and the database upon which we do recognition must be built in real-time with no control over the conditions and quality of the gathered faces. The proposed system is based on deep neural network to be able to extract the features from a detected face and a clustering system to correctly group and store the faces of the same person together. In particular, since the recognition is performed on such clusters, is crucial to keep them of the higher quality possible. To do so every face is tied with a quality score, which will play a vital role in choosing which faces keep in the sets, which not, and in general in the cluster update and creation operations. The system has achieved an accuracy of 0.78 on faces detected by a mobile robot in an unconstrained environment. With a False Acceptance Rate of 0.15 and a False Rejection Rate of 0.07 mainly due to misclassification of sideview of the faces

A real-time and unsupervised face re-identification system for human-robot interaction

In the context of Human-Robot Interaction (HRI), face Re-Identification (face Re-ID) aims to verify if certain detected faces have already been observed by robots. The ability of distinguishing between different users is crucial in social robots as it will enable the robot to tailor the interaction strategy toward the users’ individual preferences. So far face recognition research has achieved great success, however little attention has been paid to the realistic applications of Face Re-ID in social robots. In this paper, we present an effective and unsupervised face Re-ID system which simultaneously re-identifies multiple faces for HRI. This Re-ID system employs Deep Convolutional Neural Networks to extract features, and an online clustering algorithm to determine the face's ID. Its performance is evaluated on two datasets: the TERESA video dataset collected by the TERESA robot, and the YouTube Face Dataset (YTF Dataset). We demonstrate that the optimised combination of techniques achieves an overall 93.55% accuracy on TERESA dataset and an overall 90.41% accuracy on YTF dataset. We have implemented the proposed method into a software module in the HCI^2 Framework [1] for it to be further integrated into the TERESA robot [2] , and has achieved real-time performance at 10–26 Frames per second

Desarrollo de un sistema computacional para el análisis de procesos emocionales a través de las técnicas de reconocimiento facial y de potenciales relacionados a eventos

Objetivo: Establecer una metodología basada en las técnicas de reconocimiento facial y de potenciales relacionados a eventos para los procesos de orientación vocacional de universitarios. Metodología: Para la realización de este trabajo se consideró la metodología de desarrollo de software Proceso Unificado Racional (RUP), la cual incluyó las fases de Inicio, Elaboración, Construcción y Transición. Estas fases contemplan los análisis de requerimientos, diseño y construcción de la herramienta para el análisis de emociones, las pruebas con estudiantes, el análisis de pruebas y la entrega final del software. El sistema de análisis de emociones se construyó a través de Reconocimiento Facial de Emociones RFE (Affectiva), evaluación de Electroencefalografía EEG (Emotiv), sistema de gestión de protocolos, aplicación en formato digital de la prueba de Kuder y evaluación automática de las respuestas brindadas por los encuestados según área de interés. Resultados: Se consolidó una metodología basada en RFE y EEG para el análisis de emociones en procesos de orientación vocacional de estudiantes universitarios. Se automatizó la aplicación de la prueba de Kuder. Se realizó el desarrollo de la herramienta computacional para la presentación de protocolos, el seguimiento de emociones con RFE y EEG. Se validó la herramienta con 25 sujetos de prueba, los cuales respondieron la prueba de Kuder y fueron evaluados mediante la herramienta mientras observaban protocolos de estimulación asociados con sus áreas de interés y de no interés. Se analizaron los datos adquiridos y se encontró la efectividad de la herramienta encontrándose un porcentaje de afinidad con la prueba de orientación vocacional con un acierto cercano al 85%, especificidad del 87% y sensibilidad del 88%, resultados de valor para un ambiente de alta variabilidad. Conclusiones: Fue posible consolidar una metodología basada en el análisis de RFE y EEG para complementar las pruebas de orientación vocacional. Se utilizó como medio de referencia la prueba de Kuder que permitió validar la capacidad del método a la hora de identificar emociones al tiempo que se observan protocolos de estimulación asociados con áreas de desempeño vocacional. La metodología propuesta puede ser usada como complemento en los procesos de orientación vocacional y como una prueba rápida para encontrar afinidad entre el evaluado y las diferentes áreas de interés.Objective: To establish a methodology based on facial recognition techniques and event-related potentials for the vocational orientation processes of university students. Methodology: For the realization of this work, the Rational Unified Process (RUP) software development methodology was considered, which included the phases of Initiation, Elaboration, Construction and Transition. These phases contemplate the requirements analysis, design and construction of the emotion analysis tool, student testing, test analysis and final delivery of the software. The emotion analysis system was built through Facial Recognition of Emotions RFE (Affectiva), Electroencephalography EEG evaluation (Emotiv), protocol management system and automatic Kuder test evaluation. Results: A methodology based on RFE and EEG was consolidated for the analysis of emotions in vocational orientation processes of university students. The application of the Kuder test was automated. A computational tool was developed for the presentation of protocols, monitoring of emotions with RFE and EEG. The tool was validated with 25 test subjects, who responded to the Kuder test and were evaluated using the tool while observing stimulation protocols associated with their areas of interest and non-interest. The acquired data were analyzed and the effectiveness of the tool was found, finding a percentage of affinity with the vocational orientation test with a hit rate close to 85%, specificity of 87% and sensitivity of 88%, results of value for an environment of high variability.Conclusions: It was possible to consolidate a methodology based on RFE and EEG analysis to complement vocational orientation tests. The Kuder test was used as a reference medium, which allowed validating the ability of the method to identify emotions while observing stimulation protocols associated with areas of vocational performance. The proposed methodology can be used as a complement in vocational orientation processes and as a quick test to find affinity between the evaluated and the different areas of interest