Mobile manipulation with the TIAGo robot: perception and task manager

This project aims to design and implement a solution for a complex mobile manipulation task using PAL Robotics’ robot TIAGo. The goal is to use the platform as an assistive robot, making it capable of picking a chosen object from a certain location and moving it to another. More precisely the task in which the project will focus will be to pick a certain soda can from a table with several of them on it and pour its contents into a glass placed on another table for the user to have his/her drink . For that, all the perception, interaction, planning and mobile capabilities of the robotic platform will be exploited in order to develop a suitable and complete solution. The generated solution is a modular method that can be launched as a whole process to complete the entire challenge, but this solution also gives the possibility of using each one of the modules independently. This way it is feasible to easily integrate them to other processes in order to complete other similar tasks, making the packages more versatile and adaptable. The entire project has been divided in two parts. One focused on developing the packages in charge of navigation and arm manipulation, carried out by Xavier Garcia Peroy. The other one, focused on developing perception and task management part, and described in this report. For the perception part, some computer vision capabilities have been implemented using TIAGo’s camera. These capabilities were added in order to get knowledge about the objects with which the robot needs to interact. In this case, these objects were cans of soda which the robot needed to detect and, using some image processing steps, determine their position in order to pick the can of soda chosen by the user. On the other hand, for task management part, a solution based on behavior trees was developed. This solution has been done in a modular way, removing a big part of the complexity of executing each necessary task to achieve the goal of this project, and also decreasing the necessity of going deep in programming in order to make changes and check partial functionalities. That has been achieved with the use of the BehaviorTree library and Groot, a visual tool that has allowed to create the task manager functionalities graphically. Both solutions have been checked to work properly in order to achieve the established goal. However, specially in perception part, some functionalities should still be improved in order to increase the robustness of the method and decrease some limitations. Future work is suggested in order to make these enhancements

Robustness and accuracy evaluation of a multicamera tracking system

Intra-Operative Electron Radiation Therapy (IOERT) consists on radiating the post-resected tumour bed or the unresected tumour during a surgery. This therapy is planned with the software radiance, developed from a collaborative work between Universidad Carlos III de Madrid (UC3M) and GMV (Madrid, Spain). The recently added tracking system (OptiTrack, NaturalPoint Inc., USA) allows for navigation and thus helps the clinician to follow the planning ([1]). In order to study the dependence and the magnitude of the positioning error obtained with this tracking system, this project aims at comparing the positions given by the aforementioned camera system with those of an industrial robot that will act as a gold standard. To achieve this, a Matlab client was developed to control the OptiTrack system by means of the Application Programming Interface provided and communicates with a RAPID robot server through a TCP/IP protocol, sending the positions and ordering the movements. A specific trajectory was designed in RobotStudio in order to cover the working volume in an operating room. The results show the magnitude of the error of OptiTrack system and its spatial dependence, as well as the effect of different numbers of occluded cameras, different tracked tools and different calibration qualities. This study will improve the new IOERT navigator using OptiTrack at HGGM by means of providing tracking error estimation during the procedure.Ingeniería Biomédic

Advances in Human Robot Interaction for Cloud Robotics applications

In this thesis are analyzed different and innovative techniques for Human Robot Interaction. The focus of this thesis is on the interaction with flying robots. The first part is a preliminary description of the state of the art interactions techniques. Then the first project is Fly4SmartCity, where it is analyzed the interaction between humans (the citizen and the operator) and drones mediated by a cloud robotics platform. Then there is an application of the sliding autonomy paradigm and the analysis of different degrees of autonomy supported by a cloud robotics platform. The last part is dedicated to the most innovative technique for human-drone interaction in the User’s Flying Organizer project (UFO project). This project wants to develop a flying robot able to project information into the environment exploiting concepts of Spatial Augmented Realit

Virtual Reality applications design for physiotherapy and physical training

Els estudis i les evidències dels avantatges que aporta la realitat virtual en teràpies de rehabilitació son cada vegada més abundants. En aquest projecte, amb la col·laboració de l'Associació de Diversitat Funcional d’Osona (ADFO), s’ha dissenyat dos jocs amb l’objectiu de crear aplicacions de realitat virtual per millorar la rehabilitació dels pacients després de patir un ictus. El problema principal de les teràpies de rehabilitació és que es fan molt monòtones pels pacients, ja que els exercicis solen ser repetitius d’una sessió a l’altre. Gràcies a la creació d’aquests jocs, aquest problema es redueix. A més, disminueix la saturació de pacients a les consultes i, el fisioterapeuta pot tenir una base de dades amb un altre tipus de resultats, tant a nivell tècnic com visual. Un dels jocs creats està dissenyat per fer-lo servir amb l’ordinador i el ratolí, ja que no es disposava d’equips de realitat virtual (SteamVR HTC Vive) en els domicilis pertinents. Tot i així, aquesta aplicació pot adaptar-se a una configuració en realitat virtual si en un futur es volgués utilitzar el joc en aquest sentit. L’altre joc creat està dissenyat exclusivament per ser una aplicació en realitat virtual. Per fer-ho, s’han utilitzat uns dispositius mòbils amb unes aplicacions de realitat virtual per poder simular les ulleres i els controladors. No és exactament el mateix que utilitzar un equip real de realitat virtual, però tot i així han estat de gran utilitat. Per altra banda, les proves de les aplicacions no s’han pogut realitzar amb els pacients de l’ADFO. En un futur es té previst fer-ho per avaluar millor la funcionalitat de les aplicacions creades. Tot i així, s’ha obtingut algunes opinions de persones sanes que sí que han pogut provar aquests jocs i s’ha pogut rectificar en funció d’això.Los estudios y las evidencias de las ventajas que aporta la realidad virtual en terapias de rehabilitación son cada vez más. En este proyecto, con la colaboración de la Associació de Diversitat Funcional d’Osona (ADFO), se han diseñado dos juegos con el objetivo de crear aplicaciones de realidad virtual para mejorar la rehabilitación de los pacientes después de sufrir un ictus. El problema principal de las terapias de rehabilitación es que se hacen muy monótonas por el paciente ya que los ejercicios suelen ser repetitivos de una sesión a la otra. Gracias a la creación de serious games (juegos cuyo propósito principal es distinto al entretenimiento) este problema se reduce. A demás, disminuye la saturación de pacientes en las consultas y, el fisioterapeuta puede tener una base de datos con otro tipo de resultados, tanto a nivel técnico como visual. Uno de los juegos creados está diseñado para usarlo con el ordenador y el ratón, ya que no se disponía de equipos de realidad virtual (SteamVR HTC Vive) en los domicilios pertinentes. Sin embargo, esta aplicación puede adaptarse a una configuración en realidad virtual si en un futuro se quisiera utilizar el juego en ese sentido. El otro juego creado sí que está diseñado exclusivamente para ser una aplicación en realidad virtual. Para ello, se han utilizado unos dispositivos móviles con unas aplicaciones de realidad virtual para poder simular las gafas y los controladores. No es exactamente lo mismo que usar un equipo real de realidad virtual, pero aun así han sido de gran utilidad. Por otra parte, las pruebas de las aplicaciones no se han podido realizar con los pacientes de la ADFO. En un futuro se tiene previsto hacerlo para evaluar mejor la funcionalidad de las aplicaciones creadas. Aun así, se ha conseguido algunas opiniones de personas sanas que si han podido probar estos juegos y se ha podido rectificar en función de ello.Studies and evidence of the benefits of virtual reality in rehabilitation therapies are increasingly abundant. In this project, with the collaboration of the Associació de Diversitat Funcional d’Osona (ADFO), two games have been designed with the aim of creating virtual reality applications to improve the rehabilitation of patients after suffering a stroke. The main problem with rehabilitation therapies is that they become very monotonous for the patient since the exercises are usually repetitive from one session to the other. Thanks to the creation of our games this problem has been reduced. In addition, the saturation of patients in the consultations will decrease and the physical therapist may have a database with other types of results, technically and visually. One of the games created is designed to be used with the computer and mouse, because virtual reality equipment (SteamVR HTC Vive) was not available in the relevant homes. However, this application can be adapted to a virtual reality configuration if in the future it wants to be used in this aspect. The other game created is designed exclusively to be a virtual reality application. To make this possible, mobile device with virtual reality applications have been used to simulate headset and controllers. It is not the same as using a real virtual reality equipment, however they have been especially useful. On the other hand, testing of the applications could not be performed with ADFO patients. It is planned to do it in the future for a better evaluation of the functionality of the applications created. However, some opinions have been obtained from healthy people who have been able to try these games and have been able to rectify based on that.Els estudis i les evidències dels avantatges que aporta la realitat virtual en teràpies de rehabilitació son cada vegada més abundants. En aquest projecte, amb la col·laboració de l'Associació de Diversitat Funcional d’Osona (ADFO), s’ha dissenyat dos jocs amb l’objectiu de crear aplicacions de realitat virtual per millorar la rehabilitació dels pacients després de patir un ictus. El problema principal de les teràpies de rehabilitació és que es fan molt monòtones pels pacients, ja que els exercicis solen ser repetitius d’una sessió a l’altre. Gràcies a la creació d’aquests jocs, aquest problema es redueix. A més, disminueix la saturació de pacients a les consultes i, el fisioterapeuta pot tenir una base de dades amb un altre tipus de resultats, tant a nivell tècnic com visual. Un dels jocs creats està dissenyat per fer-lo servir amb l’ordinador i el ratolí, ja que no es disposava d’equips de realitat virtual (SteamVR HTC Vive) en els domicilis pertinents. Tot i així, aquesta aplicació pot adaptar-se a una configuració en realitat virtual si en un futur es volgués utilitzar el joc en aquest sentit. L’altre joc creat està dissenyat exclusivament per ser una aplicació en realitat virtual. Per fer-ho, s’han utilitzat uns dispositius mòbils amb unes aplicacions de realitat virtual per poder simular les ulleres i els controladors. No és exactament el mateix que utilitzar un equip real de realitat virtual, però tot i així han estat de gran utilitat. Per altra banda, les proves de les aplicacions no s’han pogut realitzar amb els pacients de l’ADFO. En un futur es té previst fer-ho per avaluar millor la funcionalitat de les aplicacions creades. Tot i així, s’ha obtingut algunes opinions de persones sanes que sí que han pogut provar aquests jocs i s’ha pogut rectificar en funció d’això.Los estudios y las evidencias de las ventajas que aporta la realidad virtual en terapias de rehabilitación son cada vez más. En este proyecto, con la colaboración de la Associació de Diversitat Funcional d’Osona (ADFO), se han diseñado dos juegos con el objetivo de crear aplicaciones de realidad virtual para mejorar la rehabilitación de los pacientes después de sufrir un ictus. El problema principal de las terapias de rehabilitación es que se hacen muy monótonas por el paciente ya que los ejercicios suelen ser repetitivos de una sesión a la otra. Gracias a la creación de serious games (juegos cuyo propósito principal es distinto al entretenimiento) este problema se reduce. A demás, disminuye la saturación de pacientes en las consultas y, el fisioterapeuta puede tener una base de datos con otro tipo de resultados, tanto a nivel técnico como visual. Uno de los juegos creados está diseñado para usarlo con el ordenador y el ratón, ya que no se disponía de equipos de realidad virtual (SteamVR HTC Vive) en los domicilios pertinentes. Sin embargo, esta aplicación puede adaptarse a una configuración en realidad virtual si en un futuro se quisiera utilizar el juego en ese sentido. El otro juego creado sí que está diseñado exclusivamente para ser una aplicación en realidad virtual. Para ello, se han utilizado unos dispositivos móviles con unas aplicaciones de realidad virtual para poder simular las gafas y los controladores. No es exactamente lo mismo que usar un equipo real de realidad virtual, pero aun así han sido de gran utilidad. Por otra parte, las pruebas de las aplicaciones no se han podido realizar con los pacientes de la ADFO. En un futuro se tiene previsto hacerlo para evaluar mejor la funcionalidad de las aplicaciones creadas. Aun así, se ha conseguido algunas opiniones de personas sanas que si han podido probar estos juegos y se ha podido rectificar en función de ello.Studies and evidence of the benefits of virtual reality in rehabilitation therapies are increasingly abundant. In this project, with the collaboration of the Associació de Diversitat Funcional d’Osona (ADFO), two games have been designed with the aim of creating virtual reality applications to improve the rehabilitation of patients after suffering a stroke. The main problem with rehabilitation therapies is that they become very monotonous for the patient since the exercises are usually repetitive from one session to the other. Thanks to the creation of our games this problem has been reduced. In addition, the saturation of patients in the consultations will decrease and the physical therapist may have a database with other types of results, technically and visually. One of the games created is designed to be used with the computer and mouse, because virtual reality equipment (SteamVR HTC Vive) was not available in the relevant homes. However, this application can be adapted to a virtual reality configuration if in the future it wants to be used in this aspect. The other game created is designed exclusively to be a virtual reality application. To make this possible, mobile device with virtual reality applications have been used to simulate headset and controllers. It is not the same as using a real virtual reality equipment, however they have been especially useful. On the other hand, testing of the applications could not be performed with ADFO patients. It is planned to do it in the future for a better evaluation of the functionality of the applications created. However, some opinions have been obtained from healthy people who have been able to try these games and have been able to rectify based on that

Overcoming barriers and increasing independence: service robots for elderly and disabled people

This paper discusses the potential for service robots to overcome barriers and increase independence of elderly and disabled people. It includes a brief overview of the existing uses of service robots by disabled and elderly people and advances in technology which will make new uses possible and provides suggestions for some of these new applications. The paper also considers the design and other conditions to be met for user acceptance. It also discusses the complementarity of assistive service robots and personal assistance and considers the types of applications and users for which service robots are and are not suitable

Diagnosing Robotic Swarms 2 (Dr. Swarm2)

Robots are envisioned to work alongside humans. However, humans struggle to interpret the state and goals of a robot. The use of multiple robots further exacerbates this issue. To solve this problem, we propose Dr. Swarm 2, an augmented reality (AR) application built on the Magic Leap. The overlay provides concise information in a manner unachievable with existing methods

Assistente de navegação com apontador laser para conduzir cadeiras de rodas robotizadas

Orientador: Eric RohmerDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: As soluções de robótica assistida ajudam as pessoas a recuperar sua mobilidade e autonomia perdidas em suas vidas diárias. Este documento apresenta um assistente de navegação de baixo custo projetado para pessoas tetraplégicas para dirigir uma cadeira de rodas robotizada usando a combinação da orientação da cabeça e expressões faciais (sorriso e sobrancelhas para cima) para enviar comandos para a cadeira. O assistente fornece dois modos de navegação: manual e semi-autônomo. Na navegação manual, uma webcam normal com o algoritmo OpenFace detecta a orientação da cabeça do usuário e expressões faciais (sorriso, sobrancelhas para cima) para compor comandos e atuar diretamente nos movimentos da cadeira de rodas (parar, ir à frente, virar à direita, virar à esquerda). No modo semi-autônomo, o usuário controla um laser pan-tilt com a cabeça para apontar o destino desejado no solo e valida com o comando sobrancelhas para cima que faz com que a cadeira de rodas robotizada realize uma rotação seguida de um deslocamento linear para o alvo escolhido. Embora o assistente precise de melhorias, os resultados mostraram que essa solução pode ser uma tecnologia promissora para pessoas paralisadas do pescoço para controlar uma cadeira de rodas robotizadaAbstract: Assistive robotics solutions help people to recover their lost mobility and autonomy in their daily life. This document presents a low-cost navigation assistant designed for people paralyzed from down the neck to drive a robotized wheelchair using the combination of the head's posture and facial expressions (smile and eyebrows up) to send commands to the chair. The assistant provides two navigation modes: manual and semi-autonomous. In the manual navigation, a regular webcam with the OpenFace algorithm detects the user's head orientation and facial expressions (smile, eyebrows up) to compose commands and actuate directly on the wheelchair movements (stop, go front, turn right, turn left). In the semi-autonomous, the user controls a pan-tilt laser with his/her head to point the desired destination on the ground and validates with eyebrows up command which makes the robotized wheelchair performs a rotation followed by a linear displacement to the chosen target. Although the assistant need improvements, results have shown that this solution may be a promising technology for people paralyzed from down the neck to control a robotized wheelchairMestradoEngenharia de ComputaçãoMestre em Engenharia ElétricaCAPE

a robotized projective interface for human robot learning scenarios

Abstract In this work we discuss a novel robotics interface with perception and projection capabilities for facilitating the skill transfer process. The interface aims at allowing humans and robots to interact with each other in the same environment, with respect to visual feedback. During the learning process, the real workspace can be used as a graphical interface for helping the user to better understand what the robot has learned up to then, to display information about the task or to get feedback and guidance. Thus, the user can incrementally visualize and assess the learner's state and, at the same time, focus on the skill transfer without disrupting the continuity of the teaching interaction. We also propose a proof-of-concept, as a core element of the architecture, based on an experimental setting where a picoprojector and an rgb-depth sensor are mounted onto the end-effector of a 7-DOF robotic arm