Haptic role allocation and intention negotiation in human-robot collaboration

This dissertation aims to present a perspective to build more natural shared control systems for physical human-robot cooperation. As the tasks become more complex and more dynamic, many shared control schemes fail to meet the expectation of an effortless interaction that resembles human-human sensory communication. Since such systems are mainly built to improve task performance, the richness of sensory communication is of secondary concern. We suggest that effective cooperation can be achieved when the human’s and the robot’s roles within the task are dynamically updated during the execution of the task. These roles define states for the system, in which the robot’s control leads or follows the human’s actions. In such a system, a state transition can occur at certain times if the robot can determine the user’s intention for gaining/relinquishing control. Specifically, with these state transitions we assign certain roles to the human and the robot. We believe that only by employing the robot with tools to change its behavior during collaboration, we can improve the collaboration experience. We explore how human-robot cooperation in virtual and physical worlds can be improved using a force-based role-exchange mechanism. Our findings indicate that the proposed role exchange framework is beneficial in a sense that it can improve task performance and the efficiency of the partners during the task, and decrease the energy requirement of the human. Moreover, the results imply that the subjective acceptability of the proposed model is attained only when role exchanges are performed in a smooth and transparent fashion. Finally, we illustrate that adding extra sensory cues on top of a role exchange scheme is useful for improving the sense of interaction during the task, as well as making the system more comfortable and easier to use, and the task more enjoyable

Modified Cooper Harper Scales for Assessing Unmanned Vehicle Displays

In unmanned vehicle (UV) operations, displays are often the only information link between operators and vehicles. It is essential these displays present information clearly and efficiently so that operators can interact with the UVs to achieve mission objectives. While there are a variety of metrics to evaluate displays, there is no current standardized methodology for operators to subjectively assess a display’s support and identify specific deficiencies. Such a methodology could improve current displays and ensure that displays under development support operator processes. This report presents a quasi- subjective display evaluation tool called the Modified Cooper-Harper for Unmanned Vehicle Displays (MCH-UVD) diagnosis tool. This tool, adapted from the Cooper-Harper aircraft handling scale, allows operators to assess a display, translating their judgments on potential display shortcomings into a number corresponding to a particular deficiency in operator support. The General MCH-UVD can be used to diagnose deficiencies for any UV display, while the Specific MCH-UVD is UV and mission specific in its evaluation of displays. This report presents the General MCH-UVD and provides guidance on how to adapt it to create a Specific MCH-UVD through the use of UV mission taxonomies and a questioning method. A UGV search mission case study provides a how-to guide example for generating a Specific MCH-UVD. The report also presents an experiment conducted to validate the MCH-UVD and assess if a mission-specific version is necessary, or if the general form of the MCH-UVD is sufficient for different UV display evaluation. The report concludes with discussion on how to administer the scale, when a Specific scale is necessary, MCH-UVD diagnosis tool limitations, and future work.Prepared for US Army Aberdeen Testing Cente

Design, Analysis and Fabrication of a Reconfigurable Stair Climbing Robot

Over the past few years, the scientists have tried to develop robots that can move on rough terrains. However, there are few robots that are suitable for use in rough terrains. A number of new technologies have evolved for reliable localization, obstacle avoidance and even autonomous map building in dynamically changing environment. However, mobility in very rough terrain is often very limited due to the absence of adequate locomotion concepts. The aim of this project is to introduce a new class of locomotive concept that will have excellent off-road capabilities. As a first prototype of this class, this four-wheeled robot will have the capability of climbing the stairs of height equal to its diameter. It will possess maximum gripping capacity and stability during motion in rough terrain owing to the 4 differential driven wheel configurations. The long -term goal of our research is to develop a robust outdoor platform which is suitable to be included in disaster mitigation as well as in security and surveillance missions. The platform should be able to transport application sensors to areas that are dangerous for humans to access, e.g. a collapse-endangered building or an industrial compound after a chemical accident. In those cases, before they enter, the rescue personnel might need some information about the air contamination or the whereabouts of people inside an area. The robot should be upgradeable with a variety of application sensors, e.g. cameras, thermal vision, or chemical sensors. To be usable in any search and rescue or security application, the robot has to be operational without changing batteries for at least two hours. As the first step into these future goals, our work has wireless control of the robot, which will steer the robot in the target area from remote. The robot will be wirelessly controlled through PC using ZigBee technology. In the future work, sensors, cameras, manipulators can be added to the robot frame. The robot can then serve complex tasks in dangerous areas remotely

Automated decision making and problem solving. Volume 2: Conference presentations

Related topics in artificial intelligence, operations research, and control theory are explored. Existing techniques are assessed and trends of development are determined

Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version

Design of Interactive Service Robots applying methods of Systems Engineering and Decision Making

Interaktive Service Roboter werden heute bereits in einigen Anwendungsszenarien eingesetzt, in denen sie beispielsweise Menschen durch Gebäude geleiten oder bei häuslichen Aufgaben unterstützen. Dennoch gibt es bislang kein System, das den erwarteten Marktdurchbruch geschafft hat. Die hohe Komplexität solcher Systeme und vielfältige Anforderungen durch Benutzer und Betreiber erschweren die Entwicklung von erfolgreichen Service Robotern. In dieser Arbeit wurden zwei interaktive Service Roboter entwickelt, die das Potential haben, die beschriebenen Hinderungsgründe für einen breiten Einsatz zu überwinden. Das erste Robotersystem wurde als Shopping Roboter für Baumärkte entwickelt, in denen es Kunden zu gesuchten Produkten führt. Das zweite System dient als interaktiver Pflegeroboter älteren Menschen in häuslicher Umgebung bei der Bewältigung täglicher Aufgaben. Diese Arbeit beschreibt die Realisierung der Embedded Systems beider Robotersysteme und umfasst insbesondere die Entwicklung der Low-Level System Architekturen, Energie Management Systeme, Kommunikationssysteme, Sensorsysteme, sowie ausgewählte Aspekte der mechanischen Umsetzung. Die Entwicklung einer Vielzahl von Steuerungsmodulen, notwendig für die Realisierung interaktiver Service Roboter, wird beschrieben. Die vorliegende Arbeit verwendet und erweitert Methoden des Systems Engineerings, um die hohe Systemkomplexität von interaktiven Service Robotern sowie die vielfältigen Anforderungen an deren späteren Einsatz beherrschen zu können. Der Entwicklungsprozess der beiden Roboter basiert auf dem V-Model, welches einen strukturierten Entwurfsablauf unter Berücksichtigung aller Systemanforderungen erlaubt. Es zwingt ferner zur frühzeitigen Spezifikation von Prüfabläufen, was die Qualität und Zuverlässigkeit der Entwicklungsergebnisse verbessert. Für die Unterstützung von Entscheidungen im Entwicklungsprozess schlägt diese Arbeit eine Kombination aus dem V-Model und dem Analytic Hierarchy Process (AHP) vor. Der AHP hilft bei der Auswahl verfügbarer technischer Alternativen unter Berücksichtigung von Prioritäten im Entwicklungsprozess. Diese Arbeit spezifiziert sieben Kriterien, die Service Roboter charakterisieren: Anpassbarkeit, Laufzeit, Benutzbarkeit, Robustheit, Sicherheit, Features und Kosten. Die Prioritäten dieser Kriterien im Entwicklungsprozess werden für jeden Roboter individuell bestimmt. Der AHP ermittelt die beste Lösung basierend auf diesen gewichteten Kriterien und den bewerteten technischen Alternativen. Die Einbindung des AHP in den V-Model Prozess wurde am Entwurf des Shopping Roboter entwickelt und geprüft. Die Allgemeingültigkeit dieser Methode wurde während der Entwicklung des Pflegeroboters verifiziert.Interactive service robots have already been developed and operate as example installations taking over guidance tasks or serving as home assistants. However, none of these systems have become an off-the-shelf product or have achieved the predicted breakthrough so far. The challenges of the design of such systems are, on the one hand, the combination of cutting edge technologies to a complex product; on the other hand, the consideration of requirements important for the later marketing during the design process. In the framework of this dissertation, two interactive service robot systems are developed that have the potential to overcome current market entry barriers. These robots are designed to operate in two different environments: one robot guides walked-in users in large home improvement stores to requested product locations and interacts with the customer to provide product information; the other robot assists elderly people to stay longer in their homes and takes over home-care tasks. This work describes the realization of the embedded systems of both robots. In particular, the design of low-level system architectures, energy management systems, communication systems, sensor systems, and selected aspects of mechanical implementations are carried out in this work. Multiple embedded system modules are developed for the control of the robots' functionalities; the development processes as well as the composition and evaluation of these modules are presented in this work. To cope with the complexity and the various factors that are important for the design of the robots, this thesis applies and further develops system engineering methods. The development process is based on the V-Model system design method. The V-Model helps to structure the design process under consideration of all system requirements. It involves evaluation procedures at all design levels, and thus increases the quality and reliability of the development outputs. To support design decisions, this thesis proposes to combine the V-Model with the Analytic Hierarchy Process (AHP) method. The AHP helps to evaluate technical alternatives for design decisions according to overall criteria, a system has to fulfill. This thesis defines seven criteria that characterize a service robot: Adaptability, Operation Time, Usability, Robustness, Safeness, Features, and Costs. These criteria are weighted for each individual robot application. The AHP evaluates technical design alternatives based on the weighted criteria to reveal the best technical solution. The integration of the AHP into the V-Model development is tested and improved during the design process of the shopping robot system. The generality of this combined systematic design approach is validated during the design of the home-care robot system

Quantifying protocol evaluation for autonomous collision avoidance

Collision avoidance protocols such as COLREGS are written primarily for human operators resulting in a rule set that is open to some interpretation, difficult to quantify, and challenging to evaluate. Increasing use of autonomous control of vehicles emphasizes the need to more uniformly establish entry and exit criteria for collision avoidance rules, adopt a means to quantitatively evaluate performance, and establish a “road test” for autonomous marine vehicle collision avoidance. This paper presents a means to quantify and subsequently evaluate the otherwise subjective nature of COLREGS thus providing a path toward standardized evaluation and certification of protocol-constrained collision avoidance systems based on admiralty case law and on-water experience. Notional algorithms are presented for evaluation of COLREGS collision avoidance rules to include overtaking, head-on, crossing, give-way, and stand-on rules as well as applicable entry criteria. These rules complement and enable an autonomous collision avoidance road test as a first iteration of algorithm certification prior to vessels operating in human-present environments. Additional COLREGS rules are discussed for future development. Both real-time and post-mission protocol evaluation tools are introduced. While the motivation of these techniques applies to improvement of autonomous marine collision avoidance, the concepts for protocol evaluation and certification extend naturally to human-operated vessels. Evaluation of protocols governing other physical domains may also benefit from adapting these techniques to their cases. Keywords: COLREGS; Autonomous collision avoidance; Human–robot collaboration; Marine navigatio