Effects of Control Device and Task Complexity on Performance and Task Shedding During a Robotic Arm Task

The use of robotic arms across domains is increasing, but the relationship between control features and performance is not fully understood. The goal of this research was to investigate the difference in task performance when using two different control devices at high and low task complexities when participants can shed tasks to automation. In this experiment, 40 undergraduates (24 females) used two control devices, a Leap Motion controller and an Xbox controller, to teleoperate a robotic arm in a high or low complexity peg placement task. Simultaneously, participants were tasked with scanning images for tanks. During the experiment, participants had the option to task shed the peg task to imperfect automation. Analyses indicated a significant main effect of control device on task completion rate and time to first grasp the peg, with completion rate higher and time lower when using the Leap. However, participants made significantly more errors with the Leap Motion controller than with the Xbox controller. Participants in both conditions task shed similarly with both control devices and task shed at similar times. The 2 x 2 mixed ANOVAs somewhat supported the proposed hypotheses. The results of this study indicate that control device impacts performance on a robotic arm task. The Leap Motion controller supports increased task completion rate and quicker peg grasps in high and low task complexity when compared with the Xbox controller. This supports the extension of Control Order Theory into three-dimensional space and suggests that the Leap Motion controller can be implemented in some domains. However, the criticality and frequency of errors should be carefully considered

How to keep drivers engaged while supervising driving automation? A literature survey and categorization of six solution areas

This work aimed to organise recommendations for keeping people engaged during human supervision of driving automation, encouraging a safe and acceptable introduction of automated driving systems. First, heuristic knowledge of human factors, ergonomics, and psychological theory was used to propose solution areas to human supervisory control problems of sustained attention. Driving and non-driving research examples were drawn to substantiate the solution areas. Automotive manufacturers might (1) avoid this supervisory role altogether, (2) reduce it in objective ways or (3) alter its subjective experiences, (4) utilize conditioning learning principles such as with gamification and/or selection/training techniques, (5) support internal driver cognitive processes and mental models and/or (6) leverage externally situated information regarding relations between the driver, the driving task, and the driving environment. Second, a cross-domain literature survey of influential human-automation interaction research was conducted for how to keep engagement/attention in supervisory control. The solution areas (via numeric theme codes) were found to be reliably applied from independent rater categorisations of research recommendations. Areas (5) and (6) were addressed by around 70% or more of the studies, areas (2) and (4) in around 50% of the studies, and areas (3) and (1) in less than around 20% and 5%, respectively. The present contribution offers a guiding organisational framework towards improving human attention while supervising driving automation.submittedVersio

Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version

Human-Machine Interfaces for Service Robotics

L'abstract è presente nell'allegato / the abstract is in the attachmen

An Analysis of Heterogeneity in Futuristic Unmanned Vehicle Systems

Recent studies have shown that with appropriate operator decision support and with enough automation aboard unmanned vehicles, inverting the multiple operators to single-vehicle control paradigm is possible. These studies, however, have generally focused on homogeneous teams of vehicles, and have not completely addressed either the manifestation of heterogeneity in vehicle teams, or the effects of heterogeneity on operator capacity. An important implication of heterogeneity in unmanned vehicle teams is an increase in the diversity of possible team configurations available for each operator, as well as an increase in the diversity of possible attention allocation schemes that can be utilized by operators. To this end, this paper introduces a resource allocation framework that defines the strategies and processes that lead to alternate team configurations. The framework also highlights the sub-components of operator attention allocation schemes that can impact overall performance when supervising heterogeneous unmanned vehicle teams. A subsequent discrete event simulation model of a single operator supervising multiple heterogeneous vehicles and tasks explores operator performance under different heterogeneous team compositions and varying attention allocation strategies. Results from the discrete event simulation model show that the change in performance when switching from a homogeneous team to a heterogeneous one is highly dependent on the change in operator utilization. Heterogeneous teams that result in lower operator utilization can lead to improved performance under certain operator strategies.Prepared for Charles River Analytic

A Hierarchical Hybrid Architecture for Mission-Oriented Robot Control

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-03413-3_26In this work is presented a general architecture for a multi physical agent network system based on the coordination and the behaviour management. The system is organised in a hierarchical structure where are distinguished the individual agent actions and the collective ones linked to the whole agent network. Individual actions are also organised in a hybrid layered system that take advantages from reactive and deliberative control. Sensing system is involved as well in the behaviour architecture improving the information acquisition performance.This work has been partially supported by the Spanish Ministry of Economy and Competitiveness under the CICYT project Mission Based Control (COBAMI): DPI2011-28507-C02-02, under coordinated project High Integrity Partitioned Embedded Systems (Hi-PartES): TIN2011-28567-C03-03, and under the collaborative research project supported by the European Union MultiPARTES Project: FP7-ICT 287702. 2011-14.Muñoz Alcobendas, M.; Munera Sánchez, E.; Blanes Noguera, F.; Simó Ten, JE. (2013). A Hierarchical Hybrid Architecture for Mission-Oriented Robot Control. En ROBOT2013: First Iberian Robotics Conference: Advances in Robotics, Vol. 1. Springer. 363-380. https://doi.org/10.1007/978-3-319-03413-3_26S363380Aragues, R.: Consistent data association in multi-robot systems with limited communications. Robotics: Science and Systems, 97–104 (2010)Aragues, R., Cortes, J., Sagues, C.: Distributed consensus on robot networks for dynamically merging feature-based maps. IEEE Transactions on Robotics (2012)Arkin, R.C.: Motor schema based mobile robot navigation. The International Journal of Robotics Research 8(4), 92–112 (1989)Asama, H., Habib, M.K., Endo, I., Ozaki, K., Matsumoto, A., Ishida, Y.: Functional distribution among multiple mobile robots in an autonomous and decentralized robot system. In: Proceedings of the 1991 IEEE International Conference on Robotics and Automation. IEEE (1991)Benet, G., Blanes, F., Martínez, M., Simó, J.: A multisensor robot distributed architecture. In: IFAC Conference INCOM 1998 (1998)Brooks, R.: A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation 2(1), 14–23 (1986)Canas, J.M., Matellán, V.: Dynamic schema hierarchies for an autonomous robot. In: Garijo, F.J., Riquelme, J.-C., Toro, M. (eds.) IBERAMIA 2002. LNCS (LNAI), vol. 2527, pp. 903–912. Springer, Heidelberg (2002)Choset, H., Nagatani, K.: Topological simultaneous localization and mapping (SLAM): toward exact localization without explicit localization. IEEE Transactions on Robotics and Automation 17(2), 125–137 (2001)Fox, D., Burgard, W., Dellaert, F., Thrun, S.: Monte carlo localization: Efficient position estimation for mobile robots. American Association for Artificial Intelligence, 343–349 (1999)Hu, J., Xie, L., Xu, J.: Vision-based multi-agent cooperative target search. In: Control Automation Robotics & Vision (ICARCV), pp. 895–900 (2012)Huq, R., Mann, G.K.I., Gosine, R.G.: Behavior-modulation technique in mobile robotics using fuzzy discrete event system. IEEE Transactions on Robotics 22(5), 903–916 (2006)Jayasiri, A., Mann, G., Gosine, R.G.: Mobile robot behavior coordination using supervisory control of fuzzy discrete event systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2009, pp. 690–695 (2009)Jayasiri, A., Mann, G.K.I., Gosine, R.G.: Behavior coordination of mobile robotics using supervisory control of fuzzy discrete event systems. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 41(5), 1224–1238 (2011)Koenig, N., Howard, A.: Gazebo-3d multiple robot simulator with dynamics. Technical report (2006)Lin, F., Ying, H.: Modeling and control of fuzzy discrete event systems. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 32(4), 408–415 (2002)Madden, J.D.: Multi-robot system based on model of wolf hunting behavior to emulate wolf and elk interactions. In: 2010 IEEE International Conference on Robotics and Biomimetics, ROBIO, pp. 1043–1050 (2010)Mataric, M.J.: Interaction and intelligent behavior. Technical report, DTIC Document (1994)Olivera, V.M., Molina, J.M., Sommaruga, L., et al.: Fuzzy cooperation of autonomous robots. In: Fourth International System on Intelligent Robotics Systems, Lisboa, Portugal (1996)McGann, C., Py, F., Rajan, K., Thomas, H., Henthorn, R., McEwen, R.: A deliberative architecture for auv control. In: IEEE International Conference on Robotics and Automation, ICRA 2008, pp. 1049–1054. IEEE (2008)Munera, E., Muñoz, M., Simó, J., Blanes, F.: Humanoid Robot Self-Location In SPL League. In: Comité Español de Automática (CEA), XXXIII Jornadas de Automatica, 797–804 (2012)Proetzsch, M., Luksch, T., Berns, K.: Development of complex robotic systems using the behavior-based control architecture iB2C. Robotics and Autonomous Systems 58(1), 46–67 (2010)Qiu, D.: Supervisory control of fuzzy discrete event systems: a formal approach. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 35(1), 72–88 (2005)Aladebaran Robotics. NAO Software Documentation 1.12. Technical report (2012)St-Pierre, M., Gingras, D.: Comparison between the unscented Kalman filter and the extended Kalman filter for the position estimation module of an integrated navigation information system. In: 2004 IEEE Intelligent Vehicles Symposium, pp. 831–835 (2004)Stoytchev, A., Arkin, R.C.: Combining deliberation, reactivity, and motivation in the context of a behavior-based robot architecture. In: Proceedings of the 2001 IEEE International Symposium on Computational Intelligence in Robotics and Automation, pp. 290–295 (2001)Nicolau, V., Muñoz, M., Simó, J.: KertrolBot Platform. SiDiReLi: Distributed System with Limited Resources. Technical report, Institute of Control Systems and Industrial Computing - Polytechnic University of Valencia, Valencia, Spain (2011