Robust Localization and Efficient Path Planning for Mobile Sensor Networks

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2016. 2. 오성회.The area of wireless sensor networks has flourished over the past decade due to advances in micro-electro-mechanical sensors, low power communication and computing protocols, and embedded microprocessors. Recently, there has been a growing interest in mobile sensor networks, along with the development of robotics, and mobile sensor networks have enabled networked sensing system to solve the challenging issues of wireless sensor networks by adding mobility into many different applications of wireless sensor networks. Nonetheless, there are many challenges to be addressed in mobile sensor networks. Among these, the estimation for the exact location is perhaps the most important to obtain high fidelity of the sensory information. Moreover, planning should be required to send the mobile sensors to sensing location considering the region of interest, prior to sensor placements. These are the fundamental problems in realizing mobile sensor networks which is capable of performing monitoring mission in unstructured and dynamic environment. In this dissertation, we take an advantage of mobility which mobile sensor networks possess and develop localization and path planning algorithms suitable for mobile sensor networks. We also design coverage control strategy using resource-constrained mobile sensors by taking advantages of the proposed path planning method. The dissertation starts with the localization problem, one of the fundamental issue in mobile sensor networks. Although global positioning system (GPS) can perform relatively accurate localization, it is not feasible in many situations, especially indoor environment and costs a tremendous amount in deploying all robots equipped with GPS sensors. Thus we develop the indoor localization system suitable for mobile sensor networks using inexpensive robot platform. We focus on the technique that relies primarily on the camera sensor. Since it costs less than other sensors, all mobile robots can be easily equipped with cameras. In this dissertation, we demonstrate that the proposed method is suitable for mobile sensor networks requiring an inexpensive off-the-shelf robotic platform, by showing that it provides consistently robust location information for low-cost noisy sensors. We also focus on another fundamental issue of mobile sensor networks which is a path planning problem in order to deploy mobile sensors in specific locations. Unlike the traditional planning methods, we present an efficient cost-aware planning method suitable for mobile sensor networks by considering the given environment, where it has environmental parameters such as temperature, humidity, chemical concentration, stealthiness and elevation. A global stochastic optimization method is used to improve the efficiency of the sampling based planning algorithm. This dissertation presents the first approach of sampling based planning using global tree extension. Based on the proposed planning method, we also presents a general framework for modeling a coverage control system consisting of multiple robots with resource constraints suitable for mobile sensor networks. We describe the optimal informative planning methods which deal with maximization problem with constraints using global stochastic optimization method. In addition, we describe how to find trajectories for multiple robots efficiently to estimate the environmental field using information obtained from all robots.Chapter 1 Introduction 1 1.1 Mobile Sensor networks 1 1.1.1 Challenges 3 1.2 Overview of the Dissertation 4 Chapter 2 Background 7 2.1 Localization in MSNs 7 2.2 Path planning in MSNs 10 2.3 Informative path planning in MSNs 12 Chapter 3 Robust Indoor Localization 15 3.1 An Overview of Coordinated Multi-Robot Localization 16 3.2 Multi-Robot Localization using Multi-View Geometry 19 3.2.1 Planar Homography for Robot Localization 20 3.2.2 Image Based Robot Control 21 3.3 Multi-Robot Navigation System 25 3.3.1 Multi-Robot System 26 3.3.2 Multi-Robot Navigation 30 3.4 Experimental Results 32 3.4.1 Coordinated Multi-Robot Localization: Single-Step 32 3.4.2 Coordinated Multi-Robot Localization: Multi-Step 36 3.5 Discussions and Comparison to Leap-Frog 42 3.5.1 Discussions 42 3.5.2 Comparison to Leap-Frog 45 3.6 Summary 51 Chapter 4 Preliminaries to Cost-Aware Path Planning 53 4.1 Related works 54 4.2 Sampling based path planning 56 4.3 Cross entropy method 59 4.3.1 Cross entropy based path planning 63 Chapter 5 Fast Cost-Aware Path Planning using Stochastic Optimization 65 5.1 Problem formulation 66 5.2 Issues with sampling-based path planning for complex terrains or high dimensional spaces 68 5.3 Cost-Aware path planning (CAPP) 73 5.3.1 CE Extend 75 5.4 Analysis of CAPP 81 5.4.1 Probabilistic Completeness 81 5.4.2 Asymptotic optimality 83 5.5 Simulation and experimental results 84 5.5.1 (P1) Cost-Aware Navigation in 2D 85 5.5.2 (P2) Complex Terrain Navigation 88 5.5.3 (P3) Humanoid Motion Planning 96 5.6 Summary 103 Chapter 6 Effcient Informative Path Planning 105 6.1 Problem formulation 106 6.2 Cost-Aware informative path planning (CAIPP) 109 6.2.1 Overall procedure 110 6.2.2 Update Bound 112 6.2.3 CE Estimate 115 6.3 Analysis of CAIPP 118 6.4 Simulation and experimental results 120 6.4.1 Single robot informative path planning 120 6.4.2 Multi robot informative path planning 122 6.5 Summary 125 Chapter 7 Conclusion and Future Work 129 Appendices 131 Appendix A Proof of Theorem 1 133 Appendix B Proof of Theorem 2 135 Appendix C Proof of Theorem 3 137 Appendix D Proof of Theorem 4 139 Appendix E Dubins' curve 141 Bibliography 147 초 록 163Docto

An Olfactory-Based Robot Swarm Navigation Method

This paper presents a novel robot swarming navigation algorithm in order to find the odor sources in an unknown environment, based on the ability of each swarm member to sense the odor. Each robot in the swarm has a cooperative localization system which uses wireless network as a mean of measuring the distance from the other robots. In this method, at least three robots act as stationary measurement beacons while the other robots of the swarm navigate in the environment towards the odor source. In the next step, the roles of the robots will be switched and some other robots will act as beacons. The experimental tests report a good result in finding the odor source and also the accuracy of localization system

Active Collaborative Localization in Heterogeneous Robot Teams

Accurate and robust state estimation is critical for autonomous navigation of robot teams. This task is especially challenging for large groups of size, weight, and power (SWAP) constrained aerial robots operating in perceptually-degraded GPS-denied environments. We can, however, actively increase the amount of perceptual information available to such robots by augmenting them with a small number of more expensive, but less resource-constrained, agents. Specifically, the latter can serve as sources of perceptual information themselves. In this paper, we study the problem of optimally positioning (and potentially navigating) a small number of more capable agents to enhance the perceptual environment for their lightweight,inexpensive, teammates that only need to rely on cameras and IMUs. We propose a numerically robust, computationally efficient approach to solve this problem via nonlinear optimization. Our method outperforms the standard approach based on the greedy algorithm, while matching the accuracy of a heuristic evolutionary scheme for global optimization at a fraction of its running time. Ultimately, we validate our solution in both photorealistic simulations and real-world experiments. In these experiments, we use lidar-based autonomous ground vehicles as the more capable agents, and vision-based aerial robots as their SWAP-constrained teammates. Our method is able to reduce drift in visual-inertial odometry by as much as 90%, and it outperforms random positioning of lidar-equipped agents by a significant margin. Furthermore, our method can be generalized to different types of robot teams with heterogeneous perception capabilities. It has a wide range of applications, such as surveying and mapping challenging dynamic environments, and enabling resilience to large-scale perturbations that can be caused by earthquakes or storms.Comment: To appear in Robotics: Science and Systems (RSS) 202

Leader-assisted localization approach for a heterogeneous multi-robot system

This thesis presents the design, implementation, and validation of a novel leader assisted localization framework for a heterogeneous multi-robot system (MRS) with sensing and communication range constraints. It is assumed that the given heterogeneous MRS has a more powerful robot (or group of robots) with accurate self localization capabilities (leader robots) while the rest of the team (child robots), i.e. less powerful robots, is localized with the assistance of leader robots and inter-robot observation between teammates. This will eventually pose a condition that the child robots should be operated within the sensing and communication range of leader robots. The bounded navigation space therefore may require added algorithms to avoid inter-robot collisions and limit robots’ maneuverability. To address this limitation, first, the thesis introduces a novel distributed graph search and global pose composition algorithm to virtually enhance the leader robots’ sensing and communication range while avoiding possible double counting of common information. This allows child robots to navigate beyond the sensing and communication range of the leader robot, yet receive localization services from the leader robots. A time-delayed measurement update algorithm and a memory optimization approach are then integrated into the proposed localization framework. This eventually improves the robustness of the algorithm against the unknown processing and communication time-delays associated with the inter-robot data exchange network. Finally, a novel hierarchical sensor fusion architecture is introduced so that the proposed localization scheme can be implemented using inter-robot relative range and bearing measurements. The performance of the proposed localization framework is evaluated through a series of indoor experiments, a publicly available multi-robot localization and mapping data-set and a set of numerical simulations. The results illustrate that the proposed leader-assisted localization framework is capable of establishing accurate and nonoverconfident localization for the child robots even when the child robots operate beyond the sensing and communication boundaries of the leader robots

Navigational Strategies for Control of Underwater Robot using AI based Algorithms

Autonomous underwater robots have become indispensable marine tools to perform various tedious and risky oceanic tasks of military, scientific, civil as well as commercial purposes. To execute hazardous naval tasks successfully, underwater robot needs an intelligent controller to manoeuver from one point to another within unknown or partially known three-dimensional environment. This dissertation has proposed and implemented various AI based control strategies for underwater robot navigation. Adaptive versions of neuro-fuzzy network and several stochastic evolutionary algorithms have been employed here to avoid obstacles or to escape from dead end situations while tracing near optimal path from initial point to destination of an impulsive underwater scenario. A proper balance between path optimization and collision avoidance has been considered as major aspects for evaluating performances of proposed navigational strategies of underwater robot. Online sensory information about position and orientation of both target and nearest obstacles with respect to the robot’s current position have been considered as inputs for path planners. To validate the feasibility of proposed control algorithms, numerous simulations have been executed within MATLAB based simulation environment where obstacles of different shapes and sizes are distributed in a chaotic manner. Simulation results have been verified by performing real time experiments of robot in underwater environment. Comparisons with other available underwater navigation approaches have also been accomplished for authentication purpose. Extensive simulation and experimental studies have ensured the obstacle avoidance and path optimization abilities of proposed AI based navigational strategies during motion of underwater robot. Moreover, a comparative study has been performed on navigational performances of proposed path planning approaches regarding path length and travel time to find out most efficient technique for navigation within an impulsive underwater environment

Coverage Path Planning with Real‐time Replanning and Surface Reconstruction for Inspection of Three‐dimensional Underwater Structures using Autonomous Underwater Vehicles

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/113717/1/rob21554.pd

Image-Aided Navigation Using Cooperative Binocular Stereopsis

This thesis proposes a novel method for cooperatively estimating the positions of two vehicles in a global reference frame based on synchronized image and inertial information. The proposed technique - cooperative binocular stereopsis - leverages the ability of one vehicle to reliably localize itself relative to the other vehicle using image data which enables motion estimation from tracking the three dimensional positions of common features. Unlike popular simultaneous localization and mapping (SLAM) techniques, the method proposed in this work does not require that the positions of features be carried forward in memory. Instead, the optimal vehicle motion over a single time interval is estimated from the positions of common features using a modified bundle adjustment algorithm and is used as a measurement in a delayed state extended Kalman filter (EKF). The developed system achieves improved motion estimation as compared to previous work and is a potential alternative to map-based SLAM algorithms

A review on multi-robot systems: current challenges for operators and new developments of interfaces

[ES] Los sistemas multi-robot están experimentando un gran desarrollo en los últimos tiempos, ya que mejoran el rendimiento de las misiones actuales y permiten realizar nuevos tipos de misiones. Este artículo analiza el estado del arte de los sistemas multi-robot, abordando un conjunto de temas relevantes: misiones, flotas, operadores, interacción humano-sistema e interfaces. La revisión se centra en los retos relacionados con factores humanos como la carga de trabajo o la conciencia de la situación, así como en las propuestas de interfaces adaptativas e inmersivas para solucionarlos.[EN] Multi-robot systems are experiencing great development in recent times, since they are improving the performance of current missions and allowing new types of missions. This article analyzes the state of the art of multi-robot systems, addressing a set of relevant topics: missions, fleets, operators, human-system interaction and interfaces. The review focuses on the challenges related to human factors such as workload and situational awareness, as well as the proposals of adaptive and immersive interfaces to solve them.Esta investigación ha recibido fondos de los proyectos SAVIER (Situational Awareness VIrtual EnviRonment) de Airbus; RoboCity2030-DIH-CM, Madrid Robotics Digital Innovation Hub, S2018/ NMT-4331, financiado por los Programas de Actividades I+D de la Comunidad de Madrid y confinanciado por los Fondos Estructurales de la UE; y DPI2014-56985-R (Protección Robotizada de Infraestructuras Críticas) financiado por el ministerio de Economía y Competitividad del Gobierno de España.Roldan-Gómez, JJ.; De León Rivas, J.; Garcia-Aunon, P.; Barrientos, A. (2020). Una revisión de los sistemas multi-robot: desafíos actuales para los operadores y nuevos desarrollos de interfaces. Revista Iberoamericana de Automática e Informática industrial. 17(3). https://doi.org/10.4995/riai.2020.13100OJS305173Abbadi, A. and Prenosil., 2015. Safe path planning using cell decomposition approximation. Distance Learning, Simulation and Communication, 8.Adams, B. and Suykens, F., 2013 Astute: Increased Situational Awareness through proactive decision support and adaptive map-centric user interfaces. 2013 IEEE European Intelligence and Security Informatics Conference (EISIC), 289-293. https://doi.org/10.1109/EISIC.2013.74Almeida, L., Menezes, P. and Dias, J., 2017. Improving robot teleoperation experience via immersive interfaces. In 2017 4th IEEE Experiment@ International Conference (exp. at'17), 87-92. https://doi.org/10.1109/EXPAT.2017.7984414Arnold, K. P., 2016. The UAV Ground Control Station: Types, Components, Safety, Redundancy, and Future Applications. International Journal of Unmanned Systems Engineering., 4(1), 37.Ayaz, H., Shewokis, P. A., Bunce, S., Izzetoglu, K., Willems, B. and Onaral, B., 2012. Optical brain monitoring for operator training and mental workload assessment. Neuroimage, 59(1), 36-47. https://doi.org/10.1016/j.neuroimage.2011.06.023Beer, J., Fisk, A. D. and Rogers, W. A., 2014. Toward a framework for levels of robot autonomy in human-robot interactions. Journal of Human-Robot Interaction, 3(2), 74. https://doi.org/10.5898/JHRI.3.2.BeerBourguet, M. L., 2003. Designing and Prototyping Multimodal Commands. Interact, 3, 717-720.Brutschy, A., Pini, G., Pinciroli, C., Birattari, M. and Dorigo, M., 2014. Self-organized task allocation to sequentially interdependent tasks in swarm robotics. Autonomous agents and multi-agent systems, 28(1), 101-125. https://doi.org/10.1007/s10458-012-9212-yCantelli, L., Mangiameli, M., Melita, C. D. and Muscato, G., 2013. UAV/UGV cooperation for surveying operations in humanitarian demining. 2013 IEEE international symposium on safety, security, and rescue robotics (SSRR),1-6). https://doi.org/10.1109/SSRR.2013.6719363Chang, H. and Jin, T., 2013. Command fusion based fuzzy controller design for moving obstacle avoidance of mobile robot. Future information communication technology and applications, Springer, 905-913. https://doi.org/10.1007/978-94-007-6516-0_99Chen, J. Y. C., Haas, E. C. and Barnes, M. J., 2007. Human performance issues and user interface design for teleoperated robots. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 37(6), 1231-1245. https://doi.org/10.1109/TSMCC.2007.905819Chen, J. Y. C., Barnes, M. J. and Harper-Sciarini, M., 2011. Supervisory control of multiple robots: Human-performance issues and user-interface design. IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications an Reviews), 41(4), 435-454. https://doi.org/10.1109/TSMCC.2010.2056682Clark, C. M., 2005. Probabilistic road map sampling strategies for multi-robot motion planning. Robotics and Autonomous Systems, 53(3), 244-264. https://doi.org/10.1016/j.robot.2005.09.002Cummings, M. L., Bruni, S., Mercier, S. and Mitchell, P. J., 2007. Automation architecture for single operator, multiple UAV command and control. Tech. Rep DTIC Document.Cummings, M. L. and Mitchell P. J., 2008. Predicting controller capacity in supervisory control of multiple UAVs. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 38(2), 451-460. https://doi.org/10.1109/TSMCA.2007.914757Cummings, M. L., Mastracchio, C., Thornburg, K. M. and Mkrtchyan, A., 2013. Boredom and distraction in multiple unmanned vehicle supervisory control. Interacting with computers, 25(1), 34-47. https://doi.org/10.1093/iwc/iws011De Cubber, G., Doroftei, D., Serrano, D., Chintamani, K., Sabino, R. and Ourevitch, S., 2013. The EU-ICARUS project: developing assistive robotic tools for search and rescue operations. 2013 IEEE international symposium on safety, security, and rescue robotics (SSRR), 1-4. https://doi.org/10.1109/SSRR.2013.6719323De Greeff, J., Hindriks, K., Neerincx, M. A. and Kruijff-Korbayova, I., 2015. Human-robot teamwork in USAR environments: the TRADR project. Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts, pp. 151-152. https://doi.org/10.1145/2701973.2702031Dezfoulian, S. H., Wu, D. and Ahmad, I. S., 2013. A generalized neural network approach to mobile robot navigation and obstacle avoidance. Intelligent Autonomous Systems, Springer, 12, 25-52. https://doi.org/10.1007/978-3-642-33926-4_3Di, B., Zhou, R. and Duan, H., 2015. Potential field based receding horizon motion planning for centrality-aware multiple UAV cooperative surveillance. Aerospace Science and Technology, 46, 386-397. https://doi.org/10.1016/j.ast.2015.08.006Dixon, S. R., Wickens, C. D. and Chang, D, 2005. Mission control of multiple unmanned aerial vehicles: A workload analysis. Human Factors: The Journal of the Human Factors and Ergonomics Society 47(3), 479-487. https://doi.org/10.1518/001872005774860005Donmez, B., Nehme, C. and Cummings, M.L., 2010. Modeling workload impact in multiple unmanned vehicle supervisory control. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 40(6), 1180-1190. https://doi.org/10.1109/TSMCA.2010.2046731Drury, J. L., Scholtz, J. and Yanco, H.A., 2003. Awareness in human-robot interactions. IEEE International Conference on Systems, Man and Cybernetics, 1, 912-918.Drury, J. L., Riek, L. and Rackliffe, N., 2006A. A decomposition of UAV-related situation awareness. Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, 88-94. https://doi.org/10.1145/1121241.1121258Drury, J. L., Richer, J., Rackliffe, N. and Goodrich, M. A., 2006B. Comparing situation awareness for two unmanned aerial vehicle human interfaces approaches. Technical rpeort, Mitre Corp Bedford MA.Endsley, M. R., 1988A. Design and evaluation for situation awareness enhancement. Proceedings of the human factors and ergonomics society annual meeting, SAGE Publications, 32(2), 97-101. https://doi.org/10.1177/154193128803200221Endsley, M. R., 1988B. Situation awareness global assessment technique (SAGAT). Proceedings of the IEEE 1988 National Aerospace and Electronics Conference (NAECON), 789-795.Endsley, M. R. and Garland, D. J., 2000. Situation awareness analysis and measurement. CRC Press. https://doi.org/10.1201/b12461Endsley M. R., 1999. Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics, 42(3), 462-492. https://doi.org/10.1080/001401399185595Flushing, E. F., Gambardella, L. and Di Caro, G. A., 2012. Gis-based mission support system for wilderness search and rescue with heterogeneous agents. Proc 2nd Workshop on robots and sensors integration in future rescue INformation system (ROSIN), IEEE/RSJ Int Conference on Intelligent Robots and Systems (IROS).Foit, K., 2014. Mixed reality as a tool supporting programming of the robot. Advanced Materials Research, Trans Tech Publications, 1036, 737-742. https://doi.org/10.4028/www.scientific.net/AMR.1036.737Frische, F. and Lüdtke, A., 2013. SA-tracer: A tool for assessment of UAV swarm operator SA during mission execution. 2013 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), 203-211. https://doi.org/10.1109/CogSIMA.2013.6523849Fuchs, C., Borst, C., de Croon, G. C., Van Paassen, M. M. and Mulder, M., 2014. An ecological approach to the supervisory control of UAV swarms. International Journal of Micro Air Vehicles, 6(4), 211-229. https://doi.org/10.1260/1756-8293.6.4.211Galceran, E. and Carreras, M., 2013. A survey on coverage path planning for robotics. Robotics and Autonomous Systems, 61(12), 1258-1276. https://doi.org/10.1016/j.robot.2013.09.004García, J. C., Patrão, B., Pérez, J., Seabra, J., Menezes, P., Dias, J. and Sanz, P.J., 2015. Towards an immersive and natural gesture controlled interface for intervention underwater robots. IEEE OCEANS 2015-Genova, 1-5. https://doi.org/10.1109/OCEANS-Genova.2015.7271501García, S. E., Slawiñski, E., Mut, V., and Penizzotto, F., 2018. Collision avoidance method for multi-operator multi-robot teleoperation system. Robotica, 36(1), 78-95. https://doi.org/10.1017/S0263574717000169Garzón, M., Valente, J., Zapata, D. and Barrientos, A., 2013. An aerial-ground robotic system for navigation and obstacle mapping in large outdoor areas. Sensors, 13(1), 1247-1267. https://doi.org/10.3390/s130101247Garzón, M., Valente, J., Roldán, J. J., Cancar, L., Barrientos, A. and Del Cerro, J., 2016. A multirobot system for distributed area coverage and signal searching in large outdoor scenarios. Journal of Field Robotics, 33(8), 1087-1106. https://doi.org/10.1002/rob.21636Garzón, M., Valente, J., Roldán, J. J., Garzón-Ramos, D., de León, J., Barrientos, A. and del Cerro, J., 2017. Using ros in multi-robot systems: Experiences and lessons learned from real-world field tests. In Robot Operating System (ROS), Springer, Cham, 449-483. https://doi.org/10.1007/978-3-319-54927-9_14Goerzen, C., Kong, Z. and Mettler, B., 2009. A survey of motion planning algorithms from the perspective of autonomous UAV guidance. Selected papers from the 2nd International Symposium on UAVs, Reno, Nevada, USA, Springer, 64-100. https://doi.org/10.1007/978-90-481-8764-5_5Goodrich, M. A. and Schultz, A. C., 2007. Human-robot interaction: a survey. Foundations and trends in human-computer interaction, 1(3), 203-275. https://doi.org/10.1561/1100000005Gregory, J., Fink, J., Stump, E., Twigg, J., Rogers, J., Baran, D., Fung, N. and Young, S., 2016. Application of multi-robot systems to disaster-relief scenarios with limited communication. In Field and Service Robotics, Springer, Cham, 639-653. https://doi.org/10.1007/978-3-319-27702-8_42Haas, E. C., Pillalamarri, K., Stachowiak, C. C. and Fields, M., 2011. Multimodal controls for soldier/swarm interaction. IEEE RO-MAN, 223-228. https://doi.org/10.1109/ROMAN.2011.6005227Hagiwara, Y., 2015. Cloud based VR system with immersive interfaces to collect multimodal data in human-robot interaction. 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE), 256-259. https://doi.org/10.1109/GCCE.2015.7398709Hart, S. G. and Staveland E.L., 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology, 52, 139-183. https://doi.org/10.1016/S0166-4115(08)62386-9Hart, S. G., 2006. NASA-task load index (NASA-TLX); 20 years later. Proceedings of the human factors and ergonomics society annual meeting, Sage Publications Sage CA, Los Angeles, CA, 50(9), 904-908. https://doi.org/10.1177/154193120605000909Hobbs, A. and Herwitz, S. R., 2014. Human factors in the maintenance of unmanned aerial vehicle (UAV) swarm management. Applied Ergonomics, 58, 66-80. https://doi.org/10.1016/j.apergo.2016.05.011Hocraffer, A. and Nam, C. S. A meta-analysis of human-system interfaces in unmanned aerial vehicle (UAV) swarm management. Applied Ergonomics, 58, 66-80. https://doi.org/10.1016/j.apergo.2016.05.011Hong, A., 2016. Human-Robot Interactions for Single Robots and Multi-Robot Teams. PhD thesis, Department of Mechanical and Industrial Engineering, University of Toronto.Janchiv, A., Batsaikhan, D., hwan Kim, G. and Lee, S. G., 2011. Complete coverage path planning for multi-robots based on. 2011 11th IEEE International Conference on Control, Automation and Systems, 824-827.Jasper, P., Sibley, C. and Coyne, J. Using Heart Rate Variability to Assess Operator Mental Workload in a Command and Control Simulation of Multiple Unmanned Aerial Vehicles. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, SAGE Publications Sage CA, Los Angeles, CA, 60(1), 1125-1129. https://doi.org/10.1177/1541931213601264Jia, X. and Meng, M. Q. H., 2013. A survey and analysis of task allocation algorithms in multi-robot systems. 2013 IEEE International Conference on Robotics and biomimetics (ROBIO), 2280-2285. https://doi.org/10.1109/ROBIO.2013.6739809Jiang, Y., 2016. A survey of task allocation and load balancing in distributed systems. IEEE Transactions on Parallel and Distributed Systems, 27(2), 585-599. https://doi.org/10.1109/TPDS.2015.2407900Johannsmeier, L. and Haddadin, S., 2016. A hierarchical human-robot interaction-planning framework for task allocation in collaborative industrial assembly processes. IEEE Robotics and Automation Letters, 2(1), 41-48. https://doi.org/10.1109/LRA.2016.2535907Kapoutsis, A. C., Chatzichristofis, S. A., Doitsidis, L., de Sousa, J. B., Pinto, J., Braga, J. and Kosmatopoulos, E. B., 2016. Real-time adaptive multi-robot exploration with application to underwater map construction. Autonomous robots, 40(6), 987-1015. https://doi.org/10.1007/s10514-015-9510-8Kavitha, S., Veena, S. and Kumaraswamy, R., 2015. Development of automatic speech recognition system for voice activated Ground Control system. 2015 IEEE International Conference on Trends in Automation, Communications and Computing Technology (I-TACT-15), pp. 1-5. https://doi.org/10.1109/ITACT.2015.7492684Khaleghi, A. M., Xu, D., Minaeian, S., Li, M., Yuan, Y., Liu, J., Son, Y. J., Vo, C., Mousavian, A. and Lien, J. M., 2014. A comparative study of control architectures in UAV/UGV-based surveillance system. Proceedings of the IIE Annual Conference, Institute of Industrial and Systems Engineers (IISE), 3455.Khamis, A., Hussein, A. and Elmogy, A., 2015. Multi-robot Task Allocation: A Review of the State-of-the-Art. Cooperative Robots and Sensor Networks, Springer, 31-51. https://doi.org/10.1007/978-3-319-18299-5_2Kirchner, E. A., Kim, S. K., Tabie, M., Wöhrle, H., Maurus, M. and Kirchner, F., 2016. An intelligent man-machine interface - Multi-robot control adapted for task engagement based on single-trial detectability of P300. Frontiers in human neuroscience, 10, 291. https://doi.org/10.3389/fnhum.2016.00291Kolling, A., Nunnally, S. and Lewis, M., 2012. Towards human control of robot swarms. Proceedings of the seventh annual ACM/IEEE international conference on human-robot interaction, 89-96. https://doi.org/10.1145/2157689.2157704Kothari, M., Postlethwaite, I. and Gu, D. W. Multi-UAV path planning in obstacle rich environment using rapidly-exploring random trees. Proceedings of the 48 IEEE Conference on Decision and Control, 3069-3074.Kruijff-Korbayová, I., Colas, F., Gianni, M., Pirri, F., de Greeff, J., Hindriks, K., Neerincx, M., Ögren, P., Svoboda, T. and Worst, R., 2015. Tradr project: Long-term human-robot teaming for robot assisted disaster response. KI-Künstliche Intelligenz, 29(2), 193-201. https://doi.org/10.1007/s13218-015-0352-5Kurniawan, H., Maslov, A. V. and Pechenizkiy, M., 2013. Stress detection from speech and galvanic skin response signals. IEEE 26th International Symposium on Computer-Based Medical Systems (CBMS), 209-214. https://doi.org/10.1109/CBMS.2013.6627790Lang, R. G., da Silva, I. N., Romero, R. A. F., 2014. Development of distributed control architecture for multi-robot systems. IEEE 2014 Joint Conference on Robotics: SBR-LARS Robotics Symposium and Robocontrol, 163-168. https://doi.org/10.1109/SBR.LARS.Robocontrol.2014.49Larochelle, B., Kruijff, G. J. M., Smets, N., Mioch, T. and Groenewegen, P., 2011. Establishing human situation awareness using a multi-modal operator control unit in an urban search & rescue human-robot team, IEEE, 229-234. https://doi.org/10.1109/ROMAN.2011.6005237Lesire, C., Infantes, G., Gateau, T. and Barbier, M., 2016. A distributed architecture for supervision of autonomous multi-robot missions. Autonomous Robots, 40(7), 1343-1362. https://doi.org/10.1007/s10514-016-9603-zLindemuth, M., Murphy, R., Steimle, E., Armitage, W., Dreger, K., Elliot, T., Hall, M., Kalyadin, D., Kramer, J., Palankar, M. and Pratt, K., 2011. Sea robot-assisted inspection. IEEE robotics & automation magazine, 18(2), 96-107. https://doi.org/10.1109/MRA.2011.940994Lysaght, R. J., 1989. Operator workload: Comprehensive review and evaluation of operator workload methodologies. Tech. Rep. DTIC Document. https://doi.org/10.21236/ADA212879Mantecón, T., del Blanco, C. R., Jaureguizar, F. and García, N., 2014. New generation of human machine interfaces for controlling UAV through depth-based gesture recognition. Unmanned Systems Technology XVI, International Society for Optics and Photonics, 9084. https://doi.org/10.1117/12.2053244Marino, A., Parker, L. E., Antonelli, G. and Caccavale, F., 2013. A decentralized architecture for multi-robot systems based on the null-space-behavioral control with application to multi-robot border patrolling. Journal of Intelligent & Robotic Systems, 71(3-4), 423-444. https://doi.org/10.1007/s10846-012-9783-5Martín-Barrio, A., Terrile, S., Barrientos, A. and del Cerro, J., 2018. Hyper-Redundant Robots: Classification, State-of-the-Art and Issues. Revista Iberoamericana de Automática e Informática Industrial, 15(4), 351-362. https://doi.org/10.4995/riai.2018.9207Martín-Barrio, A., Roldán, J. J., Terrile, S., del Cerro, J. and Barrientos, A., 2019. Application of Immersive Technologies and Natural Language to Hyper-Redundant Robot Teleoperation. Virtual Reality, 1-15. https://doi.org/10.1007/s10055-019-00414-9Martins, H., Oakley, I. and Ventura, R., 2015. Design and evaluation of a head-mounted display for immersive 3D teleoperation of field robots. Robotica, 33(10), 2166-2185. https://doi.org/10.1017/S026357471400126XMatellán, V. and Borrajo, D., 2001. ABC2 an agenda based multi-agent model for robots control and cooperation. Journal of Intelligent & Robotic Systems, 32(1), 93-114. https://doi.org/10.1023/A:1012009429991McCarley, J.S. and Wickens, C. D., 2005. Human factors implications of UAVs in the national airspace. University of Illinois at Urbana-Champaign, Aviation Human Factors Division.McDuff, D., Gontarek, S. and Picard, R., 2014. Remote measurement of cognitive stress via heart rate variability. 36th Annual International Conference on the IEEE Engineering in Medicine and Biology Society (EMBC), 2957-2960. https://doi.org/10.1109/EMBC.2014.6944243Menda, J., Hing, J. T., Ayaz, H., Shewokis, P. A., Izzetoglu, K., Onaral, B. and Oh, P., 2011. Optical brain imaging to enhance UAV operator training, evaluation, and interface development. Journal of intelligent & robotic systems, 61(1-4), 423-443. https://doi.org/10.1007/s10846-010-9507-7Monajjemi, V. M., Pourmehr, S., Sadat, S. A., Zhan, F., Wawerla, J., Mori, G. and Vaughan, R., 2014. Integrating multi-modal interfaces to command UAVs. Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, 106-106. https://doi.org/10.1145/2559636.2559646Moore, J., Wolfe, K. C., Johannes, M. S., Katyal, K. D., Para, M. P., Murphy, R. J., Hatch, J., Taylor, C. J., Bamberger, R. J. and Tunstel, E., 2016. Nested marsupial robotic system for search and sampling in increasingly constrained environments. 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2279-2286 https://doi.org/10.1109/SMC.2016.7844578Mosteo, A. R. and Montano, L., 2010. A survey of multi-robot task allocation. Technical Report No. AMI-009-10-TEC, Instituto de Investigación en Ingeniería de Aragón, University of Zaragoza, Zaragoza, Spain.Murphy, R. R., 2004. Human-robot interaction in rescue robotics. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 34(2), 138-153. https://doi.org/10.1109/TSMCC.2004.826267Nagi, J., Giusti, A., Di Caro, G. A. and Gambardella, L. M., 2014. Human control of UAVs using face pose estimates and hand gestures. 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 1-2. https://doi.org/10.1145/2559636.2559644Nam, C. S., Johnson, S., Li, Y. and Seong, Y., 2009. Evaluation of human-agent user interfaces in multi-agent systems. International Journal of Industrial Ergonomics, 39(1), 192-201. https://doi.org/10.1016/j.ergon.2008.08.008Nestmeyer, T., Giordano, P. R., Bülthoff, H. H. and Franchi, A., 201

Analyzing and improving multi-robot missions by using process mining

Multi-robot missions can be compared to industrial processes or public services in terms of complexity, agents and interactions. Process mining is an emerging discipline that involves process modeling, analysis and improvement through the information collected by event logs. Currently, this discipline is successfully used to analyze several types of processes, but is hardly applied in the context of robotics. This work proposes a systematic protocol for the application of process mining to analyze and improve multi-robot missions. As an example, this protocol is applied to a scenario of fire surveillance and extinguishing with a fleet of UAVs. The results show the potential of process mining in the analysis of multi-robot missions and the detection of problems such as bottlenecks and inefficiencies. This work opens the way to an extensive use of these techniques in multi-robot missions, allowing the development of future systems for optimizing missions, allocating tasks to robots, detecting anomalies or supporting operator decisions