Research with Collaborative Unmanned Aircraft Systems

We provide an overview of ongoing research which targets development of a principled framework for mixed-initiative interaction with unmanned aircraft systems (UAS). UASs are now becoming technologically mature enough to be integrated into civil society. Principled interaction between UASs and human resources is an essential component in their future uses in complex emergency services or bluelight scenarios. In our current research, we have targeted a triad of fundamental, interdependent conceptual issues: delegation, mixed- initiative interaction and adjustable autonomy, that is being used as a basis for developing a principled and well-defined framework for interaction. This can be used to clarify, validate and verify different types of interaction between human operators and UAS systems both theoretically and practically in UAS experimentation with our deployed platforms

Mixed Initiative Systems for Human-Swarm Interaction: Opportunities and Challenges

Human-swarm interaction (HSI) involves a number of human factors impacting human behaviour throughout the interaction. As the technologies used within HSI advance, it is more tempting to increase the level of swarm autonomy within the interaction to reduce the workload on humans. Yet, the prospective negative effects of high levels of autonomy on human situational awareness can hinder this process. Flexible autonomy aims at trading-off these effects by changing the level of autonomy within the interaction when required; with mixed-initiatives combining human preferences and automation's recommendations to select an appropriate level of autonomy at a certain point of time. However, the effective implementation of mixed-initiative systems raises fundamental questions on how to combine human preferences and automation recommendations, how to realise the selected level of autonomy, and what the future impacts on the cognitive states of a human are. We explore open challenges that hamper the process of developing effective flexible autonomy. We then highlight the potential benefits of using system modelling techniques in HSI by illustrating how they provide HSI designers with an opportunity to evaluate different strategies for assessing the state of the mission and for adapting the level of autonomy within the interaction to maximise mission success metrics.Comment: Author version, accepted at the 2018 IEEE Annual Systems Modelling Conference, Canberra, Australi

A Localized Autonomous Control Algorithm For Robots With Heterogeneous Capabilities In A Multi-Tier Architecture

This dissertation makes two contributions to the use of the Blackboard Architecture for command. The use of boundary nodes for data abstraction is introduced and the use of a solver-based blackboard system with pruning is proposed. It also makes contributions advancing the engineering design process in the area of command system selection for heterogeneous robotic systems. It presents and analyzes data informing decision making between centralized and distributed command systems and also characterizes the efficacy of pruning across different experimental scenarios, demonstrating when it is effective or not. Finally, it demonstrates the operations of the system, raising the technology readiness level (TRL) of the technology towards a level suitable for actual mission use. The context for this work is a multi-tier mission architecture, based on prior work by Fink on a “tier scalable” architecture. This work took a top-down approach where the superior tiers (in terms of scope of visibility) send specific commands to craft in lower tiers. While benefitting from the use of a large centralized processing center, this approach is limited in responding to failures and interference. The work presented herein has involved developing and comparatively characterizing centralized and decentralized (where superior nodes provide information and goals to the lower-level craft, but decisions are made locally) Blackboard Architecture based command systems. Blackboard Architecture advancements (a solver, pruning, boundary nodes) have been made and tested under multiple experimental conditions

MISSION-ORIENTED HETEROGENEOUS ROBOT COOPERATION BASED ON SMART RESOURCES EXECUTION

Home environments are changing as more technological devices are used to improve daily life. The growing demand for high technology in our homes means that robot integration will soon arrive. Home devices are evolving in a connected paradigm in which data flows to perform efficient home task management. Heterogeneous home robots connected in a network can establish a workflow that complements their capabilities and so increases performance within a mission execution. This work addresses the definition and requirements of a robot-group mission in the home context. The proposed solution relies on a network of smart resources, which are defined as cyber-physical systems that provide high-level service execution. Firstly, control middleware architecture is introduced as the execution base for the Smart resources. Next, the Smart resource topology and its integration within a robotic platform are addressed. Services supplied by Smart resources manage their execution through a robot behavior architecture. Robot behavior execution is hierarchically organized through a mission definition that can be established as an individual or collective approach. Environment model and interaction tasks characterize the operation capabilities of each robot within a mission. Mission goal achievement in a heterogeneous group is enhanced through the complement of the interaction capabilities of each robot. To offer a clearer explanation, a full use case is presented in which two robots cooperate to execute a mission and the previously detailed steps are evaluated. Finally, some of the obtained results are discussed as conclusions and future works is introduced.Los entornos domésticos se encuentran sometidos a un proceso de cambio gracias al empleo de dispositivos tecnológicos que mejoran la calidad de vida de las personas. La creciente demanda de alta tecnología en los hogares señala una próxima incorporación de la robótica de servicio. Los dispositivos domésticos están evolucionando hacia un paradigma de conexión en el cual la información fluye para ofrecer una gestión más eficiente. En este entorno, robots heterogéneos conectados a la red pueden establecer un flujo de trabajo que ofreciendo nuevas soluciones y incrementando la eficiencia en la ejecución de tareas. Este trabajo aborda la definición y los requisitos necesarios para la ejecución de misiones en grupos de robots heterogéneos en entornos domésticos. La solución propuesta se apoya en una red de Smart resources, que son definidos como sistemas ciber-físicos que proporcionan servicios de alto nivel. En primer lugar, se presenta la arquitectura del middleware de control en la cual se basa la ejecución de los Smart resources. A continuación se detalla la topología de los Smart resources, así como su integración en plataformas robóticas. Los servicios proporcionados por los Smart resources gestionan su ejecución mediante una arquitectura de comportamientos para robots. La ejecución de estos comportamientos se organiza de forma jerárquica mediante la definición de una misión con un objetivo establecido de forma individual o colectiva a un grupo de robots. Dentro de una misión, las tareas de modelado e interacción con el entorno define las capacidades de operación de los robots dentro de una misión. Mediante la integración de un grupo heterogéneo de robots sus diversas capacidades son complementadas para el logro un objetivo común. A fin de caracterizar esta propuesta, los mecanismos presentados en este documento se evaluarán en detalle a lo largo de una serie experimentos en los cuales un grupo de robots heterogéneos ejecutan una misión colaborativa para alcanzar un objetivo común. Finalmente, los resultados serán discutidos a modo de conclusiones dando lugar el establecimiento de un trabajo futuro.Els entorns domèstics es troben sotmesos a un procés de canvi gràcies a l'ocupació de dispositius tecnològics que milloren la qualitat de vida de les persones. La creixent demanda d'alta tecnologia a les llars assenyala una propera incorporació de la robòtica de servei. Els dispositius domèstics estan evolucionant cap a un paradigma de connexió en el qual la informació flueix per oferir una gestió més eficient. En aquest entorn, robots heterogenis connectats a la xarxa poden establir un flux de treball que ofereix noves solucions i incrementant l'eficiència en l'execució de tasques. Aquest treball aborda la definició i els requisits necessaris per a l'execució de missions en grups de robots heterogenis en entorns domèstics. La solució proposada es recolza en una xarxa de Smart resources, que són definits com a sistemes ciber-físics que proporcionen serveis d'alt nivell. En primer lloc, es presenta l'arquitectura del middleware de control en la qual es basa l'execució dels Smart resources. A continuació es detalla la tipologia dels Smart resources, així com la seva integració en plataformes robòtiques. Els serveis proporcionats pels Smart resources gestionen la seva execució mitjançant una arquitectura de comportaments per a robots. L'execució d'aquests comportaments s'organitza de forma jeràrquica mitjançant la definició d'una missió amb un objectiu establert de forma individual o col·lectiva a un grup de robots. Dins d'una missió, les tasques de modelatge i interacció amb l'entorn defineix les capacitats d'operació dels robots dins d'una missió. Mitjançant la integració d'un grup heterogeni de robots seves diverses capacitats són complementades per a l'assoliment un objectiu comú. Per tal de caracteritzar aquesta proposta, els mecanismes presentats en aquest document s'avaluaran en detall mitjançant d'una sèrie experiments en els quals un grup de robots heterogenis executen una missió col·laborativa per aconseguir un objectiu comú. Finalment, els resultats seran discutits a manera de conclusions donant lloc a l'establiment d'un treball futur.Munera Sánchez, E. (2017). MISSION-ORIENTED HETEROGENEOUS ROBOT COOPERATION BASED ON SMART RESOURCES EXECUTION [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/88404TESI

An Evaluation Schema for the Ethical Use of Autonomous Robotic Systems in Security Applications

We propose a multi-step evaluation schema designed to help procurement agencies and others to examine the ethical dimensions of autonomous systems to be applied in the security sector, including autonomous weapons systems

An architecture for adaptive task planning in support of IoT-based machine learning applications for disaster scenarios

The proliferation of the Internet of Things (IoT) in conjunction with edge computing has recently opened up several possibilities for several new applications. Typical examples are Unmanned Aerial Vehicles (UAV) that are deployed for rapid disaster response, photogrammetry, surveillance, and environmental monitoring. To support the flourishing development of Machine Learning assisted applications across all these networked applications, a common challenge is the provision of a persistent service, i.e., a service capable of consistently maintaining a high level of performance, facing possible failures. To address these service resilient challenges, we propose APRON, an edge solution for distributed and adaptive task planning management in a network of IoT devices, e.g., drones. Exploiting Jackson's network model, our architecture applies a novel planning strategy to better support control and monitoring operations while the states of the network evolve. To demonstrate the functionalities of our architecture, we also implemented a deep-learning based audio-recognition application using the APRON NorthBound interface, to detect human voices in challenged networks. The application's logic uses Transfer Learning to improve the audio classification accuracy and the runtime of the UAV-based rescue operations

History of the Institut de Robòtica i Informàtica Industrial

The Institut de Robòtica i Informàtica Industrial is a Joint University Research Institute participated by the Spanish National Research Council and the Universitat Politècnica de Catalunya. Founded in 1995, its scientists have addressed over the years many research topics spanning from robot kinematics, to computer graphics, automatic control, energy systems, and human-robot interaction, among others. This book, prepared for its 25th anniversary, covers its evolution over the years, and serves as a mean of appreciation to the many students, administrative personnel, research engineers, or scientists that have formed part of it.Postprint (published version

Framework de planeamento de missões para frotas de drones interligados

The usage of aerial drones has become more popular as they also become more accessible, both in economic and usability terms. Nowadays, these vehicles can present reduced dimensions and a good cost-benefit ratio, which makes it possible for several services and applications supported by aerial drone networks to emerge. Some scenarios that benefit from the use of aerial drones are the monitoring of emergency situations and natural disasters, the patrolling of urban areas and support to police forces, and tourist applications such as the real-time video transmission of points of interest. It is common for the control of the drone to be dependent on human intervention in these situations, which requires professionals specialized in its control. However, in recent years, several solutions have emerged that enable the autonomous flight of these vehicles, minimizing manual interference. Taking into account the enormous diversity of use cases, many of the existing solutions for autonomous control focus on specific scenarios. Generic mission planning platforms also exist, but most of them only allow missions consisting of linear waypoints to be traversed. These situations translate into a mission support that is not very flexible. In this dissertation, we propose a modular infrastructure that can be used in various scenarios, enabling the autonomous control and monitoring of a fleet of aerial drones in a mission context. This platform has two main components, one integrated into the onboard computer of the vehicle, and the other one in the ground control. The former allows the communication with the flight controller so that it can collect telemetry data and send movement instructions to the drone. The latter allows to monitor this data and send the commands remotely, also enabling robust mission planning with multiple drones. A mission can be described in a script that the ground module interprets, sending the commands to the assigned vehicles. These missions can describe different paths, modifying the behaviour of the drones according to external factors, such as a sensor reading. It is also possible to define plugins to be reused in various missions, for example, by integrating an algorithm that ensures that all drones maintain connectivity. The solution was evaluated in scenarios with a single drone and with the collaboration of multiple drones. The tests were performed in a simulated environment and also in an environment with real drones. The observed behaviour is similar in both scenarios.A utilização de drones aéreos tem-se vindo a popularizar à medida que estes se tornam mais acessíveis, quer em termos económicos quer em usabilidade. Atualmente, estes veículos são capazes de apresentar dimensões reduzidas e uma boa relação de custo-benefício, o que potencia que diversos serviços e aplicações suportados por redes de drones aéreos estejam a emergir. Alguns cenários que beneficiam da utilização de drones aéreos são a monitorização de situações de emergência e catástrofes naturais, a patrulha de áreas urbanas e apoio às forças policiais e aplicações turísticas como a transmissão de vídeo em tempo real de pontos de interesse. É comum que o controlo do drone esteja dependente de intervenção humana nestas situações, o que requer profissionais especializados no seu controlo. No entanto, nos últimos anos têm surgido diversas soluções que possibilitam o vôo autónomo destes veículos, minimizando a interferência manual. Perante a enorme diversidade de casos de aplicação, muitas das soluções existentes para o controlo autónomo focam-se em cenários específicos de intervenção. Existem também plataformas de planeamento genérico de missões, mas que na sua maioria apenas permitem missões constituídas por conjuntos lineares de pontos a ser percorridos. Estas situações traduzem-se num suporte a missões que é pouco flexível. Nesta dissertação propomos uma infraestrutura modular passível de ser utilizada em cenários variados, possibilitando o controlo autónomo de uma frota de drones aéreos num contexto de missão e a sua monitorização. Esta plataforma tem dois componentes principais, um integrado no computador a bordo do veículo e o outro no controlo terrestre. O primeiro permite a comunicação com o controlador de vôo para que se possa recolher diversos dados de telemetria e enviar instruções de movimento para o drone. O segundo permite monitorizar esses dados e enviar os comandos remotamente, possibilitando também um planeamento robusto de missões com múltiplos drones. Uma missão pode ser descrita num script que o módulo terrestre interpreta, enviando os comandos para os veículos atribuídos. Estas missões podem descrever diversos caminhos, modificando o comportamento dos drones de acordo com factores externos, como a leitura de um sensor. Também é possível definir plugins para serem reutilizados em várias missões, como por exemplo, integrando um algoritmo que garante que todos os drones mantêm a conectividade. A solução foi avaliada em cenários com um único drone e com a colaboração de múltiplos drones. Os testes foram executados em ambiente simulado e também num ambiente com drones reais. O comportamento observado nas missões é semelhante em ambos os cenários.Mestrado em Engenharia de Computadores e Telemátic