Virtual reality simulation of a quadrotor to monitor dependent people at home

Unmanned aerial vehicles (UAVs) represent an assistance solution for home care of dependent persons. These aircraft can cover the home, accompany the person, and position themselves to take photographs that can be analyzed to determine the person's mood and the assistance needed. In this context, this work principally aims to design a tool to aid in the development and validation of the navigation algorithms of an autonomous vision-based UAV for monitoring dependent people. For that, a distributed architecture has been proposed based on the real-time communication of two modules, one of them in charge of the dynamics of the UAV, the trajectory planning and the control algorithms, and the other devoted to visualizing the simulation in an immersive virtual environment. Thus, a system has been developed that allows the evaluation of the behavior of the assistant UAV from a technological point of view, as well as to carry out studies from the assisted person's viewpoint. An initial validation of a quadrotor model monitoring a virtual character demonstrates the advantages of the proposed system, which is an effective, safe and adaptable tool for the development of vision-based UAVs to help dependents at home.This work was partially supported by Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación/European Regional Development Fund under PID2019106084RB-I00 and DPI2016-80894-R grants, and by CIBERSAM of the Instituto de Salud Carlos III

The Underpinnings of Workload in Unmanned Vehicle Systems

This paper identifies and characterizes factors that contribute to operator workload in unmanned vehicle systems. Our objective is to provide a basis for developing models of workload for use in design and operation of complex human-machine systems. In 1986, Hart developed a foundational conceptual model of workload, which formed the basis for arguably the most widely used workload measurement techniquethe NASA Task Load Index. Since that time, however, there have been many advances in models and factor identification as well as workload control measures. Additionally, there is a need to further inventory and describe factors that contribute to human workload in light of technological advances, including automation and autonomy. Thus, we propose a conceptual framework for the workload construct and present a taxonomy of factors that can contribute to operator workload. These factors, referred to as workload drivers, are associated with a variety of system elements including the environment, task, equipment and operator. In addition, we discuss how workload moderators, such as automation and interface design, can be manipulated in order to influence operator workload. We contend that workload drivers, workload moderators, and the interactions among drivers and moderators all need to be accounted for when building complex, human-machine systems

Visual-Inertial Odometry for 3D Pose Estimation and Scene Reconstruction using Unmanned Aerial Vehicles

As Unmanned Aerial Vehicles (UAVs) become increasingly available, pose estimation remains critical for navigation. Pose estimation is also useful for scene reconstruction in certain surveillance applications, such as surveillance in the event of a natural disaster. This thesis presents a Direct Sparse Visual-Inertial Odometry with Loop Closure (VIL-DSO) algorithm design as a pose estimation solution, combining several existing algorithms to fuse inertial and visual information to improve pose estimation and provide metric scale, as initially implemented in Direct Sparse Odometry (DSO) and Direct Sparse Visual-Inertial Odometry (VI-DSO). VIL-DSO utilizes the point selection and loop closure method of the Direct Sparse Odometry with Loop Closure (LDSO) approach. This point selection method improves repeatability by calculating the Shi-Tomasi score to favor corners as point candidates and allows for generating matches for loop closure between keyframes. The proposed VIL-DSO then uses the Kabsch-Umeyama algorithm to reduce the effects of scale-drift caused by loop closure. The proposed VIL-DSO algorithm is composed of three main threads for computing: a coarse tracking thread to assist with keyframe selection and initial pose estimation, a local window optimization thread to fuse Inertial Measurement Unit (IMU) information and visual information to pose scale and pose estimate, and a global optimization thread to identify loop closure and improve pose estimates. The loop closure thread also includes the modification to mitigate scale-drift using the Kabsch-Umeyama algorithm. The trajectory analysis of the estimates yields that the loop closure improves the pose estimation, but causes to scale estimate to drift. The scale-drift mitigation method successfully improves the scale estimate after loop closure. However, the estimation error level struggles to exceed the other state-of-the-art methods, namely VI-DSO and VI-ORB SLAM. The results were evaluated on the EuRoC MAV dataset, which contains fairly short sequences. VIL-DSO is expected to show more advantages when used on a longer dataset,where loop closure is more useful. Lastly, using the odometry as a feed, scene reconstruction and the effects of various factors regarding mapping are discussed, including the use of a monocular camera, camera angle and resolution in outdoor settings

Are those real people? Memory and creative activism

Essay detailing a lineage of creative projects and actions that function at the intersections of art, technology, social engagement and interventionist strategies exploring militarism, violence and memory. These will include the 2006 project dead‐in‐iraq, to type consecutively, all names of America's military casualties from the war in Iraq into the America's Army first person shooter online recruiting game. The essay will as well examine the iraqimemorial.org project, an archive and web based exhibition created from an open call for proposed memorials to the many thousands of civilian casualties from the war in Iraq. More recent projects to be described and analyzed include Killbox, a BAFTA Scotland nominated game about drone warfare developed with the Biome Collective. These projects and ongoing efforts share an approach to critical and conceptual positioning as an artist - developing works that inquisitively engage issues of memory, politics, history, physicality and the virtual. The theoretical basis for the work lies in the belief that it is essential, as an artist and citizen of the world, to engage in, challenge and question the norms and expectations of the digital present and our larger social/political context

AutoNav4D. A co-simulator for unmanned aircraft systems

This TFC document shows the requirements and the components needed to integrate in the same scenario, real Unmanned Aircraft Systems (UAS) and simulated systems. This is called co-simulation. An UAS is a series of onboard avionics systems and an on ground platform, which might takeoff, fly a mission and land safe without a human intervention. UAS Service Abstraction Layer (USAL) is a set of available services running on top of the UAV system architecture to give support to most types of civil UAS missions. These services are managed and communicated by a thin software layer called Architecture for Remote Embedded Applications (MAREA). This Middleware promotes a publish/subscribe model for sending and receiving data, events and commands among the services of the UAS. The Icarus Simulation Integrated Scenario (ISIS) is a collection of reusable services that comprises a minimum common set of elements that are needed in most UAV missions. Some of these services conform the co-simulator. This ISIS integrated scenario is useful to test the platform before the UAS flies. The ICARUS team has developed his Service Oriented Architecture (SOA) platform where is set the real UAS. The co-simulator is an Open Architecture. It may have a Visor Service, some Virtual Vehicles and Virtual Services and a Manager of Virtuality. This co-simulator gives capabilities in fast prototyping and simulation to the UAS Service Abstraction Layer (USAL). Fast prototyping is reached by the use of standards and components. It allows a fast design and implementation of new functionalities. Simulation is used for increase safety and reduce design cost. When a service abstraction layer is used, a virtual implementation has similar effects like the real ones. My job for this TFC was to design and to implement some solutions which conform a co-simulator for Unmanned Aircraft Systems. This co-simulator has been called AutoNAV4D93. There are many things to add to the co-simulator and it may be accomplished in a future as a result of ISIS developing work. The ISIS integrated scenarios will be presented in the AIAA’09 Meeting and exhibit in 9 of January 2009

Development of a ROS environment for researching machine learning techniques applied to drones

The first part of this dissertation presents ROS-MAGNA, a general framework for the definition and management of cooperative missions for multiple Unmanned Aircraft Systems (UAS) based on the Robot Operating System (ROS) [42]. This framework makes transparent the type of autopilot on-board and creates the state machines that control the behaviour of the different UAS from the specification of the multi-UAS mission. In addition, it integrates a virtual world generation tool to manage the information of the environment and visualize the geometrical objects of interest to properly follow the progress of the mission. The framework supports the coexistence of software-in-the-loop, hardware-in-the-loop and real UAS cooperating in the same arena, being a very useful testing tool for the developer of UAS advanced functionalities. To the best of our knowledge, it is the first framework which endows all these capabilities. The document also includes simulations and real experiments which show the main features of the framework. ROS-MAGNA is used to develop and test a machine learning tool. The information generated during a mission is used to train neural networks of different architecture for navigation purposes. The data treatment and training processes are accomplished in a testbench to select the best solution from different datasets. Tensorflow is the framework selected to implement every deep learning algorithm along with its Tensorboard tool for training understanding.Furthermore, an API with the pre-trained is used during a real mission in real time. The third part of this dissertation is the design and integration of a voice control assistant inside ROSMAGNA. Employing diverse online and offline tools, oral commands are processed to perform changes to the mission state and performance and to retrieve information.Universidad de Sevilla. Máster en Ingeniería Industria

Optimal Multi-UAV Trajectory Planning for Filming Applications

Teams of multiple Unmanned Aerial Vehicles (UAVs) can be used to record large-scale outdoor scenarios and complementary views of several action points as a promising system for cinematic video recording. Generating the trajectories of the UAVs plays a key role, as it should be ensured that they comply with requirements for system dynamics, smoothness, and safety. The rise of numerical methods for nonlinear optimization is finding a ourishing field in optimization-based approaches to multi- UAV trajectory planning. In particular, these methods are rather promising for video recording applications, as they enable multiple constraints and objectives to be formulated, such as trajectory smoothness, compliance with UAV and camera dynamics, avoidance of obstacles and inter-UAV con icts, and mutual UAV visibility. The main objective of this thesis is to plan online trajectories for multi-UAV teams in video applications, formulating novel optimization problems and solving them in real time. The thesis begins by presenting a framework for carrying out autonomous cinematography missions with a team of UAVs. This framework enables media directors to design missions involving different types of shots with one or multiple cameras, running sequentially or concurrently. Second, the thesis proposes a novel non-linear formulation for the challenging problem of computing optimal multi-UAV trajectories for cinematography, integrating UAV dynamics and collision avoidance constraints, together with cinematographic aspects such as smoothness, gimbal mechanical limits, and mutual camera visibility. Lastly, the thesis describes a method for autonomous aerial recording with distributed lighting by a team of UAVs. The multi-UAV trajectory optimization problem is decoupled into two steps in order to tackle non-linear cinematographic aspects and obstacle avoidance at separate stages. This allows the trajectory planner to perform in real time and to react online to changes in dynamic environments. It is important to note that all the methods in the thesis have been validated by means of extensive simulations and field experiments. Moreover, all the software components have been developed as open source.Los equipos de vehículos aéreos no tripulados (UAV) son sistemas prometedores para grabar eventos cinematográficos, en escenarios exteriores de grandes dimensiones difíciles de cubrir o para tomar vistas complementarias de diferentes puntos de acción. La generación de trayectorias para este tipo de vehículos desempeña un papel fundamental, ya que debe garantizarse que se cumplan requisitos dinámicos, de suavidad y de seguridad. Los enfoques basados en la optimización para la planificación de trayectorias de múltiples UAVs se pueden ver beneficiados por el auge de los métodos numéricos para la resolución de problemas de optimización no lineales. En particular, estos métodos son bastante prometedores para las aplicaciones de grabación de vídeo, ya que permiten formular múltiples restricciones y objetivos, como la suavidad de la trayectoria, el cumplimiento de la dinámica del UAV y de la cámara, la evitación de obstáculos y de conflictos entre UAVs, y la visibilidad mutua. El objetivo principal de esta tesis es planificar trayectorias para equipos multi-UAV en aplicaciones de vídeo, formulando novedosos problemas de optimización y resolviéndolos en tiempo real. La tesis comienza presentando un marco de trabajo para la realización de misiones cinematográficas autónomas con un equipo de UAVs. Este marco permite a los directores de medios de comunicación diseñar misiones que incluyan diferentes tipos de tomas con una o varias cámaras, ejecutadas de forma secuencial o concurrente. En segundo lugar, la tesis propone una novedosa formulación no lineal para el difícil problema de calcular las trayectorias óptimas de los vehículos aéreos no tripulados en cinematografía, integrando en el problema la dinámica de los UAVs y las restricciones para evitar colisiones, junto con aspectos cinematográficos como la suavidad, los límites mecánicos del cardán y la visibilidad mutua de las cámaras. Por último, la tesis describe un método de grabación aérea autónoma con iluminación distribuida por un equipo de UAVs. El problema de optimización de trayectorias se desacopla en dos pasos para abordar los aspectos cinematográficos no lineales y la evitación de obstáculos en etapas separadas. Esto permite al planificador de trayectorias actuar en tiempo real y reaccionar en línea a los cambios en los entornos dinámicos. Es importante señalar que todos los métodos de la tesis han sido validados mediante extensas simulaciones y experimentos de campo. Además, todos los componentes del software se han desarrollado como código abierto