Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails

Robots hold promise in many scenarios involving outdoor use, such as search-and-rescue, wildlife management, and collecting data to improve environment, climate, and weather forecasting. However, autonomous navigation of outdoor trails remains a challenging problem. Recent work has sought to address this issue using deep learning. Although this approach has achieved state-of-the-art results, the deep learning paradigm may be limited due to a reliance on large amounts of annotated training data. Collecting and curating training datasets may not be feasible or practical in many situations, especially as trail conditions may change due to seasonal weather variations, storms, and natural erosion. In this paper, we explore an approach to address this issue through virtual-to-real-world transfer learning using a variety of deep learning models trained to classify the direction of a trail in an image. Our approach utilizes synthetic data gathered from virtual environments for model training, bypassing the need to collect a large amount of real images of the outdoors. We validate our approach in three main ways. First, we demonstrate that our models achieve classification accuracies upwards of 95% on our synthetic data set. Next, we utilize our classification models in the control system of a simulated robot to demonstrate feasibility. Finally, we evaluate our models on real-world trail data and demonstrate the potential of virtual-to-real-world transfer learning.Comment: iROS 201

A Review on IoT Deep Learning UAV Systems for Autonomous Obstacle Detection and Collision Avoidance

[Abstract] Advances in Unmanned Aerial Vehicles (UAVs), also known as drones, offer unprecedented opportunities to boost a wide array of large-scale Internet of Things (IoT) applications. Nevertheless, UAV platforms still face important limitations mainly related to autonomy and weight that impact their remote sensing capabilities when capturing and processing the data required for developing autonomous and robust real-time obstacle detection and avoidance systems. In this regard, Deep Learning (DL) techniques have arisen as a promising alternative for improving real-time obstacle detection and collision avoidance for highly autonomous UAVs. This article reviews the most recent developments on DL Unmanned Aerial Systems (UASs) and provides a detailed explanation on the main DL techniques. Moreover, the latest DL-UAV communication architectures are studied and their most common hardware is analyzed. Furthermore, this article enumerates the most relevant open challenges for current DL-UAV solutions, thus allowing future researchers to define a roadmap for devising the new generation affordable autonomous DL-UAV IoT solutions.Xunta de Galicia; ED431C 2016-045Xunta de Galicia; ED431C 2016-047Xunta de Galicia; , ED431G/01Centro Singular de Investigación de Galicia; PC18/01Agencia Estatal de Investigación de España; TEC2016-75067-C4-1-

A Survey on Human-aware Robot Navigation

Intelligent systems are increasingly part of our everyday lives and have been integrated seamlessly to the point where it is difficult to imagine a world without them. Physical manifestations of those systems on the other hand, in the form of embodied agents or robots, have so far been used only for specific applications and are often limited to functional roles (e.g. in the industry, entertainment and military fields). Given the current growth and innovation in the research communities concerned with the topics of robot navigation, human-robot-interaction and human activity recognition, it seems like this might soon change. Robots are increasingly easy to obtain and use and the acceptance of them in general is growing. However, the design of a socially compliant robot that can function as a companion needs to take various areas of research into account. This paper is concerned with the navigation aspect of a socially-compliant robot and provides a survey of existing solutions for the relevant areas of research as well as an outlook on possible future directions.Comment: Robotics and Autonomous Systems, 202

Autonomisen multikopteriparven hallinta etsintä- ja pelastustehtävissä

This thesis presents the requirements and implementation of a Ground Control Station (GCS) application for controlling a fleet of multicopters to perform a Search And Rescue (SAR) mission. The requirements are put together by analysing existing drone types, SAR practices, and available GCS applications. Multicopters are found to be the most feasible drone to use for the SAR use case because of their maneuverability, despite not having the best endurance. Several existing area coverage methods are presented and their usefulness is analyzed for SAR scenarios where different amounts of prior knowledge is available. It is stated that most search patterns can be used with a fleet of drones, by creating drone formations and by dividing the target area into sub-areas. It is noted that most currently available GCS applications are focused on controlling a single drone for either industrial or hobby use. A proof of concept prototype is developed on top of an open source GCS and tested in field tests. Based on all the previous learnings from the protype and research, a new GCS is designed and developed. The development on optimizing communications between the GCS and the autopilot leads to a filed patent application. The new software is tested with three multicopters in a water rescue scenario and several user interface improvements are made as a result of the learnings. The development of a GCS for controlling a drone fleet for search and rescue is proven feasible.Työssä esitetään multikopteriparven hallintaan käytettävän Ground Control Station (GCS) ohjelmiston vaatimukset ja toteutus Search And Rescue (SAR) etsintä- ja pelastustehtävien suorittamiseksi. Vaatimukset kootaan yhteen analysoimalla saatavilla olevia droonityyppejä, SAR pelastuskäytäntöjä, sekä GCS ohjelmistoja. Multikopterit osoittautuvat liikkuvuutensa ansiosta pelastustehtäviin sopivimmaksi vaihtoehdoksi, vaikka niiden saavutettavissa oleva lentoaika ei ole parhaimmasta päästä. Erilaisia etsintämetodeja esitetään alueiden kattamiseksi ja niiden hyödyllisyyttä analysoidaan SAR tilanteissa, joissa ennakkotietoa on saatavilla vaihtelevasti. Osoitetaan, että useimpia etsintäalgoritmeja voidaan hyödyntää drooniparvella, muodostamalla lentomuodostelmia, sekä jakamalla kohdealue pienempiin osa-alueisiin. Huomataan, että suurin osa tällä hetkellä saatavilla olevista GCS ohjelmistoista on suunnattu teollisuuden tai harrastelijoiden käyttöön, pääasiassa yksittäisen droonin hallintaan. Prototyyppi kehitetään avoimen lähdekoodin GCS ohjelmiston pohjalta ja testataan kenttätesteissä. Tästä saadun tiedon avulla suunnitellaan ja kehitetään uusi GCS ohjelmisto. Kehitystyö viestinnän optimoinniksi autopilotin ja GCS ohjelmiston välillä johtaa patenttihakemukseen. Uusi ohjelmisto testataan kolmella multikopterilla vesipelastustilanteessa ja sen seurauksena käyttöliittymään tehdään useita parannuksia. GCS ohjelmiston luominen drooniparven hallintaan etsintä- ja pelastustehtävissä todetaan mahdolliseksi

An embarrassingly simple approach for visual navigation of forest environments

Navigation in forest environments is a challenging and open problem in the area of field robotics. Rovers in forest environments are required to infer the traversability of a priori unknown terrains, comprising a number of different types of compliant and rigid obstacles, under varying lighting and weather conditions. The challenges are further compounded for inexpensive small-sized (portable) rovers. While such rovers may be useful for collaboratively monitoring large tracts of forests as a swarm, with low environmental impact, their small-size affords them only a low viewpoint of their proximal terrain. Moreover, their limited view may frequently be partially occluded by compliant obstacles in close proximity such as shrubs and tall grass. Perhaps, consequently, most studies on off-road navigation typically use large-sized rovers equipped with expensive exteroceptive navigation sensors. We design a low-cost navigation system tailored for small-sized forest rovers. For navigation, a light-weight convolution neural network is used to predict depth images from RGB input images from a low-viewpoint monocular camera. Subsequently, a simple coarse-grained navigation algorithm aggregates the predicted depth information to steer our mobile platform towards open traversable areas in the forest while avoiding obstacles. In this study, the steering commands output from our navigation algorithm direct an operator pushing the mobile platform. Our navigation algorithm has been extensively tested in high-fidelity forest simulations and in field trials. Using no more than a 16 × 16 pixel depth prediction image from a 32 × 32 pixel RGB image, our algorithm running on a Raspberry Pi was able to successfully navigate a total of over 750 m of real-world forest terrain comprising shrubs, dense bushes, tall grass, fallen branches, fallen tree trunks, small ditches and mounds, and standing trees, under five different weather conditions and four different times of day. Furthermore, our algorithm exhibits robustness to changes in the mobile platform’s camera pitch angle, motion blur, low lighting at dusk, and high-contrast lighting conditions

The Underpinnings of Workload in Unmanned Vehicle Systems

This paper identifies and characterizes factors that contribute to operator workload in unmanned vehicle systems. Our objective is to provide a basis for developing models of workload for use in design and operation of complex human-machine systems. In 1986, Hart developed a foundational conceptual model of workload, which formed the basis for arguably the most widely used workload measurement techniquethe NASA Task Load Index. Since that time, however, there have been many advances in models and factor identification as well as workload control measures. Additionally, there is a need to further inventory and describe factors that contribute to human workload in light of technological advances, including automation and autonomy. Thus, we propose a conceptual framework for the workload construct and present a taxonomy of factors that can contribute to operator workload. These factors, referred to as workload drivers, are associated with a variety of system elements including the environment, task, equipment and operator. In addition, we discuss how workload moderators, such as automation and interface design, can be manipulated in order to influence operator workload. We contend that workload drivers, workload moderators, and the interactions among drivers and moderators all need to be accounted for when building complex, human-machine systems

Autonomous, Collaborative, Unmanned Aerial Vehicles for Search and Rescue

Search and Rescue is a vitally important subject, and one which can be improved through the use of modern technology. This work presents a number of advances aimed towards the creation of a swarm of autonomous, collaborative, unmanned aerial vehicles for land-based search and rescue. The main advances are the development of a diffusion based search strategy for route planning, research into GPS (including the Durham Tracker Project and statistical research into altitude errors), and the creation of a relative positioning system (including discussion of the errors caused by fast-moving units). Overviews are also given of the current state of research into both UAVs and Search and Rescue