On-board Obstacle Avoidance in the Teleoperation of Unmanned Aerial Vehicles

Teleoperation von Drohnen in Umgebungen ohne GPS-Verbindung und wenig Bewegungsspielraum stellt den Operator vor besondere Herausforderungen. Hindernisse in einer unbekannten Umgebung erfordern eine zuverlässige Zustandsschätzung und Algorithmen zur Vermeidung von Kollisionen. In dieser Dissertation präsentieren wir ein System zur kollisionsfreien Navigation einer ferngesteuerten Drohne mit vier Propellern (Quadcopter) in abgeschlossenen Räumen. Die Plattform ist mit einem Miniaturcomputer und dem Minimum an Sensoren ausgestattet. Diese Ausstattung genügt den Anforderungen an die Rechenleistung. Dieses Setup ermöglicht des Weiteren eine hochgenaue Zustandsschätzung mit Hilfe einer Kaskaden-Architektur, sehr gutes Folgeverhalten bezüglich der kommandierten Geschwindigkeit, sowie eine kollisionsfreie Navigation. Ein Komplementärfilter berechnet die Höhe der Drohne, während ein Kalman-Filter Beschleunigung durch eine IMU und Messungen eines Optical-Flow Sensors fusioniert und in die Softwarearchitektur integriert. Eine RGB-D Kamera stellt dem Operator ein visuelles Feedback, sowie Distanzmessungen zur Verfügung, um ein Roboter-zentriertes Modell umliegender Hindernisse mit Hilfe eines Bin-Occupancy-Filters zu erstellen. Der Algorithmus speichert die Position dieser Hindernisse, auch wenn sie das Sehfeld des Sensors verlassen, mit Hilfe des geschätzten Zustandes des Roboters. Das Prinzip des Ausweich-Algorithmus basiert auf dem Ansatz einer modell-prädiktiven Regelung. Durch Vorhersage der wahrscheinlichen Position eines Hindernisses werden die durch den Operator kommandierten Sollwerte gefiltert, um eine mögliche Kollision mit einem Hindernis zu vermeiden. Die Plattform wurde experimentell sowohl in einer räumlich abgeschlossenen Umgebung mit zahlreichen Hindernissen als auch bei Testflügen in offener Umgebung mit natürlichen Hindernissen wie z.B. Bäume getestet. Fliegende Roboter bergen das Risiko, im Fall eines Fehlers, sei es ein Bedienungs- oder Berechnungsfehler, durch einen Aufprall am Boden oder an Hindernissen Schaden zu nehmen. Aus diesem Grund nimmt die Entwicklung von Algorithmen dieser Roboter ein hohes Maß an Zeit und Ressourcen in Anspruch. In dieser Arbeit präsentieren wir zwei Methoden (Software-in-the-loop- und Hardware-in-the-loop-Simulation) um den Entwicklungsprozess zu vereinfachen. Via Software-in-the-loop-Simulation konnte der Zustandsschätzer mit Hilfe simulierter Sensoren und zuvor aufgenommener Datensätze verbessert werden. Eine Hardware-in-the-loop Simulation ermöglichte uns, den Roboter in Gazebo (ein bekannter frei verfügbarer ROS-Simulator) mit zusätzlicher auf dem Roboter installierter Hardware in Simulation zu bewegen. Ebenso können wir damit die Echtzeitfähigkeit der Algorithmen direkt auf der Hardware validieren und verifizieren. Zu guter Letzt analysierten wir den Einfluss der Roboterbewegung auf das visuelle Feedback des Operators. Obwohl einige Drohnen die Möglichkeit einer mechanischen Stabilisierung der Kamera besitzen, können unsere Drohnen aufgrund von Gewichtsbeschränkungen nicht auf diese Unterstützung zurückgreifen. Eine Fixierung der Kamera verursacht, während der Roboter sich bewegt, oft unstetige Bewegungen des Bildes und beeinträchtigt damit negativ die Manövrierbarkeit des Roboters. Viele wissenschaftliche Arbeiten beschäftigen sich mit der Lösung dieses Problems durch Feature-Tracking. Damit kann die Bewegung der Kamera rekonstruiert und das Videosignal stabilisiert werden. Wir zeigen, dass diese Methode stark vereinfacht werden kann, durch die Verwendung der Roboter-internen IMU. Unsere Ergebnisse belegen, dass unser Algorithmus das Kamerabild erfolgreich stabilisieren und der rechnerische Aufwand deutlich reduziert werden kann. Ebenso präsentieren wir ein neues Design eines Quadcopters, um dessen Ausrichtung von der lateralen Bewegung zu entkoppeln. Unser Konzept erlaubt die Neigung der Propellerblätter unabhängig von der Ausrichtung des Roboters mit Hilfe zweier zusätzlicher Aktuatoren. Nachdem wir das dynamische Modell dieses Systems hergeleitet haben, synthetisierten wir einen auf Feedback-Linearisierung basierten Regler. Simulationen bestätigen unsere Überlegungen und heben die Verbesserung der Manövrierfähigkeit dieses neuartigen Designs hervor.The teleoperation of unmanned aerial vehicles (UAVs), especially in cramped, GPS-restricted, environments, poses many challenges. The presence of obstacles in an unfamiliar environment requires reliable state estimation and active algorithms to prevent collisions. In this dissertation, we present a collision-free indoor navigation system for a teleoperated quadrotor UAV. The platform is equipped with an on-board miniature computer and a minimal set of sensors for this task and is self-sufficient with respect to external tracking systems and computation. The platform is capable of highly accurate state-estimation, tracking of the velocity commanded by the user and collision-free navigation. The robot estimates its state in a cascade architecture. The attitude of the platform is calculated with a complementary filter and its linear velocity through a Kalman filter integration of inertial and optical flow measurements. An RGB-D camera serves the purpose of providing visual feedback to the operator and depth measurements to build a probabilistic, robot-centric obstacle state with a bin-occupancy filter. The algorithm tracks the obstacles when they leave the field of view of the sensor by updating their positions with the estimate of the robot's motion. The avoidance part of our navigation system is based on the Model Predictive Control approach. By predicting the possible future obstacles states, the UAV filters the operator commands by altering them to prevent collisions. Experiments in obstacle-rich indoor and outdoor environments validate the efficiency of the proposed setup. Flying robots are highly prone to damage in cases of control errors, as these most likely will cause them to fall to the ground. Therefore, the development of algorithm for UAVs entails considerable amount of time and resources. In this dissertation we present two simulation methods, i.e. software- and hardware-in-the-loop simulations, to facilitate this process. The software-in-the-loop testing was used for the development and tuning of the state estimator for our robot using both the simulated sensors and pre-recorded datasets of sensor measurements, e.g., from real robotic experiments. With hardware-in-the-loop simulations, we are able to command the robot simulated in Gazebo, a popular open source ROS-enabled physical simulator, using computational units that are embedded on our quadrotor UAVs. Hence, we can test in simulation not only the correct execution of algorithms, but also the computational feasibility directly on the robot's hardware. Lastly, we analyze the influence of the robot's motion on the visual feedback provided to the operator. While some UAVs have the capacity to carry mechanically stabilized camera equipment, weight limits or other problems may make mechanical stabilization impractical. With a fixed camera, the video stream is often unsteady due to the multirotor's movement and can impair the operator's situation awareness. There has been significant research on how to stabilize videos using feature tracking to determine camera movement, which in turn is used to manipulate frames and stabilize the camera stream. However, we believe that this process could be greatly simplified by using data from a UAV’s on-board inertial measurement unit to stabilize the camera feed. Our results show that our algorithm successfully stabilizes the camera stream with the added benefit of requiring less computational power. We also propose a novel quadrotor design concept to decouple its orientation from the lateral motion of the quadrotor. In our design the tilt angles of the propellers with respect to the quadrotor body are being simultaneously controlled with two additional actuators by employing the parallelogram principle. After deriving the dynamic model of this design, we propose a controller for this platform based on feedback linearization. Simulation results confirm our theoretical findings, highlighting the improved motion capabilities of this novel design with respect to standard quadrotors

Characterisation of a nuclear cave environment utilising an autonomous swarm of heterogeneous robots

As nuclear facilities come to the end of their operational lifetime, safe decommissioning becomes a more prevalent issue. In many such facilities there exist ‘nuclear caves’. These caves constitute areas that may have been entered infrequently, or even not at all, since the construction of the facility. Due to this, the topography and nature of the contents of these nuclear caves may be unknown in a number of critical aspects, such as the location of dangerous substances or significant physical blockages to movement around the cave. In order to aid safe decommissioning, autonomous robotic systems capable of characterising nuclear cave environments are desired. The research put forward in this thesis seeks to answer the question: is it possible to utilise a heterogeneous swarm of autonomous robots for the remote characterisation of a nuclear cave environment? This is achieved through examination of the three key components comprising a heterogeneous swarm: sensing, locomotion and control. It will be shown that a heterogeneous swarm is not only capable of performing this task, it is preferable to a homogeneous swarm. This is due to the increased sensory and locomotive capabilities, coupled with more efficient explorational prowess when compared to a homogeneous swarm

Models, algorithms and architectures for cooperative manipulation with aerial and ground robots

Les dernières années ont vu le développement de recherches portant sur l'interaction physique entre les robots aériens et leur environnement, accompagné de l'apparition de nombreux nouveaux systèmes mécaniques et approches de régulation. La communauté centrée autour de la robotique aérienne observe actuellement un déplacement de paradigmes des approches classiques de guidage, de navigation et de régulation vers des tâches moins triviales, telle le développement de l'interaction physique entre robots aériens et leur environnement. Ceci correspond à une extension des tâches dites de manipulation, du sol vers les airs. Cette thèse contribue au domaine de la manipulation aérienne en proposant un nouveau concept appelé MAGMaS, pour " Multiple Aerial Ground Manipulator System ". Les motivations qui ont conduites à l'association de manipulateurs terrestres et aériens pour effectuer des tâches de manipulation coopérative, résident dans une volonté d'exploiter leurs particularités respectives. Les manipulateurs terrestres apportant leur importante force et les manipulateurs aériens apportant leur vaste espace de travail. La première contribution de cette thèse présente une modélisation rigoureuse des MAGMaS. Les propriétés du système ainsi que ses possibles extensions sont discutées. Les méthodes de planning, d'estimation et de régulation nécessaire à l'exploitation des MAGMaS pour des tâches de manipulation collaborative sont dérivées. Ce travail propose d'exploiter les redondances des MAGMaS grâce à un algorithme optimal d'allocation de forces entre les manipulateurs. De plus, une méthode générale d'estimation de forces pour robots aériens est introduite. Toutes les techniques et les algorithmes présentés dans cette thèse sont intégrés dans une architecture globale, utilisée à la fois pour la simulation et la validation expérimentale. Cette architecture est en outre augmentée par l'addition d'une structure de télé-présence, afin de permettre l'opération à distances des MAGMaS. L'architecture générale est validée par une démonstration de levage de barre, qui est une application représentative des potentiels usages des MAGMaS. Une autre contribution relative au développement des MAGMaS consiste en une étude exploratoire de la flexibilité dans les objets manipulés par un MAGMaS. Un modèle du phénomène vibratoire est dérivé afin de mettre en exergue ses propriétés en termes de contrôle. La dernière contribution de cette thèse consiste en une étude exploratoire sur l'usage des actionneurs à raideur variable dans les robots aériens, dotant ces systèmes d'une compliance mécanique intrinsèque et de capacité de stockage d'énergie. Les fondements théoriques sont associés à la synthèse d'un contrôleur non-linéaire. L'approche proposée est validée par le biais d'expériences reposant sur l'intégration d'un actionneur à raideur variable léger sur un robot aérien.In recent years, the subject of physical interaction for aerial robots has been a popular research area with many new mechanical designs and control approaches being proposed. The aerial robotics community is currently observing a paradigm shift from classic guidance, navigation, and control tasks towards more unusual tasks, for example requesting aerial robots to physically interact with the environment, thus extending the manipulation task from the ground into the air. This thesis contributes to the field of aerial manipulation by proposing a novel concept known has Multiple Aerial-Ground Manipulator System or MAGMaS, including what appears to be the first experimental demonstration of a MAGMaS and opening a new route of research. The motivation behind associating ground and aerial robots for cooperative manipulation is to leverage their respective particularities, ground robots bring strength while aerial robots widen the workspace of the system. The first contribution of this work introduces a meticulous system model for MAGMaS. The system model's properties and potential extensions are discussed in this work. The planning, estimation and control methods which are necessary to exploit MAGMaS in a cooperative manipulation tasks are derived. This works proposes an optimal control allocation scheme to exploit the MAGMaS redundancies and a general model-based force estimation method is presented. All of the proposed techniques reported in this thesis are integrated in a global architecture used for simulations and experimental validation. This architecture is extended by the addition of a tele-presence framework to allow remote operations of MAGMaS. The global architecture is validated by robust demonstrations of bar lifting, an application that gives an outlook of the prospective use of the proposed concept of MAGMaS. Another contribution in the development of MAGMaS consists of an exploratory study on the flexibility of manipulated loads. A vibration model is derived and exploited to showcase vibration properties in terms of control. The last contribution of this thesis consists of an exploratory study on the use of elastic joints in aerial robots, endowing these systems with mechanical compliance and energy storage capabilities. Theoretical groundings are associated with a nonlinear controller synthesis. The proposed approach is validated by experimental work which relies on the integration of a lightweight variable stiffness actuator on an aerial robot

Development of an assisted-teleoperation system for a dual-manipulator nuclear decommissioning robot

This thesis concerns a robotic platform that is being used for research into assisted tele–operation for common nuclear decommissioning tasks, such as remote handling and pipe cutting. The machine consists of dual, seven–function, hydraulically actuated HYDROLEK manipulators mounted (in prior research) on a mobile BROKK base unit. Whilst the original system was operated by remote control, the present thesis focusses on the development of a visual servoing system, in which the user selects the object of interest from an on–screen image, whilst the computer control system determines and implements via feedback control the required position and orientation of the manipulators. Novel research contributions are made in three main areas: (i) the development of a detailed mechanistic model of the system; (ii) the development and preliminary testing in the laboratory of the new assisted–teleoperation user interface; and (iii) the development of improved control systems for joint angle set point tracking, and their systematic, quantitative comparison via simulation and experiment. The mechanistic model builds on previous work, while the main novelty in this thesis relates to the hydraulic component of the model, and the development and evaluation of a multi–objective genetic algorithm framework to identify the unknown parameter values. To improve on the joystick direct teleoperation currently used as standard in the nuclear industry, which is slow and requires extensive operator training, the proposed assisted–teleoperation makes use of a camera mounted on the robot. Focussing on pipe cutting as an example, the new system ensures that one manipulator automatically grasps the user–selected pipe, and appropriately positions the second for a cutting operation. Initial laboratory testing (using a plastic pipe) shows the efficacy of the approach for positioning the manipulators, and suggests that for both experienced and inexperienced users, the task is completed significantly faster than via tele-operation. Finally, classical industrial, fuzzy logic, and novel state dependent parameter approaches to control are developed and compared, with the aim being to determine a relatively simple controller that yields good performance for the hydraulic manipulators. An improved, more structured method of dealing with the dead–zone characteristics is developed and implemented, replacing the rather ad hoc approach that had been utilised in previous research for the same machine

Planning and control of robotic manipulation actions for extreme environments

A large societal and economic need arises for advanced robotic capabilities, where we need to perform complex human-like tasks such as tool-use, in environments that are hazardous for human workers. This thesis addresses a collection of problems, which arise when robotic manipulators must perform complex tasks in cluttered and constrained environments. The work is illustrated by example scenarios of robotic tool use, grasping and manipulating, motivated by the challenges of dismantling operations in the extreme environments of nuclear decommissioning Contrary to popular assumptions, legacy nuclear facilities (which can date back three-quarters of a century in the UK) can be highly unstructured and uncertain environments, with insufficient a-priori information available for e.g. conventional pre-programming of robot tasks. Meanwhile, situational awareness and direct teleoperation can be extremely difficult for human operators working in a safe zone that is physically remote from the robot. This engenders a need for significant autonomous capabilities. Robots must use vision and sensory systems to perceive their environment, plan and execute complex actions on complex objects in cluttered and constrained environments. Significant radiation, of different types and intensities, provides further challenges in terms of sensor noise. Perception uncertainty can also result from e.g. vision systems observing shiny featureless metal structures. Robotic actions therefore need to be: i) planned in ways that are robust to uncertainties; and ii) controlled in ways which enable the robust reaction to disturbances. In particular, we investigate motion planning and control in tasks where the robot must: maintain contact while moving over arbitrarily shaped surfaces with end-effector tools; exert forces and withstand perturbations during forceful contact actions; while also avoiding collisions with obstacles; avoiding singularity configurations; and increasing robustness by maximising manipulability during task execution. Furthermore, we consider the issues of robust planning and control with respect to uncertain information, derived from noisy sensors in challenging environments. We explore the Riemannian geometry and robot's manipulability to yield path planners that produce paths for both fixed-based and floating-based robots, whose tools always stay in contact with the object's surface. Our planners overcome disturbances in the perception and account for robot/environment interactions that may demand unexpected forces. The task execution is entrusted to a hybrid force/motion controller whose motion space behaves with compliance to accommodate unexpected stiffness changes throughout the contact. We examine the problem of grasping a tool for performing a task. Firstly, we introduce a method for selecting the grasp candidate onto an object yielding collision-free motion for the robot in the post-grasp movements. Furthermore, we study the case of a dual-arm robot performing full-force tasks on an object and slippage on the grasping is allowed. We account for the slippage throughout the task execution using a novel controller based on the sliding mode controllers

Robotics 2010

Without a doubt, robotics has made an incredible progress over the last decades. The vision of developing, designing and creating technical systems that help humans to achieve hard and complex tasks, has intelligently led to an incredible variety of solutions. There are barely technical fields that could exhibit more interdisciplinary interconnections like robotics. This fact is generated by highly complex challenges imposed by robotic systems, especially the requirement on intelligent and autonomous operation. This book tries to give an insight into the evolutionary process that takes place in robotics. It provides articles covering a wide range of this exciting area. The progress of technical challenges and concepts may illuminate the relationship between developments that seem to be completely different at first sight. The robotics remains an exciting scientific and engineering field. The community looks optimistically ahead and also looks forward for the future challenges and new development

NASA Capability Roadmaps Executive Summary

This document is the result of eight months of hard work and dedication from NASA, industry, other government agencies, and academic experts from across the nation. It provides a summary of the capabilities necessary to execute the Vision for Space Exploration and the key architecture decisions that drive the direction for those capabilities. This report is being provided to the Exploration Systems Architecture Study (ESAS) team for consideration in development of an architecture approach and investment strategy to support NASA future mission, programs and budget requests. In addition, it will be an excellent reference for NASA's strategic planning. A more detailed set of roadmaps at the technology and sub-capability levels are available on CD. These detailed products include key driving assumptions, capability maturation assessments, and technology and capability development roadmaps

In Review Magazine / January 2012

In Review magazine, published quarterly, created to showcase the best of NPS