Master of Science

thesisThis thesis provides details on the development of automatic collision avoidance for manually tele-operated unmanned aerial vehicles. We note that large portions of this work are also reprinted with permission, from 2014 IEEE International Conference on Robotics and Automation, \Automatic Collision Avoidance for Manually Tele-operated Unmanned Aerial Vehicles", by J. Israelsen, M. Beall, D. Bareiss, D. Stuart, E. Keeney, and J. van den Berg c 2014 IEEE. We provide a method to aid the operator of unmanned aerial vehicles. We do this by automatically performing collision avoidance with obstacles in the environment. Our method allows the operator to focus on the overall motion of the vehicle rather than requiring the operator to perform collision avoidance. Where other currently existing systems override the controls of the operator only as a last resort, our approach was developed such that the operator can rely on the automatic collision avoidance for maneuverability. Given the current operator control input, our approach continually determines the future path of the vehicle. If along the future path a collision is predicted, then our algorithm will minimally override the operator's control such that the vehicle will not collide with the obstacles in the environment. Such an approach ensures the safety of the operator's controls while simultaneously maintaining the original intent of the operator. We successfully implemented this approach in a simulated environment, as well as on a physical quadrotor system in a laboratory environment. Our experiments show that, even when intentionally trying to do so, the operator failed to crash the vehicle into environment obstacles

Haptic Feedback Effects on Human Control of a UAV in a Remote Teleoperation Flight Task

The remote manual teleoperation of an unmanned aerial vehicle (UAV) by a human operator creates a human-in-the loop system that is of great concern. In a remote teleoperation task, a human pilot must make control decisions based upon sensory information provided by the governed system. Often, this information consists of limited visual feedback provided by onboard cameras that do not provide an operator with an accurate portrayal of their immediate surroundings compromising the safety of the mobile robot. Due to this shortfall, haptic force feedback is often provided to the human in an effort to increase their perceptual awareness of the surrounding world. To investigate the effects of this additional sensory information provided to the human op-erator, we consider two haptic force feedback strategies. They were designed to provide either an attractive force to influence control behavior towards a reference trajectory along a flight path, or a repulsive force directing operators away from obstacles to prevent collision. Subject tests were con-ducted where human operators manually operated a remote UAV through a corridor environment under the conditions of the two strategies. For comparison, the conditions of no haptic feedback and the liner combination of both attractive and repulsive strategies were included in the study. Experi-mental results dictate that haptic force feedback in general (including both attractive and repulsive force feedback) improves the average distance from surrounding obstacles up to 21%. Further statis-tical comparison of repulsive and attractive feedback modalities reveal that even though a repulsive strategy is based directly on obstacles, an attractive strategy towards a reference trajectory is more suitable across all performance metrics. To further examine the effects of haptic aides in a UAV teleoperation task, the behavior of the human system as part of the control loop was also investigated. Through a novel device placed on the end effector of the haptic device, human-haptic interaction forces were captured and further analyzed. With this information, system identification techniques were carried out to determine the plausibility of deriving a human control model for the system. By defining lateral motion as a one-dimensional compensatory tracking task the results show that general human control behavior can be identified where lead compensation in invoked to counteract second-order UAV dynamics

The use of modern tools for modelling and simulation of UAV with Haptic

Unmanned Aerial Vehicle (UAV) is a research field in robotics which is in high demand in recent years, although there still exist many unanswered questions. In contrast, to the human operated aerial vehicles, it is still far less used to the fact that people are dubious about flying in or flying an unmanned vehicle. It is all about giving the control right to the computer (which is the Artificial Intelligence) for making decisions based on the situation like human do but this has not been easy to make people understand that it’s safe and to continue the enhancement on it. These days there are many types of UAVs available in the market for consumer use, for applications like photography to play games, to map routes, to monitor buildings, for security purposes and much more. Plus, these UAVs are also being widely used by the military for surveillance and for security reasons. One of the most commonly used consumer product is a quadcopter or quadrotor. The research carried out used modern tools (i.e., SolidWorks, Java Net Beans and MATLAB/Simulink) to model controls system for Quadcopter UAV with haptic control system to control the quadcopter in a virtual simulation environment and in real time environment. A mathematical model for the controlling the quadcopter in simulations and real time environments were introduced. Where, the design methodology for the quadcopter was defined. This methodology was then enhanced to develop a virtual simulation and real time environments for simulations and experiments. Furthermore, the haptic control was then implemented with designed control system to control the quadcopter in virtual simulation and real time experiments. By using the mathematical model of quadcopter, PID & PD control techniques were used to model the control setup for the quadcopter altitude and motion controls as work progressed. Firstly, the dynamic model is developed using a simple set of equations which evolves further by using complex control & mathematical model with precise function of actuators and aerodynamic coefficients Figure5-7. The presented results are satisfying and shows that flight experiments and simulations of the quadcopter control using haptics is a novel area of research which helps perform operations more successfully and give more control to the operator when operating in difficult environments. By using haptic accidents can be minimised and the functional performance of the operator and the UAV will be significantly enhanced. This concept and area of research of haptic control can be further developed accordingly to the needs of specific applications

The use of modern tools for modelling and simulation of UAV with Haptic

Unmanned Aerial Vehicle (UAV) is a research field in robotics which is in high demand in recent years, although there still exist many unanswered questions. In contrast, to the human operated aerial vehicles, it is still far less used to the fact that people are dubious about flying in or flying an unmanned vehicle. It is all about giving the control right to the computer (which is the Artificial Intelligence) for making decisions based on the situation like human do but this has not been easy to make people understand that it’s safe and to continue the enhancement on it. These days there are many types of UAVs available in the market for consumer use, for applications like photography to play games, to map routes, to monitor buildings, for security purposes and much more. Plus, these UAVs are also being widely used by the military for surveillance and for security reasons. One of the most commonly used consumer product is a quadcopter or quadrotor. The research carried out used modern tools (i.e., SolidWorks, Java Net Beans and MATLAB/Simulink) to model controls system for Quadcopter UAV with haptic control system to control the quadcopter in a virtual simulation environment and in real time environment. A mathematical model for the controlling the quadcopter in simulations and real time environments were introduced. Where, the design methodology for the quadcopter was defined. This methodology was then enhanced to develop a virtual simulation and real time environments for simulations and experiments. Furthermore, the haptic control was then implemented with designed control system to control the quadcopter in virtual simulation and real time experiments. By using the mathematical model of quadcopter, PID & PD control techniques were used to model the control setup for the quadcopter altitude and motion controls as work progressed. Firstly, the dynamic model is developed using a simple set of equations which evolves further by using complex control & mathematical model with precise function of actuators and aerodynamic coefficients Figure5-7. The presented results are satisfying and shows that flight experiments and simulations of the quadcopter control using haptics is a novel area of research which helps perform operations more successfully and give more control to the operator when operating in difficult environments. By using haptic accidents can be minimised and the functional performance of the operator and the UAV will be significantly enhanced. This concept and area of research of haptic control can be further developed accordingly to the needs of specific applications

Aerial Vehicles

This book contains 35 chapters written by experts in developing techniques for making aerial vehicles more intelligent, more reliable, more flexible in use, and safer in operation.It will also serve as an inspiration for further improvement of the design and application of aeral vehicles. The advanced techniques and research described here may also be applicable to other high-tech areas such as robotics, avionics, vetronics, and space

Development and evaluation of a collision avoidance system for supervisory control of a micro aerial vehicle

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 195-108).Recent technological advances have enabled Unmanned Aerial Vehicles (UAVs) and Micro Aerial Vehicles (MAVs) to become increasingly prevalent in a variety of domains. From military surveillance to disaster relief to search-and-rescue tasks, these systems have the capacity to assist in difficult or dangerous tasks and to potentially save lives. To enable operation by minimally trained personnel, the control interfaces require increased usability in order to maintain safety and mission effectiveness. In particular, as these systems are used in the real world, the operator must be able to navigate around obstacles in unknown and unstructured environments. In order to address this problem, the Collision and Obstacle Detection and Alerting (CODA) display was designed and integrated into a smartphone-based MAV control interface. The CODA display uses a combination of visual and haptic alerts to warn the operator of potential obstacles in the environment to help the operator navigate more effectively and avoid collisions. To assess the usability of this system, a within-subjects experiment was conducted in which participants used the mobile interface to pilot a MAV both with and without the assistance of the CODA display. The task consisted of navigating though a simulated indoor environment and locating visual targets. Metrics for the two conditions examined performance, control strategies, and subjective feedback from each participant. Overall, the addition of the CODA display resulted in higher performance, lowering the crash rate and decreasing the amount of time required to complete the tasks. Despite increasing the complexity of the interface, adding the CODA display did not significantly impact usability, and participants preferred operating the MAV with the CODA display. These results demonstrate that the CODA display provides the basis for an effective alerting tool to assist with MAV operation for exploring unknown environments. Future work should explore expansion to three-dimensional sensing and alerting capabilities as well as validation in an outdoor environment.by Kimberly F. Jackson.S.M

Autonomous Collision Avoidance for a Teleoperated UAV Based on a Super-Ellipsoidal Potential Function

This thesis presents the design of a super-ellipsoidal potential function (SEPF) that can be used, in a static and dynamic environment, for autonomous collision avoidance of an unmanned aerial vehicle (UAV) in a 3-dimensional space. In the design of the SEPF, we have the full control over the shape and size of the potential function. In our proposed approach, a teleoperated UAV can not only autonomously avoid collision with surrounding objects but also track the operator\u27 control input as closely as possible. As a result, an operator can always be in control of the UAV for his/her high-level guidance and navigation task without worrying too much about the UAV collision avoidance while it is being teleoperated. The effectiveness of the proposed approach is demonstrated through a human-in-the-loop simulation using virtual robot experimentation platform (v-rep) and Matlab programs and experimentation using a physical quadrotor UAV in a laboratory environment

Motion feedback in the teleoperation of Unmanned Aerial Vehicles

Teleoperation of unmanned vehicles is a valuable tool in scenarios where the operator can not or should not operate the vehicle from on-board. Applications range from hazardous environments where exposure needs to be avoided, control of Unmanned Aerial Vehicles (UAV) to retrieve overviews of inaccessible disaster areas, to deep sea exploration where on-board operation is simply not possible. However, limitations in sensor performance, noise and laten- cies introduced in the transmission, and ineffective display of the information to the operator can lead to a reduced amount of infor- mation, reduced performance, a loss of situation awareness, and in the worst case a loss of the remote vehicle. The spatial decoupling between the operator and the vehicle is one of the main challenges in teleoperation. Most setups include one or more control sticks to steer the ve- hicle, a monitor displaying the live video feed of the main vehicle camera, and a seat for the operator. This can be extended by display- ing additional state information using monitors or visual overlay, rendered on top of the main video stream [Tvaryanas, 2004; van Erp, 2000]. However, processing of multiple screens can increase mental workload. This can cause the operator to miss important information, leading to a loss of situation awareness and reduced performance or a crash of the vehicle. Instead of presenting information purely visually, other feedback modalities can be used to convey vehicle state or information about the task. The goal of this PhD thesis is to investigate the possibility of providing additional information using motion feedback. Here, motion feedback is defined as physically moving the operator using a motion simulator. In the work presented in this thesis a distinction between two motion feedback types is made. Vehicle-state motion feedback describes vehicle motion, while task-related motion feedback is the result of the combination of desired and actual vehicle motion. To investigate the effects of motion feedback in teleoperation several studies have been conducted. In the experiments presented participants either controlled a virtual quadrotor flying in a simu- lated environment or a real octorotor. Participants controlled the UAV from within the CyberMotion Simulator (CMS), an 8-DOF motion simulator located at the Max Planck Institute for Biological Cybernetics. The results show that providing motion feedback has a positive effect on performance in teleoperation of remote UAVs. If the remote vehicle is subject to external disturbances, e.g., wind gusts, vehicle- state feedback showed to improve disturbance rejection capabilities leading to increased performance. Furthermore, motion feedback can be shaped to include additional information about the task with positive effects on performance. This shows that the additional information included in the motion feedback can be used by the operator to improve performance and control behavior.Die Teleoperation eines unbemannten Gefährts ist ein wertvolles Werkzeug in Situationen, in denen der Pilot das Gefährt nicht von Bord aus steuern kann oder sollte. Beispiele hierfür reichen von, für den Piloten, toxischen Umgebungen, über Luftaufnahmen von Katastrophengebieten mithilfe von unbemannten Flugzeugen (engl. Unmanned Aerial Vehicle(UAV)), bis zur Erforschung der Tiefsee, bei der die Steuerung von Bord schlichtweg unmöglich wird. Allerdings führen Einschränkungen in der Sensorerfassung, Rau- schen und Latenzen in der Übertragung, sowie eine ineffiziente Darstellung der Informationen für den Piloten dann zu einem redu- zierten Informationsfluss, reduzierter Leistung, einem Verlust des Situationsbewusstseins und im schlimmsten Fall zu einem Verlust des Gefährts. Die räumliche Entkopplung zwischen dem Piloten und des Flugobjekts ist eine der wichtigsten Herausforderungen in der Teleoperation von UAVs. Die meisten Kontrollstationen beinhalten ein oder mehrere Steu- erknüppel um das Gefährt zu steuern, einen Monitor der eine di- rekte Videoübertragung der Hauptkamera anzeigt und ein Sitzplatz für den Piloten. Dies kann erweitert werden, in dem zusätzliche Statusinformationen mit weiteren Monitoren oder visuellen Über- lagerungen, die über die Hauptübertragung gezeichnet werden, angezeigt werden [Tvaryanas, 2004; van Erp, 2000]. Jedoch kann die Verarbeitung mehrerer Bildschirme die mentale Belastung erhö- hen. Dies kann dazu führen, dass der Pilot wichtige Informationen nicht aufnimmt, was zu einem Verlust des Situationsbewusstseins und einhergehender reduzierten Leistung oder einem Unfall des Gefährts führt. Anstatt Information rein visuell zu präsentieren, können ande- re Modalitäten genutzt werden Rückmeldungen über den Status des Gefährts oder Informationen über die Aufgabe zu präsentieren. Das Ziel dieser Doktorarbeit ist die Untersuchung der Modalität der Bewegung. Es soll untersucht werden, ob Bewegungen genutzt werden können, um dem Piloten zusätzliche Rückmeldungen über den Zustand des Gefährts bereit zu stellen. Bewegungsfeedback beschreibt hier die physikalische Bewegung des Piloten mit Hilfe eines Bewegungssimulators. In dieser Arbeit wird zwischen zwei Typen von Bewegungsfeedback unterschieden. Fahrzeugzustandsbe- wegungsfeedback beschreibt die Bewegung des Fahrzeugs, während Aufgabenabhängiges Bewegungsfeedback die Kombination aus tatsächli- chem und gewünschtem Fahrzeugzustand ist. Die Effekte von Bewegungsfeedback in der Teleoperation wurden in mehreren Studien untersucht. In den vorgestellten Experimenten kontrollierten Teilnehmer entweder einen virtuellen Quadrotor, der in einer simulierten Umgebung flog, oder einen echten Octorotor. Die Teilnehmer steuerten das UAV von der Kanzel des CyberMotion Simulators (CMS) aus, ein 8-DOF Bewegungssimulator, der sich am Max-Planck-Institut für biologische Kybernetik befindet. Die Ergebnisse zeigen, dass die Bereitstellung von Bewegungs- feedback positive Effekte auf die Leistung und das Verhalten des Piloten in der Steuerung des UAVs hat. Ist das UAV externen Stö- rungen ausgesetzt, wie z.B. Windstößen, zeigte sich, dass Fahr- zeugzustandsbewegungsfeedback die Fähigkeit der Störungsunter- drückung des Piloten verbessert, was zu Leistungsteigerungen führt. Außerdem zeigte sich, dass Bewegungsfeedback dahingehend ge- formt werden kann, zusätzliche Informationen über die Aufgabe bereitzustellen. Dies zeigt, dass die zusätzlichen Informationen vom Piloten genutzt werden können um Leistung und Kontrollverhalten zu verbessern