329 research outputs found
A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera
The real-time dynamic environment perception has become vital for autonomous
robots in crowded spaces. Although the popular voxel-based mapping methods can
efficiently represent 3D obstacles with arbitrarily complex shapes, they can
hardly distinguish between static and dynamic obstacles, leading to the limited
performance of obstacle avoidance. While plenty of sophisticated learning-based
dynamic obstacle detection algorithms exist in autonomous driving, the
quadcopter's limited computation resources cannot achieve real-time performance
using those approaches. To address these issues, we propose a real-time dynamic
obstacle tracking and mapping system for quadcopter obstacle avoidance using an
RGB-D camera. The proposed system first utilizes a depth image with an
occupancy voxel map to generate potential dynamic obstacle regions as
proposals. With the obstacle region proposals, the Kalman filter and our
continuity filter are applied to track each dynamic obstacle. Finally, the
environment-aware trajectory prediction method is proposed based on the Markov
chain using the states of tracked dynamic obstacles. We implemented the
proposed system with our custom quadcopter and navigation planner. The
simulation and physical experiments show that our methods can successfully
track and represent obstacles in dynamic environments in real-time and safely
avoid obstacles
Automated multi-rotor draft survey of large vessels
In maritime sector draft survey has a significant importance as it is used to determine many important factors used in maritime transportation. Draft is the vertical displacement from the bottom of the keel (the bottom-most element of a vessel) to the water line (the line of meeting point of hull and the water surface). It is used to measure the minimum water depth for safe navigation of vessel and to evaluate mass of cargo in the vessel by the change in displacement on the draft scale after loading of the cargo in the vessel. Draft measurement of a vessel has a vital role in maritime sector to ensure a safe equilibrium between maximum and minimum cargo that can be loaded in the vessel. Draft survey performed at the time of loading and unloading of cargo (Iron Ore) at the Narvik port to read out draft markings traditionally involved a round trip around the vessel in a small crew boat and it is a time consuming and challenging task specially in darkness (during night), shadows and when difficult to safely reach the crew boat close enough due to anchors and buoys. The goal of this study is to develop an autonomous multi-rotor system that can survey the large vessel to capture all the necessary draft measurements by reaching close enough even in challenging environments like nighttime and in presence of obstacles. This involves developing the solution for path planning to perform flight operation autonomously, developing guidance and control algorithm for the flight operation to enable the multi-rotor to follow the designated path and perform the inspection while avoiding all the hurdles using collision avoidance system. Along with developing the specifications for a multi-rotor that can perform the inspection and suggest necessary system components including multi-rotor itself and additional components such as sensors, lights and camera, and necessities for on-board data handling
Efficient 2D SLAM for a Mobile Robot with a Downwards Facing Camera
As digital cameras become cheaper and better, computers more powerful, and robots more abundant the merging of these three techniques also becomes more common and capable. The combination of these techniques is often inspired by the human visual system and often strives to give machines the same capabilities that humans already have, such as object identification, navigation, limb coordination, and event detection. One such field that is particularly popular is that of SLAM, or Simultaneous Localization and Mapping, which has high-profile applications in self-driving cars and delivery drones. This thesis proposes and describes an online SLAM algorithm for a specific scenario: that of a robot with a downwards facing camera exploring a flat surface (e.g., a floor). The method is based on building homographies from robot odometry data, which are then used to rectify the images so that the tilt of the camera with regards to the floor is eliminated, thereby moving the problem from 3D to 2D. The 2D pose of the robot in the plane is estimated using registrations of SURF features, and then a bundle adjustment algorithm is used to consolidate the most recent measurements with the older ones in order to optimize the map. The algorithm is implemented and tested with an AR.Drone 2.0 quadcopter. The results are mixed, but hardware seems to be the limiting factor: the algorithm performs well and runs at 5-20 Hz on a i5 desktop computer; but the bad quality, high compression and low resolution of the drone’s bottom camera makes the algorithm unstable and this cannot be overcome, even with several tiers of outlier filtering.För att robotar skall vara praktiska behöver de ha en flexibel uppfattning om sin omgivning och deras egen position i den, men de metoder som finns för detta idag är ofta väldigt krävande. I det här projektet har en förenklad metod för kartläggning i realtid med en drönare utvecklats. Algoritmen behandlar ett enklare problem än de vanliga tredimensionella problemen - istället för att titta framåt i rummet tittar drönaren neråt och försöker bygga en karta genom att pussla ihop bilder av golvet. Metoden är effektiv, men kvalitén på drönarens kamera som användes är för dålig för att metoden skall ge pålitliga resultat
Computer Vision without Vision : Methods and Applications of Radio and Audio Based SLAM
The central problem of this thesis is estimating receiver-sender node positions from measured receiver-sender distances or equivalent measurements. This problem arises in many applications such as microphone array calibration, radio antenna array calibration, mapping and positioning using ultra-wideband and mapping and positioning using round-trip-time measurements between mobile phones and Wi-Fi-units. Previous research has explored some of these problems, creating minimal solvers for instance, but these solutions lack real world implementation. Due to the nature of using different media, finding reliable receiver-sender distances is tough, with many of the measurements being erroneous or to a worse extent missing. Therefore in this thesis, we explore using minimal solvers to create robust solutions, that encompass small erroneous measurements and work around missing and grossly erroneous measurements.This thesis focuses mainly on Time-of-Arrival measurements using radio technologies such as Two-way-Ranging in Ultra-Wideband and a new IEEE standard 802.11mc found on many WiFi modules. The methods investigated, also related to Computer Vision problems such as Stucture-from-Motion. As part of this thesis, a range of new commercial radio technologies are characterised in terms of ranging in real world enviroments. In doing so, we have shown how these technologies can be used as a more accurate alternative to the Global Positioning System in indoor enviroments. Further to these solutions, more methods are proposed for large scale problems when multiple users will collect the data, commonly known as Big Data. For these cases, more data is not always better, so a method is proposed to try find the relevant data to calibrate large systems
Enriching remote labs with computer vision and drones
165 p.With the technological advance, new learning technologies are being developed in order to contribute to better learning experience. In particular, remote labs constitute an interesting and a practical way that can motivate nowadays students to learn. The studen can at anytime, and from anywhere, access the remote lab and do his lab-work. Despite many advantages, remote tecnologies in education create a distance between the student and the teacher. Without the presence of a teacher, students can have difficulties, if no appropriate interventions can be taken to help them. In this thesis, we aim to enrich an existing remote electronic lab made for engineering students called "LaboREM" (for remote Laboratory) in two ways: first we enable the student to send high level commands to a mini-drone available in the remote lab facility. The objective is to examine the front panels of electronic measurement instruments, by the camera embedded on the drone. Furthermore, we allow remote student-teacher communication using the drone, in case there is a teacher present in the remote lab facility. Finally, the drone has to go back home when the mission is over to land on a platform for automatic recharge of the batteries. Second, we propose an automatic system that estimates the affective state of the student (frustrated/confused/flow) in order to take appropriate interventions to ensure good learning outcomes. For example, if the studen is having major difficulties we can try to give him hints or to reduce the difficulty level of the lab experiment. We propose to do this by using visual cues (head pose estimation and facil expression analysis). Many evidences on the state of the student can be acquired, however these evidences are incomplete, sometims inaccurate, and do not cover all the aspects of the state of the student alone. This is why we propose to fuse evidences using the theory of Dempster-Shafer that allows the fusion of incomplete evidence
Enriching remote labs with computer vision and drones
165 p.With the technological advance, new learning technologies are being developed in order to contribute to better learning experience. In particular, remote labs constitute an interesting and a practical way that can motivate nowadays students to learn. The studen can at anytime, and from anywhere, access the remote lab and do his lab-work. Despite many advantages, remote tecnologies in education create a distance between the student and the teacher. Without the presence of a teacher, students can have difficulties, if no appropriate interventions can be taken to help them. In this thesis, we aim to enrich an existing remote electronic lab made for engineering students called "LaboREM" (for remote Laboratory) in two ways: first we enable the student to send high level commands to a mini-drone available in the remote lab facility. The objective is to examine the front panels of electronic measurement instruments, by the camera embedded on the drone. Furthermore, we allow remote student-teacher communication using the drone, in case there is a teacher present in the remote lab facility. Finally, the drone has to go back home when the mission is over to land on a platform for automatic recharge of the batteries. Second, we propose an automatic system that estimates the affective state of the student (frustrated/confused/flow) in order to take appropriate interventions to ensure good learning outcomes. For example, if the studen is having major difficulties we can try to give him hints or to reduce the difficulty level of the lab experiment. We propose to do this by using visual cues (head pose estimation and facil expression analysis). Many evidences on the state of the student can be acquired, however these evidences are incomplete, sometims inaccurate, and do not cover all the aspects of the state of the student alone. This is why we propose to fuse evidences using the theory of Dempster-Shafer that allows the fusion of incomplete evidence
Flight controller synthesis via deep reinforcement learning
Traditional control methods are inadequate in many deployment settings involving autonomous control of Cyber-Physical Systems (CPS). In such settings, CPS controllers must operate and respond to unpredictable interactions, conditions, or failure modes. Dealing with such unpredictability requires the use of executive and cognitive control functions that allow for planning and reasoning. Motivated by the sport of drone racing, this dissertation addresses these concerns for state-of-the-art flight control by investigating the use of deep artificial neural networks to bring essential elements of higher-level cognition to bear on the design, implementation, deployment, and evaluation of low level (attitude) flight controllers.
First, this thesis presents a feasibility analyses and results which confirm that neural networks, trained via reinforcement learning, are more accurate than traditional control methods used by commercial uncrewed aerial vehicles (UAVs) for attitude control. Second, armed with these results, this thesis reports on the development and release of an open source, full solution stack for building neuro-flight controllers. This stack consists of a tuning framework for implementing training environments (GymFC) and firmware for the world’s first neural network supported flight controller (Neuroflight). GymFC’s novel approach fuses together the digital twinning paradigm with flight control training to provide seamless transfer to hardware. Third, to transfer models synthesized by GymFC to hardware, this thesis reports on the toolchain that has been released for compiling neural networks into Neuroflight, which can be flashed to off-the-shelf microcontrollers. This toolchain includes detailed procedures for constructing a multicopter digital twin to allow the research and development community to synthesize flight controllers unique to their own aircraft. Finally, this thesis examines alternative reward system functions as well as changes to the software environment to bridge the gap between simulation and real world deployment environments.
The design, evaluation, and experimental work summarized in this thesis demonstrates that deep reinforcement learning is able to be leveraged for the design and implementation of neural network controllers capable not only of maintaining stable flight, but also precision aerobatic maneuvers in real world settings. As such, this work provides a foundation for developing the next generation of flight control systems
A Comparative Evaluation of the Detection and Tracking Capability Between Novel Event-Based and Conventional Frame-Based Sensors
Traditional frame-based technology continues to suffer from motion blur, low dynamic range, speed limitations and high data storage requirements. Event-based sensors offer a potential solution to these challenges. This research centers around a comparative assessment of frame and event-based object detection and tracking. A basic frame-based algorithm is used to compare against two different event-based algorithms. First event-based pseudo-frames were parsed through standard frame-based algorithms and secondly, target tracks were constructed directly from filtered events. The findings show there is significant value in pursuing the technology further
Vision-Based navigation system for unmanned aerial vehicles
MenciĂłn Internacional en el tĂtulo de doctorThe main objective of this dissertation is to provide Unmanned Aerial Vehicles
(UAVs) with a robust navigation system; in order to allow the UAVs to perform
complex tasks autonomously and in real-time. The proposed algorithms deal with
solving the navigation problem for outdoor as well as indoor environments, mainly
based on visual information that is captured by monocular cameras. In addition,
this dissertation presents the advantages of using the visual sensors as the main
source of data, or complementing other sensors in providing useful information; in
order to improve the accuracy and the robustness of the sensing purposes.
The dissertation mainly covers several research topics based on computer vision
techniques: (I) Pose Estimation, to provide a solution for estimating the 6D pose of
the UAV. This algorithm is based on the combination of SIFT detector and FREAK
descriptor; which maintains the performance of the feature points matching and decreases
the computational time. Thereafter, the pose estimation problem is solved
based on the decomposition of the world-to-frame and frame-to-frame homographies.
(II) Obstacle Detection and Collision Avoidance, in which, the UAV is able to
sense and detect the frontal obstacles that are situated in its path. The detection
algorithm mimics the human behaviors for detecting the approaching obstacles; by
analyzing the size changes of the detected feature points, combined with the expansion
ratios of the convex hull constructed around the detected feature points
from consecutive frames. Then, by comparing the area ratio of the obstacle and the
position of the UAV, the method decides if the detected obstacle may cause a collision.
Finally, the algorithm extracts the collision-free zones around the obstacle,
and combining with the tracked waypoints, the UAV performs the avoidance maneuver.
(III) Navigation Guidance, which generates the waypoints to determine
the flight path based on environment and the situated obstacles. Then provide
a strategy to follow the path segments and in an efficient way and perform the
flight maneuver smoothly. (IV) Visual Servoing, to offer different control solutions (Fuzzy Logic Control (FLC) and PID), based on the obtained visual information; in
order to achieve the flight stability as well as to perform the correct maneuver; to
avoid the possible collisions and track the waypoints.
All the proposed algorithms have been verified with real flights in both indoor
and outdoor environments, taking into consideration the visual conditions; such as
illumination and textures. The obtained results have been validated against other
systems; such as VICON motion capture system, DGPS in the case of pose estimate
algorithm. In addition, the proposed algorithms have been compared with several
previous works in the state of the art, and are results proves the improvement in
the accuracy and the robustness of the proposed algorithms.
Finally, this dissertation concludes that the visual sensors have the advantages
of lightweight and low consumption and provide reliable information, which is
considered as a powerful tool in the navigation systems to increase the autonomy
of the UAVs for real-world applications.El objetivo principal de esta tesis es proporcionar Vehiculos Aereos no Tripulados
(UAVs) con un sistema de navegacion robusto, para permitir a los UAVs realizar
tareas complejas de forma autonoma y en tiempo real. Los algoritmos propuestos
tratan de resolver problemas de la navegacion tanto en ambientes interiores como
al aire libre basandose principalmente en la informacion visual captada por las camaras
monoculares. Ademas, esta tesis doctoral presenta la ventaja de usar sensores
visuales bien como fuente principal de datos o complementando a otros sensores
en el suministro de informacion util, con el fin de mejorar la precision y la
robustez de los procesos de deteccion.
La tesis cubre, principalmente, varios temas de investigacion basados en tecnicas
de vision por computador: (I) Estimacion de la Posicion y la Orientacion
(Pose), para proporcionar una solucion a la estimacion de la posicion y orientacion
en 6D del UAV. Este algoritmo se basa en la combinacion del detector SIFT y el
descriptor FREAK, que mantiene el desempeno del a funcion de puntos de coincidencia
y disminuye el tiempo computacional. De esta manera, se soluciona el
problema de la estimacion de la posicion basandose en la descomposicion de las
homografias mundo a imagen e imagen a imagen. (II) Deteccion obstaculos y elusion
colisiones, donde el UAV es capaz de percibir y detectar los obstaculos frontales
que se encuentran en su camino. El algoritmo de deteccion imita comportamientos
humanos para detectar los obstaculos que se acercan, mediante el analisis de la
magnitud del cambio de los puntos caracteristicos detectados de referencia, combinado
con los ratios de expansion de los contornos convexos construidos alrededor
de los puntos caracteristicos detectados en frames consecutivos. A continuacion,
comparando la proporcion del area del obstaculo y la posicion del UAV, el metodo
decide si el obstaculo detectado puede provocar una colision. Por ultimo, el algoritmo
extrae las zonas libres de colision alrededor del obstaculo y combinandolo
con los puntos de referencia, elUAV realiza la maniobra de evasion. (III) Guiado de navegacion, que genera los puntos de referencia para determinar la trayectoria de
vuelo basada en el entorno y en los obstaculos detectados que encuentra. Proporciona
una estrategia para seguir los segmentos del trazado de una manera eficiente
y realizar la maniobra de vuelo con suavidad. (IV) Guiado por Vision, para ofrecer
soluciones de control diferentes (Control de Logica Fuzzy (FLC) y PID), basados en
la informacion visual obtenida con el fin de lograr la estabilidad de vuelo, asi como
realizar la maniobra correcta para evitar posibles colisiones y seguir los puntos de
referencia.
Todos los algoritmos propuestos han sido verificados con vuelos reales en ambientes
exteriores e interiores, tomando en consideracion condiciones visuales como
la iluminacion y las texturas. Los resultados obtenidos han sido validados con otros
sistemas: como el sistema de captura de movimiento VICON y DGPS en el caso del
algoritmo de estimacion de la posicion y orientacion. Ademas, los algoritmos propuestos
han sido comparados con trabajos anteriores recogidos en el estado del arte
con resultados que demuestran una mejora de la precision y la robustez de los algoritmos
propuestos.
Esta tesis doctoral concluye que los sensores visuales tienen las ventajes de tener
un peso ligero y un bajo consumo y, proporcionar informacion fiable, lo cual lo
hace una poderosa herramienta en los sistemas de navegacion para aumentar la
autonomia de los UAVs en aplicaciones del mundo real.Programa Oficial de Doctorado en IngenierĂa ElĂ©ctrica, ElectrĂłnica y AutomáticaPresidente: Carlo Regazzoni.- Secretario: Fernando GarcĂa Fernández.- Vocal: Pascual Campoy Cerver
UAV for Medical Equipment Distribution
This Final Design Review report documents the senior project participating in the Vertical Flight Society’s 38th Annual Student Design Competition sponsored by The Boeing Company. The goal of this project and competition is to develop an unmanned vertical lift for medical equipment distribution capable of safely delivering a 50 kg payload over distances up to 200 km. This system must be autonomous and have a backup plan to land if any part of the system malfunctions. We discuss the research and justification that drove the selection of the aircraft configuration, a winged quadcopter with a rear propeller. Furthermore, we document our reasoning and analysis for sizing and shaping of the rotors, propeller, and wings, selecting a hybrid-electric turbogenerator for the powerplant, designing the payload release mechanism, and sizing and shaping of the semi-monocoque structure. We provide analysis that numerically verifies our UAV’s ability to meet the requirements for payload capacity, range, mission time, and geometric envelope
- …