38 research outputs found
Visual control of multi-rotor UAVs
Recent miniaturization of computer hardware, MEMs sensors, and high energy density
batteries have enabled highly capable mobile robots to become available at low cost.
This has driven the rapid expansion of interest in multi-rotor unmanned aerial vehicles.
Another area which has expanded simultaneously is small powerful computers, in the
form of smartphones, which nearly always have a camera attached, many of which now
contain a OpenCL compatible graphics processing units. By combining the results of
those two developments a low-cost multi-rotor UAV can be produced with a low-power
onboard computer capable of real-time computer vision. The system should also use
general purpose computer vision software to facilitate a variety of experiments.
To demonstrate this I have built a quadrotor UAV based on control hardware from
the Pixhawk project, and paired it with an ARM based single board computer, similar
those in high-end smartphones. The quadrotor weights 980 g and has a flight time of
10 minutes. The onboard computer capable of running a pose estimation algorithm
above the 10 Hz requirement for stable visual control of a quadrotor.
A feature tracking algorithm was developed for efficient pose estimation, which relaxed
the requirement for outlier rejection during matching. Compared with a RANSAC-
only algorithm the pose estimates were less variable with a Z-axis standard deviation
0.2 cm compared with 2.4 cm for RANSAC. Processing time per frame was also faster
with tracking, with 95 % confidence that tracking would process the frame within 50 ms,
while for RANSAC the 95 % confidence time was 73 ms. The onboard computer ran the
algorithm with a total system load of less than 25 %. All computer vision software uses
the OpenCV library for common computer vision algorithms, fulfilling the requirement
for running general purpose software.
The tracking algorithm was used to demonstrate the capability of the system by per-
forming visual servoing of the quadrotor (after manual takeoff). Response to external
perturbations was poor however, requiring manual intervention to avoid crashing. This
was due to poor visual controller tuning, and to variations in image acquisition and
attitude estimate timing due to using free running image acquisition.
The system, and the tracking algorithm, serve as proof of concept that visual control of
a quadrotor is possible using small low-power computers and general purpose computer
vision software
Vision-Based navigation system for unmanned aerial vehicles
Mención Internacional en el título de doctorThe main objective of this dissertation is to provide Unmanned Aerial Vehicles
(UAVs) with a robust navigation system; in order to allow the UAVs to perform
complex tasks autonomously and in real-time. The proposed algorithms deal with
solving the navigation problem for outdoor as well as indoor environments, mainly
based on visual information that is captured by monocular cameras. In addition,
this dissertation presents the advantages of using the visual sensors as the main
source of data, or complementing other sensors in providing useful information; in
order to improve the accuracy and the robustness of the sensing purposes.
The dissertation mainly covers several research topics based on computer vision
techniques: (I) Pose Estimation, to provide a solution for estimating the 6D pose of
the UAV. This algorithm is based on the combination of SIFT detector and FREAK
descriptor; which maintains the performance of the feature points matching and decreases
the computational time. Thereafter, the pose estimation problem is solved
based on the decomposition of the world-to-frame and frame-to-frame homographies.
(II) Obstacle Detection and Collision Avoidance, in which, the UAV is able to
sense and detect the frontal obstacles that are situated in its path. The detection
algorithm mimics the human behaviors for detecting the approaching obstacles; by
analyzing the size changes of the detected feature points, combined with the expansion
ratios of the convex hull constructed around the detected feature points
from consecutive frames. Then, by comparing the area ratio of the obstacle and the
position of the UAV, the method decides if the detected obstacle may cause a collision.
Finally, the algorithm extracts the collision-free zones around the obstacle,
and combining with the tracked waypoints, the UAV performs the avoidance maneuver.
(III) Navigation Guidance, which generates the waypoints to determine
the flight path based on environment and the situated obstacles. Then provide
a strategy to follow the path segments and in an efficient way and perform the
flight maneuver smoothly. (IV) Visual Servoing, to offer different control solutions (Fuzzy Logic Control (FLC) and PID), based on the obtained visual information; in
order to achieve the flight stability as well as to perform the correct maneuver; to
avoid the possible collisions and track the waypoints.
All the proposed algorithms have been verified with real flights in both indoor
and outdoor environments, taking into consideration the visual conditions; such as
illumination and textures. The obtained results have been validated against other
systems; such as VICON motion capture system, DGPS in the case of pose estimate
algorithm. In addition, the proposed algorithms have been compared with several
previous works in the state of the art, and are results proves the improvement in
the accuracy and the robustness of the proposed algorithms.
Finally, this dissertation concludes that the visual sensors have the advantages
of lightweight and low consumption and provide reliable information, which is
considered as a powerful tool in the navigation systems to increase the autonomy
of the UAVs for real-world applications.El objetivo principal de esta tesis es proporcionar Vehiculos Aereos no Tripulados
(UAVs) con un sistema de navegacion robusto, para permitir a los UAVs realizar
tareas complejas de forma autonoma y en tiempo real. Los algoritmos propuestos
tratan de resolver problemas de la navegacion tanto en ambientes interiores como
al aire libre basandose principalmente en la informacion visual captada por las camaras
monoculares. Ademas, esta tesis doctoral presenta la ventaja de usar sensores
visuales bien como fuente principal de datos o complementando a otros sensores
en el suministro de informacion util, con el fin de mejorar la precision y la
robustez de los procesos de deteccion.
La tesis cubre, principalmente, varios temas de investigacion basados en tecnicas
de vision por computador: (I) Estimacion de la Posicion y la Orientacion
(Pose), para proporcionar una solucion a la estimacion de la posicion y orientacion
en 6D del UAV. Este algoritmo se basa en la combinacion del detector SIFT y el
descriptor FREAK, que mantiene el desempeno del a funcion de puntos de coincidencia
y disminuye el tiempo computacional. De esta manera, se soluciona el
problema de la estimacion de la posicion basandose en la descomposicion de las
homografias mundo a imagen e imagen a imagen. (II) Deteccion obstaculos y elusion
colisiones, donde el UAV es capaz de percibir y detectar los obstaculos frontales
que se encuentran en su camino. El algoritmo de deteccion imita comportamientos
humanos para detectar los obstaculos que se acercan, mediante el analisis de la
magnitud del cambio de los puntos caracteristicos detectados de referencia, combinado
con los ratios de expansion de los contornos convexos construidos alrededor
de los puntos caracteristicos detectados en frames consecutivos. A continuacion,
comparando la proporcion del area del obstaculo y la posicion del UAV, el metodo
decide si el obstaculo detectado puede provocar una colision. Por ultimo, el algoritmo
extrae las zonas libres de colision alrededor del obstaculo y combinandolo
con los puntos de referencia, elUAV realiza la maniobra de evasion. (III) Guiado de navegacion, que genera los puntos de referencia para determinar la trayectoria de
vuelo basada en el entorno y en los obstaculos detectados que encuentra. Proporciona
una estrategia para seguir los segmentos del trazado de una manera eficiente
y realizar la maniobra de vuelo con suavidad. (IV) Guiado por Vision, para ofrecer
soluciones de control diferentes (Control de Logica Fuzzy (FLC) y PID), basados en
la informacion visual obtenida con el fin de lograr la estabilidad de vuelo, asi como
realizar la maniobra correcta para evitar posibles colisiones y seguir los puntos de
referencia.
Todos los algoritmos propuestos han sido verificados con vuelos reales en ambientes
exteriores e interiores, tomando en consideracion condiciones visuales como
la iluminacion y las texturas. Los resultados obtenidos han sido validados con otros
sistemas: como el sistema de captura de movimiento VICON y DGPS en el caso del
algoritmo de estimacion de la posicion y orientacion. Ademas, los algoritmos propuestos
han sido comparados con trabajos anteriores recogidos en el estado del arte
con resultados que demuestran una mejora de la precision y la robustez de los algoritmos
propuestos.
Esta tesis doctoral concluye que los sensores visuales tienen las ventajes de tener
un peso ligero y un bajo consumo y, proporcionar informacion fiable, lo cual lo
hace una poderosa herramienta en los sistemas de navegacion para aumentar la
autonomia de los UAVs en aplicaciones del mundo real.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Carlo Regazzoni.- Secretario: Fernando García Fernández.- Vocal: Pascual Campoy Cerver
Single chip solution for stabilization control & monocular visual servoing of small-scale quadrotor helicopter
This thesis documents the research undertaken to develop a high-performing design
of a small-scale quadrotor (four-rotor) helicopter capable of delivering the speed and
robustness required for agile motion while also featuring an autonomous visual servoing
capability within the size, weight, and power (SWaP) constraint package. The
state of the art research was reviewed, and the areas in the existing design methodologies
that can potentially be improved were identified, which included development
of a comprehensive dynamics model of quadrotor, design and construction of a performance
optimized prototype vehicle, high-performance actuator design, design of a
robust attitude stabilization controller, and a single chip solution for autonomous vision
based position control. The gaps in the current art of designing each component
were addressed individually. The outcomes of the corresponding development activities
include a high-fidelity dynamics and control model of the vehicle. The model
was developed using multi-body bond graph modeling approach to incorporate the
dynamic interactions between the frame body and propulsion system. Using an algorithmic
size, payload capacity, and flight endurance optimization approach, a quadrotor
prototype was designed and constructed. In order to conform to the optimized
geometric and performance parameters, the frame of the prototype was constructed
using printed circuit board (PCB) technology and processing power was integrated
using a single chip field programmable gate array (FPGA) technology. Furthermore, to actuate the quadrotor at a high update rate while also improving the power efficiency
of the actuation system, a ground up FPGA based brushless direct current
(BLDC) motor driver was designed using a low-loss commutation scheme and hall
effect sensors. A proportional-integral-derivative (PID) technology based closed loop
motor speed controller was also implemented in the same FPGA hardware for precise
speed control of the motors. In addition, a novel control law was formulated for robust
attitude stabilization by adopting a cascaded architecture of active disturbance rejection
control (ADRC) technology and PID control technology. Using the same single
FPGA chip to drive an on-board downward looking camera, a monocular visual servoing
solution was developed to integrate an autonomous position control feature with
the quadrotor. Accordingly, a numerically simple relative position estimation technique
was implemented in FPGA hardware that relies on a passive landmark/target
for 3-D position estimation.
The functionality and effectiveness of the synthesized design were evaluated by
performance benchmarking experiments conducted on each individual component as
well as on the complete system constructed from these components. It was observed
that the proposed small-scale quadrotor, even though just 43 cm in diameter, can lift
434 gm of payload while operating for 18 min. Among the ground up designed components,
the FPGA based motor driver demonstrated a maximum of 4% improvement in
the power consumption and at the same time can handle a command update at a rate
of 16 kHz. The cascaded attitude stabilization controller can asymptotically stabilize
the vehicle within 426 ms of the command update. Robust control performance under
stochastic wind gusts is also observed from the stabilization controller. Finally, the
single chip FPGA based monocular visual servoing solution can estimate pose information
at the camera rate of 37 fps and accordingly the quadrotor can autonomously
climb/descend and/or hover over a passive target
A survey of single and multi-UAV aerial manipulation
Aerial manipulation has direct application prospects in environment, construction, forestry, agriculture, search, and rescue. It can be used to pick and place objects and hence can be used for transportation of goods. Aerial manipulation can be used to perform operations in environments inaccessible or unsafe for human workers. This paper is a survey of recent research in aerial manipulation. The aerial manipulation research has diverse aspects, which include the designing of aerial manipulation platforms, manipulators, grippers, the control of aerial platform and manipulators, the interaction of aerial manipulator with the environment, through forces and torque. In particular, the review paper presents the survey of the airborne platforms that can be used for aerial manipulation including the new aerial platforms with aerial manipulation capability. We also classified the aerial grippers and aerial manipulators based on their designs and characteristics. The recent contributions regarding the control of the aerial manipulator platform is also discussed. The environment interaction of aerial manipulators is also surveyed which includes, different strategies used for end-effectors interaction with the environment, application of force, application of torque and visual servoing. A recent and growing interest of researchers about the multi-UAV collaborative aerial manipulation was also noticed and hence different strategies for collaborative aerial manipulation are also surveyed, discussed and critically analyzed. Some key challenges regarding outdoor aerial manipulation and energy constraints in aerial manipulation are also discussed
Vision-based automatic landing of a rotary UAV
A hybrid-like (continuous and discrete-event) approach to controlling a small multi-rotor unmanned aerial system (UAS) while landing on a moving platform is described. The landing scheme is based on positioning visual markers on a landing platform in a detectable pattern. After the onboard camera detects the object pattern, the inner control algorithm sends visual-based servo-commands to align the multi-rotor with the targets. This method is less computationally complex as it uses color-based object detection applied to a geometric pattern instead of feature tracking algorithms, and has the advantage of not requiring the distance to the objects to be calculated. The continuous approach accounts for the UAV and the platform rolling/pitching/yawing, which is essential for a real-time landing on a moving target such as a ship.
A discrete-event supervisor working in parallel with the inner controller is designed to assist the automatic landing of a multi-rotor UAV on a moving target. This supervisory control strategy allows the pilot and crew to make time-critical decisions when exceptions, such as losing targets from the field of view, occur. The developed supervisor improves the low-level vision-based auto-landing system and high-level human-machine interface.
The proposed hybrid-like approach was tested in simulation using a quadcopter model in Virtual Robotics Experimentation Platform (V-REP) working in parallel with Robot Operating System (ROS). Finally, this method was validated in a series of real-time experiments with indoor and outdoor quadcopters landing on both static and moving platforms. The developed prototype system has demonstrated the capability of landing within 25 cm of the desired point of touchdown. This auto-landing system is small (100 x 100 mm), light-weight (100 g), and consumes little power (under 2 W)
Development Of A Quadrotor Testbed For Control And Sensor Development
A quadrotor is an under actuated unmanned aerial vehicle (UAV) which uses thrust from four rotors to provide six degrees of freedom. This thesis outlines the development of a general purpose test bed that can be used for sensor and control algorithm development. The system includes the means to simulate a proposed controller and then a hardware in the loop implementation using the same software. The test bed was assembled and verified with a linear controller for both attitude and position control using feedback from an IMU (Inertial measurement Unit) and a Global Position System (GPS) sensor. The linear controller was first implemented as a PID controller which attempts to control the attitude of the quadrotor. The controller was simulated successfully and then experiments were conducted on a DraganFlyer X-Pro quadrotor to verify the closed loop control. The experiments conducted checked the response of the quadrotor angles to the commanded angles. The controller gains were tuned to provide stable hover in all three angles. The Videre stereo vision system was investigated as a sensor to estimate height of the UAV above the ground. Experiments were performed that show that show static (no motion of the camera) estimates over the range 0.5 - 4 meters. The accuracy of these measurements suggest that the system may provide improved height estimation, over WAAS corrected GPS. A means to add this sensor into the UAV test bed is discussed
Autonomous Aerial Manipulation Using a Quadrotor
This paper presents an implementation of autonomous indoor aerial gripping using a low-cost, custom-built quadrotor. Such research extends the typical functionality of micro air vehicles (MAV) from passive observation and sensing to dynamic interaction with the environment. To achieve this, three major challenges are overcome: precise positioning, sensing and manipulation of the object, and stabilization in the presence of disturbance due to interaction with the object. Navigation in both indoor and outdoor unstructured, Global Positioning System-denied (GPS-denied) environments is achieved using a visual Simultaneous Localization and Mapping (SLAM) algorithm that relies on an onboard monocular camera. A secondary camera, capable of detecting infrared light sources, is used to estimate the 3D location of the object, while an under-actuated and passively compliant manipulator is designed for effective gripping under uncertainty. The system utilizes nested ProportionalIntegral-Derivative (PID) controllers for attitude stabilization, vision-based navigation, and gripping. The quadrotor is therefore able to autonomously navigate, locate, and grasp an object, using only onboard sensors
Autonomous Visual Navigation of a Quadrotor VTOL in complex and dense environments
This thesis presents a system design of a micro aerial vehicle platform, specifically a quadrotor, that is aimed at autonomous vision-based reactive obstacle avoidance in dense and complex environments. Most modern aerial system are incapable of autonomously navigating in environments with a high density of trees and bushes. The presented quadrotor design uses leading-edge technologies and inexpensive off-the-shelf components to build a system that presents a step forward in technologies aimed at overcoming the issues with dense and complex environments.
Several major system requirements were met to make the design effective and safe. It had to be completely autonomous in standard operations and have a manual override function. It had
to have its computational capability completely on-board along with vision processing ability. As such, all state estimation and visual guidance had to be performed on-board the vehicle,
removing the need for remote connection which can easily fail in forest-like environments. The quadrotor had to be made from mostly off-the-shelf components to reduce cost and make
it replicable. It also had to remain under 2kg to meet Australian commercial aerial vehicle regulations regarding licencing.
In order to meet the system requirements, many design decisions were developed and altered as needed. The main body of the quadrotor platform was based on off-the-shelf hobby
assemblies. A Pixhawk 2.1 was the flight controller used due to its open-source code and design which included all sensors needed for state estimation, has manual override for control,
and control the motors. A leading-edge computational device called the NVIDIA Tegra TX2 was used for vision processing on the quadrotor. The NVIDIA Tegra TX2's embedded
NVIDIA Graphics Processing Unit (GPU), is compact and consumes low amounts of power. It also is capable of estimating dense optical flow on the GPU at rates of 120Hz when using
a camera that outputs grey-scale images at a resolution of 376x240. The vision processor is responsible for providing directional guidance to the on-board flight controller. A design
decision during the project was to include a 3-axis gimbal to stabilise the camera. The quadrotor was shown to be able to hover and locally move both indoors and outdoors
using the optical flow measurements. Optical flow measurements give a sense of velocity which can be integrated to get a position estimate, though it was susceptible to drift. The drift
was compensated using a combination of recognisable targets and positioning systems such as GPS.
The experimental data obtained during the project showed that the algorithms presented in this thesis are capable of performing reactive obstacle avoidance. The reactive obstacle
avoidance experiments were performed in both simulation and in real world environments, including the dense forest-like environments. By fusing vehicle speed estimates with optical
flow measurements, visible points in 3D space can have their distance estimated relative to the quadrotor. By projecting a 3D cylinder in the direction of travel onto the camera plane, the
system can perform reactive obstacle avoidance by steering the cylinder (direction of travel) to a point with minimal interference. This system is intended to augment a point to point
navigation system such that the quadrotor responds to fine obstacle that may have otherwise not been detected