Vision-based Autonomous Tracking of a Non-cooperative Mobile Robot by a Low-cost Quadrotor Vehicle

The goal of this thesis is the detection and tracking of a ground vehicle, in particular a car-like robot, by a quadrotor. The first challenge to address in any pursuit or tracking scenario is the detection and unique identification of the target. From this first challenge, comes the need to precisely localize the target in a coordinate system that is common to the tracking and tracked vehicles. In most real-life scenarios, the tracked vehicle does not directly communicate information such as its position to the tracking one. From this fact, arises a non-cooperative constraint problem. The autonomous tracking aspect of the mission requires, for both the aerial and ground vehicles, robust pose estimation during the mission. The primary and crucial functions to achieve autonomous behaviors are control and navigation. The principal-agent being the quadrotor, this thesis explains in detail the derivation and analysis of the equations of motion that govern its natural behavior along with the control methods that permit to achieve desired performances. The analysis of these equations reveals a naturally unstable system, subject to non-linearities. Therefore, we explored three different control methods capable of guaranteeing stability while mitigating non-linearities. The first two control methods operate in the linear region and consist of the intuitive Proportional Integrate Derivative controller (PID). The second linear control strategy is represented by an optimal controller that is the Linear Quadratic Regulator controller (LQR). The last and final control method is a nonlinear controller designed from the Sliding Mode Control Theory. In addition to the in-depth analysis, we provide assets and limitations of each control method. In order to achieve the tracking mission, we address the detection and localization problems using respectively visual servoing and frame transform techniques. The pose estimation challenge for the aerial robot is cleared up using Kalman Filtering estimation methods that are also explored in depth. The same estimation method is used to mitigate the ground vehicle’s real-time pose estimation and tracking problem. Analysis results are illustrated using Matlab. A simulation and a real implementation using the Robot Operating System are used to support the obtained results

Visual Servoing Approach for Autonomous UAV Landing on a Moving Vehicle

We present a method to autonomously land an Unmanned Aerial Vehicle on a moving vehicle with a circular (or elliptical) pattern on the top. A visual servoing controller approaches the ground vehicle using velocity commands calculated directly in image space. The control laws generate velocity commands in all three dimensions, eliminating the need for a separate height controller. The method has shown the ability to approach and land on the moving deck in simulation, indoor and outdoor environments, and compared to the other available methods, it has provided the fastest landing approach. It does not rely on additional external setup, such as RTK, motion capture system, ground station, offboard processing, or communication with the vehicle, and it requires only a minimal set of hardware and localization sensors. The videos and source codes can be accessed from http://theairlab.org/landing-on-vehicle.Comment: 24 page

Investigations of Model-Free Sliding Mode Control Algorithms including Application to Autonomous Quadrotor Flight

Sliding mode control is a robust nonlinear control algorithm that has been used to implement tracking controllers for unmanned aircraft systems that are robust to modeling uncertainty and exogenous disturbances, thereby providing excellent performance for autonomous operation. A significant advance in the application of sliding mode control for unmanned aircraft systems would be adaptation of a model-free sliding mode control algorithm, since the most complex and time-consuming aspect of implementation of sliding mode control is the derivation of the control law with incorporation of the system model, a process required to be performed for each individual application of sliding mode control. The performance of four different model-free sliding mode control algorithms was compared in simulation using a variety of aerial system models and real-world disturbances (e.g. the effects of discretization and state estimation). The two best performing algorithms were shown to exhibit very similar behavior. These two algorithms were implemented on a quadrotor (both in simulation and using real-world hardware) and the performance was compared to a traditional PID-based controller using the same state estimation algorithm and control setup. Simulation results show the model-free sliding mode control algorithms exhibit similar performance to PID controllers without the tedious tuning process. Comparison between the two model-free sliding mode control algorithms showed very similar performance as measured by the quadratic means of tracking errors. Flight testing showed that while a model-free sliding mode control algorithm is capable of controlling realworld hardware, further characterization and significant improvements are required before it is a viable alternative to conventional control algorithms. Large tracking errors were observed for both the model-free sliding mode control and PID based flight controllers and the performance was characterized as unacceptable for most applications. The poor performance of both controllers suggests tracking errors could be attributed to errors in state estimation, which effectively introduce unknown dynamics into the feedback loop. Further testing with improved state estimation would allow for more conclusions to be drawn about the performance characteristics of the model-free sliding mode control algorithms

Vision-Based Target Tracking and Autonomous Landing of a Quadrotor on a Ground Vehicle

This paper addresses vision-based tracking and landing of a micro-aerial vehicle (MAV) on a ground vehicle (GV). The camera onboard the MAV is mounted so that the optical axis is aligned with the downward-facing axis of the body-fixed frame. A novel supervised learning vision algorithm is proposed as the method to detect the ground vehicle in the image frame. A feedback linearization technique is developed for the MAV to fly over and track the GV so that visibility with the tracked target is maintained with certain guarantees. The efficacy of the visual detection algorithm, and of the tracking and landing controller is demonstrated in simulations and experiments with static and mobile GV

Exploiting Heterogeneity in Networks of Aerial and Ground Robotic Agents

By taking advantage of complementary communication technologies, distinct sensing functionalities and varied motion dynamics present in a heterogeneous multi-robotic network, it is possible to accomplish a main mission objective by assigning specialized sub-tasks to specific members of a robotic team. An adequate selection of the team members and an effective coordination are some of the challenges to fully exploit the unique capabilities that these types of systems can offer. Motivated by real world applications, we focus on a multi-robotic network consisting off aerial and ground agents which has the potential to provide critical support to humans in complex settings. For instance, aerial robotic relays are capable of transporting small ground mobile sensors to expand the communication range and the situational awareness of first responders in hazardous environments. In the first part of this dissertation, we extend work on manipulation of cable-suspended loads using aerial robots by solving the problem of lifting the cable-suspended load from the ground before proceeding to transport it. Since the suspended load-quadrotor system experiences switching conditions during this critical maneuver, we define a hybrid system and show that it is differentially-flat. This property facilitates the design of a nonlinear controller which tracks a waypoint-based trajectory associated with the discrete states of the hybrid system. In addition, we address the case of unknown payload mass by combining a least-squares estimation method with the designed controller. Second, we focus on the coordination of a heterogeneous team formed by a group of ground mobile sensors and a flying communication router which is deployed to sense areas of interest in a cluttered environment. Using potential field methods, we propose a controller for the coordinated mobility of the team to guarantee inter-robot and obstacle collision avoidance as well as connectivity maintenance among the ground agents while the main goal of sensing is carried out. For the case of the aerial communications relays, we combine antenna diversity with reinforcement learning to dynamically re-locate these relays so that the received signal strength is maintained above a desired threshold. Motivated by the recent interest of combining radio frequency and optical wireless communications, we envision the implementation of an optical link between micro-scale aerial and ground robots. This type of link requires maintaining a sufficient relative transmitter-receiver position for reliable communications. In the third part of this thesis, we tackle this problem. Based on the link model, we define a connectivity cone where a minimum transmission rate is guaranteed. For example, the aerial robot has to track the ground vehicle to stay inside this cone. The control must be robust to noisy measurements. Thus, we use particle filters to obtain a better estimation of the receiver position and we design a control algorithm for the flying robot to enhance the transmission rate. Also, we consider the problem of pairing a ground sensor with an aerial vehicle, both equipped with a hybrid radio-frequency/optical wireless communication system. A challenge is positioning the flying robot within optical range when the sensor location is unknown. Thus, we take advantage of the hybrid communication scheme by developing a control strategy that uses the radio signal to guide the aerial platform to the ground sensor. Once the optical-based signal strength has achieved a certain threshold, the robot hovers within optical range. Finally, we investigate the problem of building an alliance of agents with different skills in order to satisfy the requirements imposed by a given task. We find this alliance, known also as a coalition, by using a bipartite graph in which edges represent the relation between agent capabilities and required resources for task execution. Using this graph, we build a coalition whose total capability resources can satisfy the task resource requirements. Also, we study the heterogeneity of the formed coalition to analyze how it is affected for instance by the amount of capability resources present in the agents

Real-time control architecture for a multi UAV test bed

The purpose of this thesis is to develop a control architecture running at real-time for a multi unmanned aerial vehicle test bed formed by three AscTec Hummingbird mini quadrotors. The reliable and reconfigurable architecture presented here has a FPGA-based embedded system as main controller. Under the implemented control system, different practical applications have been performed in the MARHES Lab at the University of New Mexico as part of its research in cooperative control of mobile aerial agents. This thesis also covers the quadrotor modeling, the design of a position controller, the real-time architecture implementation and the experimental flight tests. A hybrid approach combining first-principles with system identification techniques is used for modeling the quadrotor due to the lack of information around the structure of the onboard controller designed by AscTec. The complete quadrotor model structure is formed by a black-box subsystem and a point-mass submodel. Experimental data have been gathered for system identification and black-box submodel validation purposes; while the point-mass submodel is found applying rigid-body dynamics. Using the dynamical model, a position control block based in lead-lag and PI compensators is developed and simulated. Improvements in trajectory tracking performance are achieved estimating the linear velocity of the aerial robot and incorporating velocity lead-lag compensators to the control approach. The velocity of the aerial robot is computed by numerical differentiation of position data. Simulation results to a variety of input signals of the control block in cascade with the complete dynamic model of the quadrotor are included. The control block together with the velocity estimation is fully programmed in the embedded controller. A graphical user interface, GUI, as part of the architecture is designed to display real-time data of position and orientation streamed from the motion tracking system as well as to contain useful user controllers. This GUI facilitates that a single operator conducts and oversees all aspects of the different applications where one or multiple quadrotors are used. Experimental tests have helped to tune the control parameters determined by simulation. The performance of the whole architecture has been validated through a variety of practical applications. Autonomous take off, hovering and landing, target surveillance, trajectory tracking and suspended payload transportation are just some of the applications carried out employing the real-time control architecture proposed in this thesis

Model predictive cooperative localization control of multiple UAVs using potential function sensor constraints: a workflow to create sensor constraint based potential functions for the control of cooperative localization scenarios with mobile robots.

The global localization of multiple mobile robots can be achieved cost efficiently by localizing one robot globally and the others in relation to it using local sensor data. However, the drawback of this cooperative localization is the requirement of continuous sensor information. Due to a limited sensor perception space, the tracking task to continuously maintain this sensor information is challenging. To address this problem, this contribution is presenting a model predictive control (MPC) approach for such cooperative localization scenarios. In particular, the present work shows a novel workflow to describe sensor limitations with the help of potential functions. In addition, a compact motion model for multi-rotor drones is introduced to achieve MPC real-time capability. The effectiveness of the presented approach is demonstrated in a numerical simulation, an experimental indoor scenario with two quadrotors as well as multiple indoor scenarios of a quadrotor obstacle evasion maneuver