A Novel Real-Time Moving Target Tracking and Path Planning System for a Quadrotor UAV in Unknown Unstructured Outdoor Scenes

A quadrotor unmanned aerial vehicle (UAV) should have the ability to perform real-time target tracking and path planning simultaneously even when the target enters unstructured scenes, such as groves or forests. To accomplish this task, a novel system framework is designed and proposed to accomplish simultaneous moving target tracking and path planning by a quadrotor UAV with an onboard embedded computer, vision sensors, and a two-dimensional laser scanner. A support vector machine-based target screening algorithm is deployed to select the correct target from multiple candidates detected by single shot multibox detector. Furthermore, a new tracker named TLD-KCF is presented in this paper, in which a conditional scale adaptive algorithm is adopted to improve the tracking performance for a quadrotor UAV in cluttered outdoor environments. According to distance and position estimation for a moving target, our quadrotor UAV can acquire a control point to guide its fight. To reduce the computational burden, a fast path planning algorithm is proposed based on elliptical tangent model. A series of experiments are conducted on our quadrotor UAV platform DJI M100. Experimental video and comparison results among four kinds of target tracking algorithms are given to show the validity and practicality of the proposed approach

Fast, Autonomous Flight in GPS-Denied and Cluttered Environments

One of the most challenging tasks for a flying robot is to autonomously navigate between target locations quickly and reliably while avoiding obstacles in its path, and with little to no a-priori knowledge of the operating environment. This challenge is addressed in the present paper. We describe the system design and software architecture of our proposed solution, and showcase how all the distinct components can be integrated to enable smooth robot operation. We provide critical insight on hardware and software component selection and development, and present results from extensive experimental testing in real-world warehouse environments. Experimental testing reveals that our proposed solution can deliver fast and robust aerial robot autonomous navigation in cluttered, GPS-denied environments.Comment: Pre-peer reviewed version of the article accepted in Journal of Field Robotic

Learning Unmanned Aerial Vehicle Control for Autonomous Target Following

While deep reinforcement learning (RL) methods have achieved unprecedented successes in a range of challenging problems, their applicability has been mainly limited to simulation or game domains due to the high sample complexity of the trial-and-error learning process. However, real-world robotic applications often need a data-efficient learning process with safety-critical constraints. In this paper, we consider the challenging problem of learning unmanned aerial vehicle (UAV) control for tracking a moving target. To acquire a strategy that combines perception and control, we represent the policy by a convolutional neural network. We develop a hierarchical approach that combines a model-free policy gradient method with a conventional feedback proportional-integral-derivative (PID) controller to enable stable learning without catastrophic failure. The neural network is trained by a combination of supervised learning from raw images and reinforcement learning from games of self-play. We show that the proposed approach can learn a target following policy in a simulator efficiently and the learned behavior can be successfully transferred to the DJI quadrotor platform for real-world UAV control

Vision and Learning for Deliberative Monocular Cluttered Flight

Cameras provide a rich source of information while being passive, cheap and lightweight for small and medium Unmanned Aerial Vehicles (UAVs). In this work we present the first implementation of receding horizon control, which is widely used in ground vehicles, with monocular vision as the only sensing mode for autonomous UAV flight in dense clutter. We make it feasible on UAVs via a number of contributions: novel coupling of perception and control via relevant and diverse, multiple interpretations of the scene around the robot, leveraging recent advances in machine learning to showcase anytime budgeted cost-sensitive feature selection, and fast non-linear regression for monocular depth prediction. We empirically demonstrate the efficacy of our novel pipeline via real world experiments of more than 2 kms through dense trees with a quadrotor built from off-the-shelf parts. Moreover our pipeline is designed to combine information from other modalities like stereo and lidar as well if available

Unifying Foundation Models with Quadrotor Control for Visual Tracking Beyond Object Categories

Visual control enables quadrotors to adaptively navigate using real-time sensory data, bridging perception with action. Yet, challenges persist, including generalization across scenarios, maintaining reliability, and ensuring real-time responsiveness. This paper introduces a perception framework grounded in foundation models for universal object detection and tracking, moving beyond specific training categories. Integral to our approach is a multi-layered tracker integrated with the foundation detector, ensuring continuous target visibility, even when faced with motion blur, abrupt light shifts, and occlusions. Complementing this, we introduce a model-free controller tailored for resilient quadrotor visual tracking. Our system operates efficiently on limited hardware, relying solely on an onboard camera and an inertial measurement unit. Through extensive validation in diverse challenging indoor and outdoor environments, we demonstrate our system's effectiveness and adaptability. In conclusion, our research represents a step forward in quadrotor visual tracking, moving from task-specific methods to more versatile and adaptable operations

High-Performance Testbed for Vision-Aided Autonomous Navigation for Quadrotor UAVs in Cluttered Environments

This thesis presents the development of an aerial robotic testbed based on Robot Operating System (ROS). The purpose of this high-performance testbed is to develop a system capable of performing robust navigation tasks using vision tools such as a stereo camera. While ensuring the computation of robot odometery, the system is also capable of sensing the environment using the same stereo camera. Hence, all the navigation tasks are performed using a stereo camera and an inertial measurement unit (IMU) as the main sensor suite. ROS is used as a framework for software integration due to its capabilities to provide efficient communication and sensor interfaces. Moreover, it also allows us to use C++ which is efficient in performance especially on embedded platforms. Combining together ROS and C++ provides the necessary computation efficiency and tools to handle fast, real-time image processing and planning which are the vital parts of navigation and obstacle avoidance on such scale. The main application of this work revolves around proposing a real-time and efficient way to demonstrate vision-based navigation in UAVs. The proposed approach is developed for a quadrotor UAV which is capable of performing defensive maneuvers in case any obstacles are in its way, while constantly moving towards a user-defined final destination. Stereo depth computation adds a third axis to a two dimensional image coordinate frame. This can be referred to as the depth image space or depth image coordinate frame. The idea of planning in this frame of reference is utilized along with certain precomputed action primitives. The formulation of these action primitives leads to a hybrid control law for feasible trajectory generation. Further, a proof of stability of this system is also presented. The proposed approach keeps in view the fact that while performing fast maneuvers and obstacle avoidance simultaneously, many of the standard optimization approaches might not work in real-time on-board due to time and resource limitations. This leads to a need for the development of real-time techniques for vision-based autonomous navigation

Agile load transportation systems using aerial robots

In this dissertation, we address problems that can occur during load transport using aerial robots, i.e., small scale quadrotors. First, detailed models of such transportation system are derived. These models include nonlinear models of a quadrotor, a model of a quadrotor carrying a fixed load and a model of a quadrotor carrying a suspended load. Second, the problem of quadrotor stabilization and trajectory tracking with changes of the center of gravity of the transportation system is addressed. This problem is solved using model reference adaptive control based on output feedback linearization that compensates for dynamical changes in the center of gravity of the quadrotor. The third problem we address is a problem of a swing-free transport of suspended load using quadrotors. Flying with a suspended load can be a very challenging and sometimes hazardous task as the suspended load significantly alters the flight characteristics of the quadrotor. In order to deal with suspended load flight, we present a method based on dynamic programming which is a model based offline method. The second investigated method we use is based on the Nelder-Mead algorithm which is an optimization technique used for nonlinear unconstrained optimization problems. This method is model free and it can be used for offline or online generation of the swing-free trajectories for the suspended load. Besides the swing-free maneuvers with suspended load, load trajectory tracking is another problem we solve in this dissertation. In order to solve this problem we use a Nelder-Mead based algorithm. In addition, we use an online least square policy iteration algorithm. At the end, we propose a high level algorithm for navigation in cluttered environments considering a quadrotor with suspended load. Furthermore, distributed control of multiple quadrotors with suspended load is addressed too. The proposed hierarchical architecture presented in this doctoral dissertation is an important step towards developing the next generation of agile autonomous aerial vehicles. These control algorithms enable quadrotors to display agile maneuvers while reconfiguring in real time whenever a change in the center of gravity occurs. This enables a swing-free load transport or trajectory tracking of the load in urban environments in a decentralized fashion