Unifying Foundation Models with Quadrotor Control for Visual Tracking Beyond Object Categories

Visual control enables quadrotors to adaptively navigate using real-time sensory data, bridging perception with action. Yet, challenges persist, including generalization across scenarios, maintaining reliability, and ensuring real-time responsiveness. This paper introduces a perception framework grounded in foundation models for universal object detection and tracking, moving beyond specific training categories. Integral to our approach is a multi-layered tracker integrated with the foundation detector, ensuring continuous target visibility, even when faced with motion blur, abrupt light shifts, and occlusions. Complementing this, we introduce a model-free controller tailored for resilient quadrotor visual tracking. Our system operates efficiently on limited hardware, relying solely on an onboard camera and an inertial measurement unit. Through extensive validation in diverse challenging indoor and outdoor environments, we demonstrate our system's effectiveness and adaptability. In conclusion, our research represents a step forward in quadrotor visual tracking, moving from task-specific methods to more versatile and adaptable operations

Vision-based Autonomous Tracking of a Non-cooperative Mobile Robot by a Low-cost Quadrotor Vehicle

The goal of this thesis is the detection and tracking of a ground vehicle, in particular a car-like robot, by a quadrotor. The first challenge to address in any pursuit or tracking scenario is the detection and unique identification of the target. From this first challenge, comes the need to precisely localize the target in a coordinate system that is common to the tracking and tracked vehicles. In most real-life scenarios, the tracked vehicle does not directly communicate information such as its position to the tracking one. From this fact, arises a non-cooperative constraint problem. The autonomous tracking aspect of the mission requires, for both the aerial and ground vehicles, robust pose estimation during the mission. The primary and crucial functions to achieve autonomous behaviors are control and navigation. The principal-agent being the quadrotor, this thesis explains in detail the derivation and analysis of the equations of motion that govern its natural behavior along with the control methods that permit to achieve desired performances. The analysis of these equations reveals a naturally unstable system, subject to non-linearities. Therefore, we explored three different control methods capable of guaranteeing stability while mitigating non-linearities. The first two control methods operate in the linear region and consist of the intuitive Proportional Integrate Derivative controller (PID). The second linear control strategy is represented by an optimal controller that is the Linear Quadratic Regulator controller (LQR). The last and final control method is a nonlinear controller designed from the Sliding Mode Control Theory. In addition to the in-depth analysis, we provide assets and limitations of each control method. In order to achieve the tracking mission, we address the detection and localization problems using respectively visual servoing and frame transform techniques. The pose estimation challenge for the aerial robot is cleared up using Kalman Filtering estimation methods that are also explored in depth. The same estimation method is used to mitigate the ground vehicle’s real-time pose estimation and tracking problem. Analysis results are illustrated using Matlab. A simulation and a real implementation using the Robot Operating System are used to support the obtained results

Geometric Tracking Control of a Multi-rotor UAV for Partially Known Trajectories

This paper presents a trajectory-tracking controller for multi-rotor unmanned aerial vehicles (UAVs) in scenarios where only the desired position and heading are known without the higher-order derivatives. The proposed solution modifies the state-of-the-art geometric controller, effectively addressing challenges related to the non-existence of the desired attitude and ensuring positive total thrust input for all time. We tackle the additional challenge of the non-availability of the higher derivatives of the trajectory by introducing novel nonlinear filter structures. We formalize theoretically the effect of these filter structures on the system error dynamics. Subsequently, through a rigorous theoretical analysis, we demonstrate that the proposed controller leads to uniformly ultimately bounded system error dynamics

A review of aerial manipulation of small-scale rotorcraft unmanned robotic systems

Small-scale rotorcraft unmanned robotic systems (SRURSs) are a kind of unmanned rotorcraft with manipulating devices. This review aims to provide an overview on aerial manipulation of SRURSs nowadays and promote relative research in the future. In the past decade, aerial manipulation of SRURSs has attracted the interest of researchers globally. This paper provides a literature review of the last 10 years (2008–2017) on SRURSs, and details achievements and challenges. Firstly, the definition, current state, development, classification, and challenges of SRURSs are introduced. Then, related papers are organized into two topical categories: mechanical structure design, and modeling and control. Following this, research groups involved in SRURS research and their major achievements are summarized and classified in the form of tables. The research groups are introduced in detail from seven parts. Finally, trends and challenges are compiled and presented to serve as a resource for researchers interested in aerial manipulation of SRURSs. The problem, trends, and challenges are described from three aspects. Conclusions of the paper are presented, and the future of SRURSs is discussed to enable further research interests

A survey on fractional order control techniques for unmanned aerial and ground vehicles

In recent years, numerous applications of science and engineering for modeling and control of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) systems based on fractional calculus have been realized. The extra fractional order derivative terms allow to optimizing the performance of the systems. The review presented in this paper focuses on the control problems of the UAVs and UGVs that have been addressed by the fractional order techniques over the last decade

Single chip solution for stabilization control & monocular visual servoing of small-scale quadrotor helicopter

This thesis documents the research undertaken to develop a high-performing design of a small-scale quadrotor (four-rotor) helicopter capable of delivering the speed and robustness required for agile motion while also featuring an autonomous visual servoing capability within the size, weight, and power (SWaP) constraint package. The state of the art research was reviewed, and the areas in the existing design methodologies that can potentially be improved were identified, which included development of a comprehensive dynamics model of quadrotor, design and construction of a performance optimized prototype vehicle, high-performance actuator design, design of a robust attitude stabilization controller, and a single chip solution for autonomous vision based position control. The gaps in the current art of designing each component were addressed individually. The outcomes of the corresponding development activities include a high-fidelity dynamics and control model of the vehicle. The model was developed using multi-body bond graph modeling approach to incorporate the dynamic interactions between the frame body and propulsion system. Using an algorithmic size, payload capacity, and flight endurance optimization approach, a quadrotor prototype was designed and constructed. In order to conform to the optimized geometric and performance parameters, the frame of the prototype was constructed using printed circuit board (PCB) technology and processing power was integrated using a single chip field programmable gate array (FPGA) technology. Furthermore, to actuate the quadrotor at a high update rate while also improving the power efficiency of the actuation system, a ground up FPGA based brushless direct current (BLDC) motor driver was designed using a low-loss commutation scheme and hall effect sensors. A proportional-integral-derivative (PID) technology based closed loop motor speed controller was also implemented in the same FPGA hardware for precise speed control of the motors. In addition, a novel control law was formulated for robust attitude stabilization by adopting a cascaded architecture of active disturbance rejection control (ADRC) technology and PID control technology. Using the same single FPGA chip to drive an on-board downward looking camera, a monocular visual servoing solution was developed to integrate an autonomous position control feature with the quadrotor. Accordingly, a numerically simple relative position estimation technique was implemented in FPGA hardware that relies on a passive landmark/target for 3-D position estimation. The functionality and effectiveness of the synthesized design were evaluated by performance benchmarking experiments conducted on each individual component as well as on the complete system constructed from these components. It was observed that the proposed small-scale quadrotor, even though just 43 cm in diameter, can lift 434 gm of payload while operating for 18 min. Among the ground up designed components, the FPGA based motor driver demonstrated a maximum of 4% improvement in the power consumption and at the same time can handle a command update at a rate of 16 kHz. The cascaded attitude stabilization controller can asymptotically stabilize the vehicle within 426 ms of the command update. Robust control performance under stochastic wind gusts is also observed from the stabilization controller. Finally, the single chip FPGA based monocular visual servoing solution can estimate pose information at the camera rate of 37 fps and accordingly the quadrotor can autonomously climb/descend and/or hover over a passive target

Vision-Based Target Tracking and Autonomous Landing of a Quadrotor on a Ground Vehicle

This paper addresses vision-based tracking and landing of a micro-aerial vehicle (MAV) on a ground vehicle (GV). The camera onboard the MAV is mounted so that the optical axis is aligned with the downward-facing axis of the body-fixed frame. A novel supervised learning vision algorithm is proposed as the method to detect the ground vehicle in the image frame. A feedback linearization technique is developed for the MAV to fly over and track the GV so that visibility with the tracked target is maintained with certain guarantees. The efficacy of the visual detection algorithm, and of the tracking and landing controller is demonstrated in simulations and experiments with static and mobile GV