Grasping, Perching, And Visual Servoing For Micro Aerial Vehicles

Micro Aerial Vehicles (MAVs) have seen a dramatic growth in the consumer market because of their ability to provide new vantage points for aerial photography and videography. However, there is little consideration for physical interaction with the environment surrounding them. Onboard manipulators are absent, and onboard perception, if existent, is used to avoid obstacles and maintain a minimum distance from them. There are many applications, however, which would benefit greatly from aerial manipulation or flight in close proximity to structures. This work is focused on facilitating these types of close interactions between quadrotors and surrounding objects. We first explore high-speed grasping, enabling a quadrotor to quickly grasp an object while moving at a high relative velocity. Next, we discuss planning and control strategies, empowering a quadrotor to perch on vertical surfaces using a downward-facing gripper. Then, we demonstrate that such interactions can be achieved using only onboard sensors by incorporating vision-based control and vision-based planning. In particular, we show how a quadrotor can use a single camera and an Inertial Measurement Unit (IMU) to perch on a cylinder. Finally, we generalize our approach to consider objects in motion, and we present relative pose estimation and planning, enabling tracking of a moving sphere using only an onboard camera and IMU

A review of aerial manipulation of small-scale rotorcraft unmanned robotic systems

Small-scale rotorcraft unmanned robotic systems (SRURSs) are a kind of unmanned rotorcraft with manipulating devices. This review aims to provide an overview on aerial manipulation of SRURSs nowadays and promote relative research in the future. In the past decade, aerial manipulation of SRURSs has attracted the interest of researchers globally. This paper provides a literature review of the last 10 years (2008–2017) on SRURSs, and details achievements and challenges. Firstly, the definition, current state, development, classification, and challenges of SRURSs are introduced. Then, related papers are organized into two topical categories: mechanical structure design, and modeling and control. Following this, research groups involved in SRURS research and their major achievements are summarized and classified in the form of tables. The research groups are introduced in detail from seven parts. Finally, trends and challenges are compiled and presented to serve as a resource for researchers interested in aerial manipulation of SRURSs. The problem, trends, and challenges are described from three aspects. Conclusions of the paper are presented, and the future of SRURSs is discussed to enable further research interests

Visual Servoing of Quadrotors for Perching by Hanging From Cylindrical Objects

This paper addresses vision-based localization and servoing for quadrotors to enable autonomous perching by hanging from cylindrical structures using only a monocular camera. We focus on the problems of relative pose estimation, control, and trajectory planning for maneuvering a robot relative to cylinders with unknown orientations. We first develop a geometric model that describes the pose of the robot relative to a cylinder. Then, we derive the dynamics of the system, expressed in terms of the image features. Based on the dynamics, we present a controller which guarantees asymptotic convergence to the desired image space coordinates. Finally, we develop an effective method to plan dynamically-feasible trajectories in the image space, and we provide experimental results to demonstrate the proposed method under different operating conditions such as hovering, trajectory tracking, and perching

Effective Target Aware Visual Navigation for UAVs

In this paper we propose an effective vision-based navigation method that allows a multirotor vehicle to simultaneously reach a desired goal pose in the environment while constantly facing a target object or landmark. Standard techniques such as Position-Based Visual Servoing (PBVS) and Image-Based Visual Servoing (IBVS) in some cases (e.g., while the multirotor is performing fast maneuvers) do not allow to constantly maintain the line of sight with a target of interest. Instead, we compute the optimal trajectory by solving a non-linear optimization problem that minimizes the target re-projection error while meeting the UAV's dynamic constraints. The desired trajectory is then tracked by means of a real-time Non-linear Model Predictive Controller (NMPC): this implicitly allows the multirotor to satisfy both the required constraints. We successfully evaluate the proposed approach in many real and simulated experiments, making an exhaustive comparison with a standard approach.Comment: Conference paper at "European Conference on Mobile Robotics" (ECMR) 201

Towards Instance Segmentation-Based Litter Collection with Multi-Rotor Aerial Vehicle

This paper presents a novel aerial robotics application of instance segmentation-based floating litter collection with a multi-rotor aerial vehicle (MRAV). In the scope of the paper, we present a review of the available datasets for litter detection and segmentation. The reviewed datasets are used to train a Mask-RCNN neural network for instance segmentation. The neural network is off-board deployed on an edge computing device and used for litter position estimation. Based on the estimated litter position, we plan a path based on a quadratic Bezier curve for the litter pickup. We compare different trajectory generation methods for the object pickup. The system is verified in a laboratory environment. Eventually, we present practical considerations and improvements necessary to enable autonomous litter collection with MRAV

Nonlinear adaptive control of an aerial manipulation system

This paper presents the nonlinear adaptive control of a quadrotor endowed with a 2 degrees of freedom (DOF) manipulator. By considering the quadrotor and the robot arm as a combined system, complete modeling of the aerial manipulation system (AMS) has been presented using the Euler-Lagrange method. A hierarchical nonlinear control scheme which consists of outer and inner control loops has been utilized. Model Reference Adaptive Controller (MRAC) is designed for the outer loop where the required command signals are generated to force the quadrotor to move on a reference trajectory in the presence of uncertainties and reaction forces coming from the manipulator. For the inner loop, the attitude dynamics of the quadrotor and the dynamics of the 2-DOF robotic arm are considered as a fully actuated 5-DOF unified part of the AMS. Nonlinear adaptive control has been utilized for the low-level controller where the changes in inertias and the masses have been tackled along with the reaction forces acting on the attitude part of the AMS. The proposed technique has been validated through simulations in two different scenarios

A survey of single and multi-UAV aerial manipulation

Aerial manipulation has direct application prospects in environment, construction, forestry, agriculture, search, and rescue. It can be used to pick and place objects and hence can be used for transportation of goods. Aerial manipulation can be used to perform operations in environments inaccessible or unsafe for human workers. This paper is a survey of recent research in aerial manipulation. The aerial manipulation research has diverse aspects, which include the designing of aerial manipulation platforms, manipulators, grippers, the control of aerial platform and manipulators, the interaction of aerial manipulator with the environment, through forces and torque. In particular, the review paper presents the survey of the airborne platforms that can be used for aerial manipulation including the new aerial platforms with aerial manipulation capability. We also classified the aerial grippers and aerial manipulators based on their designs and characteristics. The recent contributions regarding the control of the aerial manipulator platform is also discussed. The environment interaction of aerial manipulators is also surveyed which includes, different strategies used for end-effectors interaction with the environment, application of force, application of torque and visual servoing. A recent and growing interest of researchers about the multi-UAV collaborative aerial manipulation was also noticed and hence different strategies for collaborative aerial manipulation are also surveyed, discussed and critically analyzed. Some key challenges regarding outdoor aerial manipulation and energy constraints in aerial manipulation are also discussed

Unifying Foundation Models with Quadrotor Control for Visual Tracking Beyond Object Categories

Visual control enables quadrotors to adaptively navigate using real-time sensory data, bridging perception with action. Yet, challenges persist, including generalization across scenarios, maintaining reliability, and ensuring real-time responsiveness. This paper introduces a perception framework grounded in foundation models for universal object detection and tracking, moving beyond specific training categories. Integral to our approach is a multi-layered tracker integrated with the foundation detector, ensuring continuous target visibility, even when faced with motion blur, abrupt light shifts, and occlusions. Complementing this, we introduce a model-free controller tailored for resilient quadrotor visual tracking. Our system operates efficiently on limited hardware, relying solely on an onboard camera and an inertial measurement unit. Through extensive validation in diverse challenging indoor and outdoor environments, we demonstrate our system's effectiveness and adaptability. In conclusion, our research represents a step forward in quadrotor visual tracking, moving from task-specific methods to more versatile and adaptable operations