Vision and Learning for Deliberative Monocular Cluttered Flight

Cameras provide a rich source of information while being passive, cheap and lightweight for small and medium Unmanned Aerial Vehicles (UAVs). In this work we present the first implementation of receding horizon control, which is widely used in ground vehicles, with monocular vision as the only sensing mode for autonomous UAV flight in dense clutter. We make it feasible on UAVs via a number of contributions: novel coupling of perception and control via relevant and diverse, multiple interpretations of the scene around the robot, leveraging recent advances in machine learning to showcase anytime budgeted cost-sensitive feature selection, and fast non-linear regression for monocular depth prediction. We empirically demonstrate the efficacy of our novel pipeline via real world experiments of more than 2 kms through dense trees with a quadrotor built from off-the-shelf parts. Moreover our pipeline is designed to combine information from other modalities like stereo and lidar as well if available

Input uncertainty sensitivity enhanced non-singleton fuzzy logic controllers for long-term navigation of quadrotor UAVs

Input uncertainty, e.g., noise on the on-board camera and inertial measurement unit, in vision-based control of unmanned aerial vehicles (UAVs) is an inevitable problem. In order to handle input uncertainties as well as further analyze the interaction between the input and the antecedent fuzzy sets (FSs) of non-singleton fuzzy logic controllers (NSFLCs), an input uncertainty sensitivity enhanced NSFLC has been developed in robot operating system (ROS) using the C++ programming language. Based on recent advances in non-singleton inference, the centroid of the intersection of the input and antecedent FSs (Cen-NSFLC) is utilized to calculate the firing strength of each rule instead of the maximum of the intersection used in traditional NSFLC (Tra-NSFLC). An 8-shaped trajectory, consisting of straight and curved lines, is used for the real-time validation of the proposed controllers for a trajectory following problem. An accurate monocular keyframe-based visual-inertial simultaneous localization and mapping (SLAM) approach is used to estimate the position of the quadrotor UAV in GPS denied unknown environments. The performance of the Cen-NSFLC is compared with a conventional proportional integral derivative (PID) controller, a singleton FLC (SFLC) and a Tra-NSFLC. All controllers are evaluated for different flight speeds, thus introducing different levels of uncertainty into the control problem. Visual-inertial SLAM-based real time quadrotor UAV flight tests demonstrate that not only does the Cen-NSFLC achieve the best control performance among the four controllers, but it also shows better control performance when compared to their singleton counterparts. Considering the bias in the use of model based controllers, e.g. PID, for the control of UAVs, this paper advocates an alternative method, namely Cen-NSFLCs, in uncertain working environments

AgriColMap: Aerial-Ground Collaborative 3D Mapping for Precision Farming

The combination of aerial survey capabilities of Unmanned Aerial Vehicles with targeted intervention abilities of agricultural Unmanned Ground Vehicles can significantly improve the effectiveness of robotic systems applied to precision agriculture. In this context, building and updating a common map of the field is an essential but challenging task. The maps built using robots of different types show differences in size, resolution and scale, the associated geolocation data may be inaccurate and biased, while the repetitiveness of both visual appearance and geometric structures found within agricultural contexts render classical map merging techniques ineffective. In this paper we propose AgriColMap, a novel map registration pipeline that leverages a grid-based multimodal environment representation which includes a vegetation index map and a Digital Surface Model. We cast the data association problem between maps built from UAVs and UGVs as a multimodal, large displacement dense optical flow estimation. The dominant, coherent flows, selected using a voting scheme, are used as point-to-point correspondences to infer a preliminary non-rigid alignment between the maps. A final refinement is then performed, by exploiting only meaningful parts of the registered maps. We evaluate our system using real world data for 3 fields with different crop species. The results show that our method outperforms several state of the art map registration and matching techniques by a large margin, and has a higher tolerance to large initial misalignments. We release an implementation of the proposed approach along with the acquired datasets with this paper.Comment: Published in IEEE Robotics and Automation Letters, 201

Vision-based localization methods under GPS-denied conditions

This paper reviews vision-based localization methods in GPS-denied environments and classifies the mainstream methods into Relative Vision Localization (RVL) and Absolute Vision Localization (AVL). For RVL, we discuss the broad application of optical flow in feature extraction-based Visual Odometry (VO) solutions and introduce advanced optical flow estimation methods. For AVL, we review recent advances in Visual Simultaneous Localization and Mapping (VSLAM) techniques, from optimization-based methods to Extended Kalman Filter (EKF) based methods. We also introduce the application of offline map registration and lane vision detection schemes to achieve Absolute Visual Localization. This paper compares the performance and applications of mainstream methods for visual localization and provides suggestions for future studies.Comment: 32 pages, 15 figure