Featureless visual processing for SLAM in changing outdoor environments

Vision-based SLAM is mostly a solved problem providing clear, sharp images can be obtained. However, in outdoor environments a number of factors such as rough terrain, high speeds and hardware limitations can result in these conditions not being met. High speed transit on rough terrain can lead to image blur and under/over exposure, problems that cannot easily be dealt with using low cost hardware. Furthermore, recently there has been a growth in interest in lifelong autonomy for robots, which brings with it the challenge in outdoor environments of dealing with a moving sun and lack of constant artificial lighting. In this paper, we present a lightweight approach to visual localization and visual odometry that addresses the challenges posed by perceptual change and low cost cameras. The approach combines low resolution imagery with the SLAM algorithm, RatSLAM. We test the system using a cheap consumer camera mounted on a small vehicle in a mixed urban and vegetated environment, at times ranging from dawn to dusk and in conditions ranging from sunny weather to rain. We first show that the system is able to provide reliable mapping and recall over the course of the day and incrementally incorporate new visual scenes from different times into an existing map. We then restrict the system to only learning visual scenes at one time of day, and show that the system is still able to localize and map at other times of day. The results demonstrate the viability of the approach in situations where image quality is poor and environmental or hardware factors preclude the use of visual features

Use of Remote Imagery and Object-based Image Methods to Count Plants in an Open-field Container Nursery

In general, the nursery industry lacks an automated inventory control system. Object-based image analysis (OBIA) software and aerial images could be used to count plants in nurseries. The objectives of this research were: 1) to evaluate the effect of an unmanned aerial vehicle (UAV) flight altitude and plant canopy separation of container-grown plants on count accuracy using aerial images and 2) to evaluate the effect of plant canopy shape, presence of flowers, and plant status (living and dead) on counting accuracy of container-grown plants using remote sensing images. Images were analyzed using Feature Analyst® (FA) and an algorithm trained using MATLAB®. Total count error, false positives and unidentified plants were recorded from output images using FA; only total count error was reported for the MATLAB algorithm. For objective 1, images were taken at 6, 12 and 22 m above the ground using a UAV. Plants were placed on black fabric and gravel, and spaced as follows: 5 cm between canopy edges, canopy edges touching, and 5 cm of canopy edge overlap. In general, when both methods were considered, total count error was smaller [ranging from -5 (undercount) to 4 (over count)] when plants were fully separated with the exception of images taken at 22 m. FA showed a smaller total count error (-2) than MATLAB (-5) when plants were placed on black fabric than those placed on gravel. For objective 2, the plan was to continue using the UAV, however, due to the unexpected disruption of the GPS-based navigation by heightened solar flare activity in 2013, a boom lift that could provide images on a more reliable basis was used. When images obtained using a boom lift were analyzed using FA there was no difference between variables measured when an algorithm trained with an image displaying regular or irregular plant canopy shape was applied to images displaying both plant canopy shapes even though the canopy shape of `Sea Green\u27 juniper is less compact than `Plumosa Compacta\u27. There was a significant difference in all variables measured between images of flowering and non-flowering plants, when non-flowering `samples\u27 were used to train the counting algorithm and analyzed with FA. No dead plants were counted as living and vice versa, when data were analyzed using FA. When the algorithm trained in MATLAB was applied, there was no significant difference in total count errors when plant canopy shape and presence of flowers were evaluated. Based on the combined results from these separate experiments, FA and MATLAB algorithms appear to be fairly robust when used to count container-grown plants from images taken at the heights specified

Vision based strategies for implementing Sense and Avoid capabilities onboard Unmanned Aerial Systems

Current research activities are worked out to develop fully autonomous unmanned platform systems, provided with Sense and Avoid technologies in order to achieve the access to the National Airspace System (NAS), flying with manned airplanes. The TECVOl project is set in this framework, aiming at developing an autonomous prototypal Unmanned Aerial Vehicle which performs Detect Sense and Avoid functionalities, by means of an integrated sensors package, composed by a pulsed radar and four electro-optical cameras, two visible and two Infra-Red. This project is carried out by the Italian Aerospace Research Center in collaboration with the Department of Aerospace Engineering of the University of Naples “Federico II”, which has been involved in the developing of the Obstacle Detection and IDentification system. Thus, this thesis concerns the image processing technique customized for the Sense and Avoid applications in the TECVOL project, where the EO system has an auxiliary role to radar, which is the main sensor. In particular, the panchromatic camera performs the aiding function of object detection, in order to increase accuracy and data rate performance of radar system. Therefore, the thesis describes the implemented steps to evaluate the most suitable panchromatic camera image processing technique for our applications, the test strategies adopted to study its performance and the analysis conducted to optimize it in terms of false alarms, missed detections and detection range. Finally, results from the tests will be explained, and they will demonstrate that the Electro-Optical sensor is beneficial to the overall Detect Sense and Avoid system; in fact it is able to improve upon it, in terms of object detection and tracking performance

Vision Based Localization under Dynamic Illumination

Localization in dynamically illuminated environments is often difficult due to static objects casting dynamic shadows. Feature extraction algorithms may detect both the objects and their shadows, producing conflict in localization algorithms. This work examines a colour model that separates brightness from chromaticity and applies it to eliminate features caused by dynamic illumination. The colour model is applied in two novel ways. Firstly, the chromaticity distortion of a single feature is used to determine if the feature is the result of illumination alone i.e. a shadow. Secondly, the chromaticity distortion of features matched between images is examined to determine if the monochrome based algorithm has matched them correctly. These two applications are put through a variety of tests in simulated then real world environments to assess their effectiveness in dynamically illuminated scenarios. The results demonstrate a significant reduction in the number of feature mismatches between images with dynamic light sources. The evaluation of the techniques individually in a Simultaneous Localization and Mapping (SLAM) task show substantial improvements in accuracy, with the combination of the two techniques producing a localization result that is highly robust to the environmental lighting

Nighttime Smartphone Reflective Flare Removal Using Optical Center Symmetry Prior

Reflective flare is a phenomenon that occurs when light reflects inside lenses, causing bright spots or a "ghosting effect" in photos, which can impact their quality. Eliminating reflective flare is highly desirable but challenging. Many existing methods rely on manually designed features to detect these bright spots, but they often fail to identify reflective flares created by various types of light and may even mistakenly remove the light sources in scenarios with multiple light sources. To address these challenges, we propose an optical center symmetry prior, which suggests that the reflective flare and light source are always symmetrical around the lens's optical center. This prior helps to locate the reflective flare's proposal region more accurately and can be applied to most smartphone cameras. Building on this prior, we create the first reflective flare removal dataset called BracketFlare, which contains diverse and realistic reflective flare patterns. We use continuous bracketing to capture the reflective flare pattern in the underexposed image and combine it with a normally exposed image to synthesize a pair of flare-corrupted and flare-free images. With the dataset, neural networks can be trained to remove the reflective flares effectively. Extensive experiments demonstrate the effectiveness of our method on both synthetic and real-world datasets.Comment: CVPR2023 (Highlight

Aircraft Attitude Estimation Using Panoramic Images

This thesis investigates the problem of reliably estimating attitude from panoramic imagery in cluttered environments. Accurate attitude is an essential input to the stabilisation systems of autonomous aerial vehicles. A new camera system which combines a CCD camera, UltraViolet (UV) filters and a panoramic mirror-lens is designed. Drawing on biological inspiration from the Ocelli organ possessed by certain insects, UV filtered images are used to enhance the contrast between the sky and ground and mitigate the effect of the sun. A novel method for real–time horizon-based attitude estimation using panoramic image that is capable of estimating an aircraft pitch and roll at a low altitude in the presence of sun, clouds and occluding features such as tree, building, is developed. Also, a new method for panoramic sky/ground thresholding, consisting of a horizon– and a sun–tracking system which works effectively even when the horizon line is difficult to detect by normal thresholding methods due to flares and other effects from the presence of the sun in the image, is proposed. An algorithm for estimating the attitude from three–dimensional mapping of the horizon projected onto a 3D plane is developed. The use of optic flow to determine pitch and roll rates is investigated using the panoramic image and image interpolation algorithm (I2A). Two methods which employ sensor fusion techniques, Extended Kalman Filter (EKF) and Artificial Neural Networks (ANNs), are used to fuse unfiltered measurements from inertial sensors and the vision system. The EKF estimates gyroscope biases and also the attitude. The ANN fuses the optic flow and horizon–based attitude to provide smooth attitude estimations. The results obtained from different parts of the research are tested and validated through simulations and real flight tests

Vision and Learning for Deliberative Monocular Cluttered Flight

Cameras provide a rich source of information while being passive, cheap and lightweight for small and medium Unmanned Aerial Vehicles (UAVs). In this work we present the first implementation of receding horizon control, which is widely used in ground vehicles, with monocular vision as the only sensing mode for autonomous UAV flight in dense clutter. We make it feasible on UAVs via a number of contributions: novel coupling of perception and control via relevant and diverse, multiple interpretations of the scene around the robot, leveraging recent advances in machine learning to showcase anytime budgeted cost-sensitive feature selection, and fast non-linear regression for monocular depth prediction. We empirically demonstrate the efficacy of our novel pipeline via real world experiments of more than 2 kms through dense trees with a quadrotor built from off-the-shelf parts. Moreover our pipeline is designed to combine information from other modalities like stereo and lidar as well if available

SynDrone -- Multi-modal UAV Dataset for Urban Scenarios

The development of computer vision algorithms for Unmanned Aerial Vehicles (UAVs) imagery heavily relies on the availability of annotated high-resolution aerial data. However, the scarcity of large-scale real datasets with pixel-level annotations poses a significant challenge to researchers as the limited number of images in existing datasets hinders the effectiveness of deep learning models that require a large amount of training data. In this paper, we propose a multimodal synthetic dataset containing both images and 3D data taken at multiple flying heights to address these limitations. In addition to object-level annotations, the provided data also include pixel-level labeling in 28 classes, enabling exploration of the potential advantages in tasks like semantic segmentation. In total, our dataset contains 72k labeled samples that allow for effective training of deep architectures showing promising results in synthetic-to-real adaptation. The dataset will be made publicly available to support the development of novel computer vision methods targeting UAV applications.Comment: Accepted at ICCV Workshops, downloadable dataset with CC-BY license, 8 pages, 4 figures, 8 table