Learning Matchable Image Transformations for Long-term Metric Visual Localization

Long-term metric self-localization is an essential capability of autonomous mobile robots, but remains challenging for vision-based systems due to appearance changes caused by lighting, weather, or seasonal variations. While experience-based mapping has proven to be an effective technique for bridging the `appearance gap,' the number of experiences required for reliable metric localization over days or months can be very large, and methods for reducing the necessary number of experiences are needed for this approach to scale. Taking inspiration from color constancy theory, we learn a nonlinear RGB-to-grayscale mapping that explicitly maximizes the number of inlier feature matches for images captured under different lighting and weather conditions, and use it as a pre-processing step in a conventional single-experience localization pipeline to improve its robustness to appearance change. We train this mapping by approximating the target non-differentiable localization pipeline with a deep neural network, and find that incorporating a learned low-dimensional context feature can further improve cross-appearance feature matching. Using synthetic and real-world datasets, we demonstrate substantial improvements in localization performance across day-night cycles, enabling continuous metric localization over a 30-hour period using a single mapping experience, and allowing experience-based localization to scale to long deployments with dramatically reduced data requirements.Comment: In IEEE Robotics and Automation Letters (RA-L) and presented at the IEEE International Conference on Robotics and Automation (ICRA'20), Paris, France, May 31-June 4, 202

Fluorescence Excitation by Enhanced Plasmon Upconversion under Continuous Wave Illumination

We demonstrate effective background-free continuous wave nonlinear optical excitation of molecules that are sandwiched between asymmetrically constructed plasmonic gold nanoparticle clusters. We observe that near infrared photons are converted to visible photons through efficient plasmonic second harmonic generation. Our theoretical model and simulations demonstrate that Fano resonances may be responsible for being able to observe nonlinear conversion using a continuous wave light source. We show that nonlinearity enhancement of plasmonic nanostructures via coupled quantum mechanical oscillators such as molecules can be several orders larger as compared to their classical counterparts.Comment: 11 pages, 8 figure

Adversarial Training for Adverse Conditions: Robust Metric Localisation using Appearance Transfer

We present a method of improving visual place recognition and metric localisation under very strong appear- ance change. We learn an invertable generator that can trans- form the conditions of images, e.g. from day to night, summer to winter etc. This image transforming filter is explicitly designed to aid and abet feature-matching using a new loss based on SURF detector and dense descriptor maps. A network is trained to output synthetic images optimised for feature matching given only an input RGB image, and these generated images are used to localize the robot against a previously built map using traditional sparse matching approaches. We benchmark our results using multiple traversals of the Oxford RobotCar Dataset over a year-long period, using one traversal as a map and the other to localise. We show that this method significantly improves place recognition and localisation under changing and adverse conditions, while reducing the number of mapping runs needed to successfully achieve reliable localisation.Comment: Accepted at ICRA201

Evaluation of the usefulness of various simulation technology options for TERPS enhancement

Current approved terminal instrument procedures (TERPS) do not permit the full exploitation of the helicopter's unique flying characteristics. Enhanced TERPS need to be developed for a host of non-standard landing sites and navigation aids. Precision navigation systems such as microwave landing systems (MLS) and the Global Positioning System (GPS) open the possibility of curved paths, steep glide slopes, and decelerating helicopter approaches. This study evaluated the feasibility, benefits, and liabilities of using helicopter cockpit simulators in place of flight testing to develop enhanced TERPS criteria for non-standard flight profiles and navigation equipment. Near-term (2 to 5 year) requirements for conducting simulator studies to verify that they produce suitable data comparable to that obtained from previous flight tests are discussed. The long-term (5 to 10 year) research and development requirements to provide necessary modeling for continued simulator-based testing to develop enhanced TERPS criteria are also outlined

Multi-Modal Enhancement Techniques for Visibility Improvement of Digital Images

Image enhancement techniques for visibility improvement of 8-bit color digital images based on spatial domain, wavelet transform domain, and multiple image fusion approaches are investigated in this dissertation research. In the category of spatial domain approach, two enhancement algorithms are developed to deal with problems associated with images captured from scenes with high dynamic ranges. The first technique is based on an illuminance-reflectance (I-R) model of the scene irradiance. The dynamic range compression of the input image is achieved by a nonlinear transformation of the estimated illuminance based on a windowed inverse sigmoid transfer function. A single-scale neighborhood dependent contrast enhancement process is proposed to enhance the high frequency components of the illuminance, which compensates for the contrast degradation of the mid-tone frequency components caused by dynamic range compression. The intensity image obtained by integrating the enhanced illuminance and the extracted reflectance is then converted to a RGB color image through linear color restoration utilizing the color components of the original image. The second technique, named AINDANE, is a two step approach comprised of adaptive luminance enhancement and adaptive contrast enhancement. An image dependent nonlinear transfer function is designed for dynamic range compression and a multiscale image dependent neighborhood approach is developed for contrast enhancement. Real time processing of video streams is realized with the I-R model based technique due to its high speed processing capability while AINDANE produces higher quality enhanced images due to its multi-scale contrast enhancement property. Both the algorithms exhibit balanced luminance, contrast enhancement, higher robustness, and better color consistency when compared with conventional techniques. In the transform domain approach, wavelet transform based image denoising and contrast enhancement algorithms are developed. The denoising is treated as a maximum a posteriori (MAP) estimator problem; a Bivariate probability density function model is introduced to explore the interlevel dependency among the wavelet coefficients. In addition, an approximate solution to the MAP estimation problem is proposed to avoid the use of complex iterative computations to find a numerical solution. This relatively low complexity image denoising algorithm implemented with dual-tree complex wavelet transform (DT-CWT) produces high quality denoised images