Procedures for condition mapping using 360° images

The identification of deterioration mechanisms and their monitoring over time is an essential phase for conservation. This work aimed at developing a novel approach for deterioration mapping and monitoring based on 360° images, which allows for simple and rapid data collection. The opportunity to capture the whole scene around a 360° camera reduces the number of images needed in a condition mapping project, resulting in a powerful solution to document small and narrow spaces. The paper will describe the implemented workflow for deterioration mapping based on 360° images, which highlights pathologies on surfaces and quantitatively measures their extension. Such a result will be available as standard outputs as well as an innovative virtual environment for immersive visualization. The case of multi-temporal data acquisition will be considered and discussed as well. Multiple 360° images acquired at different epochs from slightly different points are co-registered to obtain pixel-to-pixel correspondence, providing a solution to quantify and track deterioration effects

Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors

Visual localization is an attractive problem that estimates the camera localization from database images based on the query image. It is a crucial task for various applications, such as autonomous vehicles, assistive navigation and augmented reality. The challenging issues of the task lie in various appearance variations between query and database images, including illumination variations, dynamic object variations and viewpoint variations. In order to tackle those challenges, Panoramic Annular Localizer into which panoramic annular lens and robust deep image descriptors are incorporated is proposed in this paper. The panoramic annular images captured by the single camera are processed and fed into the NetVLAD network to form the active deep descriptor, and sequential matching is utilized to generate the localization result. The experiments carried on the public datasets and in the field illustrate the validation of the proposed system.Comment: Accepted by ITSC 201

Ground-based hyperspectral analysis of the urban nightscape

Airborne hyperspectral cameras provide the basic information to estimate the energy wasted skywards by outdoor lighting systems, as well as to locate and identify their sources. However, a complete characterization of the urban light pollution levels also requires evaluating these effects from the city dwellers standpoint, e.g. the energy waste associated to the excessive illuminance on walls and pavements, light trespass, or the luminance distributions causing potential glare, to mention but a few. On the other hand, the spectral irradiance at the entrance of the human eye is the primary input to evaluate the possible health effects associated with the exposure to artificial light at night, according to the more recent models available in the literature. In this work we demonstrate the possibility of using a hyperspectral imager (routinely used in airborne campaigns) to measure the ground-level spectral radiance of the urban nightscape and to retrieve several magnitudes of interest for light pollution studies. We also present the preliminary results from a field campaign carried out in the downtown of Barcelona.Postprint (author's final draft

Did You Miss the Sign? A False Negative Alarm System for Traffic Sign Detectors

Object detection is an integral part of an autonomous vehicle for its safety-critical and navigational purposes. Traffic signs as objects play a vital role in guiding such systems. However, if the vehicle fails to locate any critical sign, it might make a catastrophic failure. In this paper, we propose an approach to identify traffic signs that have been mistakenly discarded by the object detector. The proposed method raises an alarm when it discovers a failure by the object detector to detect a traffic sign. This approach can be useful to evaluate the performance of the detector during the deployment phase. We trained a single shot multi-box object detector to detect traffic signs and used its internal features to train a separate false negative detector (FND). During deployment, FND decides whether the traffic sign detector (TSD) has missed a sign or not. We are using precision and recall to measure the accuracy of FND in two different datasets. For 80% recall, FND has achieved 89.9% precision in Belgium Traffic Sign Detection dataset and 90.8% precision in German Traffic Sign Recognition Benchmark dataset respectively. To the best of our knowledge, our method is the first to tackle this critical aspect of false negative detection in robotic vision. Such a fail-safe mechanism for object detection can improve the engagement of robotic vision systems in our daily life.Comment: Submitted to the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019

Flow velocity mapping using contrast enhanced high-frame-rate plane wave ultrasound and image tracking: methods and initial in vitro and in vivo evaluation

Ultrasound imaging is the most widely used method for visualising and quantifying blood flow in medical practice, but existing techniques have various limitations in terms of imaging sensitivity, field of view, flow angle dependence, and imaging depth. In this study, we developed an ultrasound imaging velocimetry approach capable of visualising and quantifying dynamic flow, by combining high-frame-rate plane wave ultrasound imaging, microbubble contrast agents, pulse inversion contrast imaging and speckle image tracking algorithms. The system was initially evaluated in vitro on both straight and carotid-mimicking vessels with steady and pulsatile flows and in vivo in the rabbit aorta. Colour and spectral Doppler measurements were also made. Initial flow mapping results were compared with theoretical prediction and reference Doppler measurements and indicate the potential of the new system as a highly sensitive, accurate, angle-independent and full field-of-view velocity mapping tool capable of tracking and quantifying fast and dynamic flows

Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices.

Computational theories propose that attention modulates the topographical landscape of spatial 'priority' maps in regions of the visual cortex so that the location of an important object is associated with higher activation levels. Although studies of single-unit recordings have demonstrated attention-related increases in the gain of neural responses and changes in the size of spatial receptive fields, the net effect of these modulations on the topography of region-level priority maps has not been investigated. Here we used functional magnetic resonance imaging and a multivariate encoding model to reconstruct spatial representations of attended and ignored stimuli using activation patterns across entire visual areas. These reconstructed spatial representations reveal the influence of attention on the amplitude and size of stimulus representations within putative priority maps across the visual hierarchy. Our results suggest that attention increases the amplitude of stimulus representations in these spatial maps, particularly in higher visual areas, but does not substantively change their size

Linking the anomaly initialization approach to the mapping paradigm: a proof-of-concept study

Seasonal-to-decadal predictions are initialized using observations of the present climatic state in full field initialization (FFI). Such model integrations undergo a drift toward the model attractor due to model deficiencies that incur a bias in the model. The anomaly initialization (AI) approach reduces the drift by adding an estimate of the bias onto the observations at the expense of a larger initial error. In this study FFI is associated with the fidelity paradigm, and AI is associated with an instance of the mapping paradigm, in which the initial conditions are mapped onto the imperfect model attractor by adding a fixed error term; the mapped state on the model attractor should correspond to the nature state. Two diagnosis tools assess how well AI conforms to its own paradigm under various circumstances of model error: the degree of approximation of the model attractor is measured by calculating the overlap of the AI initial conditions PDF with the model PDF; and the sensitivity to random error in the initial conditions reveals how well the selected initial conditions on the model attractor correspond to the nature states. As a useful reference, the initial conditions of FFI are subjected to the same analysis. Conducting hindcast experiments using a hierarchy of low-order coupled climate models, it is shown that the initial conditions generated using AI approximate the model attractor only under certain conditions: differences in higher-than-first-order moments between the model and nature PDFs must be negligible. Where such conditions fail, FFI is likely to perform better

MicroED data collection and processing.

MicroED, a method at the intersection of X-ray crystallography and electron cryo-microscopy, has rapidly progressed by exploiting advances in both fields and has already been successfully employed to determine the atomic structures of several proteins from sub-micron-sized, three-dimensional crystals. A major limiting factor in X-ray crystallography is the requirement for large and well ordered crystals. By permitting electron diffraction patterns to be collected from much smaller crystals, or even single well ordered domains of large crystals composed of several small mosaic blocks, MicroED has the potential to overcome the limiting size requirement and enable structural studies on difficult-to-crystallize samples. This communication details the steps for sample preparation, data collection and reduction necessary to obtain refined, high-resolution, three-dimensional models by MicroED, and presents some of its unique challenges