Video Registration in Egocentric Vision under Day and Night Illumination Changes

With the spread of wearable devices and head mounted cameras, a wide range of application requiring precise user localization is now possible. In this paper we propose to treat the problem of obtaining the user position with respect to a known environment as a video registration problem. Video registration, i.e. the task of aligning an input video sequence to a pre-built 3D model, relies on a matching process of local keypoints extracted on the query sequence to a 3D point cloud. The overall registration performance is strictly tied to the actual quality of this 2D-3D matching, and can degrade if environmental conditions such as steep changes in lighting like the ones between day and night occur. To effectively register an egocentric video sequence under these conditions, we propose to tackle the source of the problem: the matching process. To overcome the shortcomings of standard matching techniques, we introduce a novel embedding space that allows us to obtain robust matches by jointly taking into account local descriptors, their spatial arrangement and their temporal robustness. The proposal is evaluated using unconstrained egocentric video sequences both in terms of matching quality and resulting registration performance using different 3D models of historical landmarks. The results show that the proposed method can outperform state of the art registration algorithms, in particular when dealing with the challenges of night and day sequences

Wittgenstein's Thought Experiments and Relativity Theory

In this paper, I discuss the similarity between Wittgenstein’s use of thought experiments and Relativity Theory. I begin with introducing Wittgenstein’s idea of “thought experiments” and a tentative classification of different kinds of thought experiments in Wittgenstein’s work. Then, after presenting a short recap of some remarks on the analogy between Wittgenstein’s point of view and Einstein’s, I suggest three analogies between the status of Wittgenstein’s mental experiments and Relativity theory: the topics of time dilation, the search for invariants, and the role of measuring tools in Special Relativity. This last point will help to better define Wittgenstein’s idea of description as the core of his philosophical enterprise

The generalization of the R-transform for invariant pattern representation

International audienceThe beneficial properties of the Radon transform make it an useful intermediate representation for the extraction of invariant features from pattern images for the purpose of indexing/matching. This paper revisits the problem of Radon image utilization with a generic view on a popular Radon transform-based transform and pattern descriptor, the R-transform and R-signature, bringing in a class of transforms and descriptors spatially describing patterns at all directions and at different levels, while maintaining the beneficial properties of the conventional R-transform and R-signature. The domain of this class, which is delimited due to the existence of singularities and the effect of sampling/quantization and additive noise, is examined. Moreover, the ability of the generic R-transform to encode the dominant directions of pattern is also discussed, adding to the robustness to additive noise of the generic R-signature. The stability of dominant direction encoding by the generic R-transform and the superiority of the generic R-signature over existing invariant pattern descriptors on grayscale and binary noisy datasets have been confirmed by experiments

MinMax Radon Barcodes for Medical Image Retrieval

Content-based medical image retrieval can support diagnostic decisions by clinical experts. Examining similar images may provide clues to the expert to remove uncertainties in his/her final diagnosis. Beyond conventional feature descriptors, binary features in different ways have been recently proposed to encode the image content. A recent proposal is "Radon barcodes" that employ binarized Radon projections to tag/annotate medical images with content-based binary vectors, called barcodes. In this paper, MinMax Radon barcodes are introduced which are superior to "local thresholding" scheme suggested in the literature. Using IRMA dataset with 14,410 x-ray images from 193 different classes, the advantage of using MinMax Radon barcodes over \emph{thresholded} Radon barcodes are demonstrated. The retrieval error for direct search drops by more than 15\%. As well, SURF, as a well-established non-binary approach, and BRISK, as a recent binary method are examined to compare their results with MinMax Radon barcodes when retrieving images from IRMA dataset. The results demonstrate that MinMax Radon barcodes are faster and more accurate when applied on IRMA images.Comment: To appear in proceedings of the 12th International Symposium on Visual Computing, December 12-14, 2016, Las Vegas, Nevada, US

Reconocimiento de Patrones en Imágenes Digitales a Color usando el Descriptor RFM

En este trabajo se propone un sistema digital de reconocimiento de patrones para imágenes a color basado en firmas 1D invariantes a traslación, escala y rotación (RST). La invariancia a rotación se obtiene por medio de la transformada de Radon. Para la invariancia a escala se utiliza la transformada de Fourier-Mellin normalizada. La invariancia a traslación se consigue a través del espectro de amplitud de la transformada de Fourier de la imagen. Al trabajar en el espacio de color RGB la imagen se separa en tres imágenes monocromáticas, las cuales corresponden al canal rojo (R), verde (G) y azul (B). Al aplicar las transformadas integrales a cada una de las imágenes monocromáticas se generan tres imágenes, denominadas Radon-Fourier-Mellin (RFM) que son invariantes a traslación, escala y rotación, por lo que para una imagen a color se generan tres imágenes Radon-Fourier-Mellin. Para cada una de las imágenes Radon-Fourier-Mellin (señal 2D) se construye una firma 1D invariante a traslación, escala y rotación, de la cual se calcula su potencia. Como para la imagen se tienen tres firmas 1D (una para cada canal), entonces se tienen tres potencias. Las tres potencias son los atributos que se le asignan a la imagen a color para su clasificación y se utilizan para generar un espacio de clasificación 3D que tiene un nivel de confianza de al menos el 95.4 %. Para mostrar la eficiencia del sistema se emplea una base de datos de 18 imágenes de referencia a color que contienen mariposas, dicho conjunto fue seleccionado por la similitud que presentan en su morfología y gama de colores

Geo-rectification and cloud-cover correction of multi-temporal Earth observation imagery

Over the past decades, improvements in remote sensing technology have led to mass proliferation of aerial imagery. This, in turn, opened vast new possibilities relating to land cover classification, cartography, and so forth. As applications in these fields became increasingly more complex, the amount of data required also rose accordingly and so, to satisfy these new needs, automated systems had to be developed. Geometric distortions in raw imagery must be rectified, otherwise the high accuracy requirements of the newest applications will not be attained. This dissertation proposes an automated solution for the pre-stages of multi-spectral satellite imagery classification, focusing on Fast Fourier Shift theorem based geo-rectification and multi-temporal cloud-cover correction. By automatizing the first stages of image processing, automatic classifiers can take advantage of a larger supply of image data, eventually allowing for the creation of semi-real-time mapping applications

Automated Image Registration Using Morphological Region of Interest Feature Extraction

With the recent explosion in the amount of remotely sensed imagery and the corresponding interest in temporal change detection and modeling, image registration has become increasingly important as a necessary first step in the integration of multi-temporal and multi-sensor data for applications such as the analysis of seasonal and annual global climate changes, as well as land use/cover changes. The task of image registration can be divided into two major components: (1) the extraction of control points or features from images; and (2) the search among the extracted features for the matching pairs that represent the same feature in the images to be matched. Manual control feature extraction can be subjective and extremely time consuming, and often results in few usable points. Automated feature extraction is a solution to this problem, where desired target features are invariant, and represent evenly distributed landmarks such as edges, corners and line intersections. In this paper, we develop a novel automated registration approach based on the following steps. First, a mathematical morphology (MM)-based method is used to obtain a scale-orientation morphological profile at each image pixel. Next, a spectral dissimilarity metric such as the spectral information divergence is applied for automated extraction of landmark chips, followed by an initial approximate matching. This initial condition is then refined using a hierarchical robust feature matching (RFM) procedure. Experimental results reveal that the proposed registration technique offers a robust solution in the presence of seasonal changes and other interfering factors. Keywords-Automated image registration, multi-temporal imagery, mathematical morphology, robust feature matching

DTW-Radon-based Shape Descriptor for Pattern Recognition

International audienceIn this paper, we present a pattern recognition method that uses dynamic programming (DP) for the alignment of Radon features. The key characteristic of the method is to use dynamic time warping (DTW) to match corresponding pairs of the Radon features for all possible projections. Thanks to DTW, we avoid compressing the feature matrix into a single vector which would otherwise miss information. To reduce the possible number of matchings, we rely on a initial normalisation based on the pattern orientation. A comprehensive study is made using major state-of-the-art shape descriptors over several public datasets of shapes such as graphical symbols (both printed and hand-drawn), handwritten characters and footwear prints. In all tests, the method proves its generic behaviour by providing better recognition performance. Overall, we validate that our method is robust to deformed shape due to distortion, degradation and occlusion

Building Height Extraction Based on Satellite GF-7 High-Resolution Stereo Image

High-resolution remote sensing images can distinguish objects of smaller size, so as to more clearly express the texture features and structural information of objects, and provide a data source for the development of large-scale mapping, high-precision stereo measurement and other fields. The purpose of this paper is to complete the height estimation of the buildings by analyzing the stereoscopic observation formed by the front and rear view images of the Gaofen-7 line array CCD. After determining the roof profile of the building on the rear-view image, assuming a series of object elevations of the building, that is, searching for elevations with a certain step distance within a certain elevation search range, adopt the object-based image matching VLL algorithm, Through the RFM imaging model of the Gaofen-7 sensor, the rear-view contour is projected to the front-view image, and then the PSNR is selected as the similarity measure of the window, and the similarity of the image block area corresponding to the front- and rear-view contour is calculated. Corresponds to the hypothetical object elevation as the estimated height of the building. Under the technical route of this paper, the height of buildings on high-resolution images can be estimated to a level within 3 meters of accuracy