Keypoint descriptor fusion with Dempster-Shafer Theory

Keypoint matching is the task of accurately nding the location of a scene point in two images. Many keypoint descriptors have been proposed in the literature aiming at providing robustness against scale, translation and rotation transformations, each having advantages and disadvantages. This paper proposes a novel approach to fuse the information from multiple keypoint descriptors using Dempster-Shafer Theory of evidence [1], which has proven particularly e cient in combining sources of information providing incomplete, imprecise, biased, and con ictive knowledge. The matching results of each descriptor are transformed into an evidence distribution on which a con dence factor is computed making use of its entropy. Then, the evidence distributions are fused using Dempster-Shafer Theory (DST), considering its con dence. As result of the fusion, a new evidence distribution that improves the result of the best descriptor is obtained. Our method has been tested with SIFT, SURF, ORB, BRISK and FREAK descriptors using all possible combinations of them. Results on the Oxford keypoint dataset [2] shows that the proposed approach obtains an improvement of up to 10% compared to the best one (FREAK)

Contribuciones a la estimación de pose de cámara

El problema cuya resolución tiene como objetivo determinar la orientación y localización de una cámara respecto a un sistema de coordenadas se denomina Estimación de la pose de la cámara. Las soluciones basadas en imágenes para la resolución de este problema son una opción interesante debido a su bajo coste. El inconveniente fundamental de esta opción es que su precisión puede verse afectada debido a la presencia de ruido en la imagen. Trabajar con imágenes para estimar la pose de cámara está muy relacionado con dos problemas denominados Perspective-n-Point (PnP) y Bundle Adjustment (ajuste del haz). Dado un conjunto de n correspondencias entre puntos del espacio 3D y sus proyecciones 2D en una imagen, los métodos PnP tratan de obtener la pose de la cámara. Cuando la información acerca de la posición 3D de los puntos es desconocida, pero sí se tiene conocimiento de una serie de proyecciones 2D tomadas desde diferentes puntos de vista del mismo punto 3D, el ajuste del haz trata de estimar simultáneamente la posición tridimensional de los puntos y la pose de la cámara. Debido a esto la tarea de buscar correspondencias, ya sea entre puntos de la escena 3D y su proyección 2D en la imagen, o entre varias proyecciones 2D de imágenes diferentes no es trivial y resulta fundamental para la resolución de los problemas mencionados anteriormente. En esta Tesis Doctoral se han propuesto dos métodos novedosos para el problema de búsqueda de correspondencias usando marcas naturales y artificiales. En nuestra primera contribución, basada en el uso de marcas naturales, proponemos un método para encontrar correspondencias entre puntos 2D de diferentes imágenes, utilizando un nuevo enfoque de fusión que combina la información proporcionada por varios descriptores haciendo uso de la Teoría de Dempster-Shafer. El método propuesto es capaz de fusionar diferentes fuentes de información teniendo en cuenta además su confianza relativa con el fin de obtener una mejor solución. La segunda contribución se centra en el problema de búsqueda de proyecciones 2D de puntos 3D conocidos. Proponemos un enfoque novedoso para identificar marcadores artificiales, que son una alternativa muy popular cuando se requiere robustez y velocidad. En concreto, proponemos abordar el problema de identificación de marcadores artificiales como un problema de clasificación. Como consecuencia, hemos entrenado métodos capaces de detectar marcadores en imágenes afectadas por situaciones complejas como el desenfoque o la luz no uniforme. Ambas propuestas realizadas en esta Tesis han sido comparadas con métodos del estado del arte mostrando mejoras que son estadísticamente significativas.Camera pose estimation is the problem of finding the orientation and localization of a camera with respect to an arbitrary coordinate system. Image-based solutions for this problem are an interesting option because its reduced cost. However, their main drawback is that the accuracy of the results is afected by the presence of noise in the images. The use of images for the camera pose estimation task is strongly related to the Perspective-n-Point (PnP) and Bundle Adjustment problem. Given a set of n correspondences between 3D points and its 2D projections on the image, PnP methods provide estimations of the camera pose. In addition, when the information about the 3D positions is unknow but a set of 2D projections taken from diferent viewpoints of the same 3D point are known, Bundle Adjustment methods are capable of finding simultaneously the 3D position of the points and the camera pose. Then the task of finding correspondences between 3D points and its 2D projections, and between 2D projections of diferent images is a fundamental step for the above mentioned problems. This PhD Thesis proposes two novel approaches to solve the problem of finding correspondeces using both natural and artificial features. In our first contribution, based on natural features, we propose a novel approach to find 2D correspondeces between images by a novel fusion approach combining information provided by several descriptors using the Dempster-Shafer Theory. The proposed method is able to fuse diferent sources of information considering their relative confidence in order to provide a better solution. Our second contribution focuses on the problem of nding the 2D projections of 3D points. We propose a novel approach for identification of artificial landmarks, which are a very popular method when robustness and speed are required. In particular, we propose to tackle the marker identi cation problem as a classi cation one. As a consequence, we develop methods able to detect such markers in complex real situations such as blurring and non-uniform lightning. The two contributions made in this Thesis have been compared with the state-of-art methods showing statistically significant improvements

Mathematics of biomimetics for active echo- and electro-sensing

Active sensing animals may inspire the development of new technologies that mimic their sensing behavior. Electric fish, for instance, orient themselves at night in complete darkness by using their active electro-sensing system. They generate a stable, relatively high-frequency, weak electric field and perceive the transdermal potential modulations caused by nearby targets with different electromagnetic properties than the surrounding water. Since they have an electric sense that allows underwater navigation, target classification and intraspecific communication, they are privileged animals for bio-inspiring man-built autonomous systems. Bats, on the other hand, process the reflected echoes due to the presence of acoustic inclusions for echolocation. In general, they use acoustic waves for most of the perceptual tasks, that range from hunting to navigating. This thesis introduces premier algorithms in electro-sensing and echo-sensing. The weakly electric fish is able to retrieve much more information about the target by approaching it. To mimic this behavior, an innovative (real-time) multi-scale method for target classification in electro-sensing is presented. The method is based on a family of transform-invariant shape descriptors computed from generalized polarization tensors (GPTs) reconstructed at multiple scales. The evidence provided by the different descriptors at each scale is fused using Dempster-Shafer Theory. Numerical simulations show that the recognition algorithm we proposed performs undoubtedly well and yields a robust classification. For real-world applications, inhomogeneous targets have to be identified. The shape descriptor-based classification algorithm is extended in order to consider inhomogenous material parameters. The approach is based on new invariants for the contracted generalized polarization tensors associated with inhomogeneous objects. The numerical simulations show that by comparing these invariants with those in a dictionary of precomputed homogeneous and inhomogeneous targets, one can successfully classify the inhomogeneous target. Another problem concerns intraspecific electro-communication for weakly electric fish. In particular, a description on how the fish circumvent the jamming issue for both electro-communication and active electro-sensing is presented. The main result is a real-time tracking algorithm, which provides a new approach to the communication problem. It finds a natural application in robotics, where efficient communication strategies are needed to be implemented by bio-inspired underwater robots. The concept of time-dependent polarization tensors (TDPTs) for the wave equation associated to a diametrically small acoustic inclusion, with constitutive parameters different from those of the background and size smaller than the operating wavelength, is used to mimic the echo-sensing capabilities of a static bat. Firstly, the solution to the Helmholtz equation is considered, and a rigorous systematic derivation of a complete asymptotic expansion of the scattered field due to the presence of the inclusion is presented. Then, by applying the Fourier transform, the corresponding time-domain expansion is readily obtained after truncating the high frequencies. The new concept of TDPTs is shown to be promising for performing imaging. Numerical simulations are presented, showing that the TDPTs reconstructed from noisy measurements allow to image fine shape details of the inclusion