Faster and better: a machine learning approach to corner detection

The repeatability and efficiency of a corner detector determines how likely it is to be useful in a real-world application. The repeatability is importand because the same scene viewed from different positions should yield features which correspond to the same real-world 3D locations [Schmid et al 2000]. The efficiency is important because this determines whether the detector combined with further processing can operate at frame rate. Three advances are described in this paper. First, we present a new heuristic for feature detection, and using machine learning we derive a feature detector from this which can fully process live PAL video using less than 5% of the available processing time. By comparison, most other detectors cannot even operate at frame rate (Harris detector 115%, SIFT 195%). Second, we generalize the detector, allowing it to be optimized for repeatability, with little loss of efficiency. Third, we carry out a rigorous comparison of corner detectors based on the above repeatability criterion applied to 3D scenes. We show that despite being principally constructed for speed, on these stringent tests, our heuristic detector significantly outperforms existing feature detectors. Finally, the comparison demonstrates that using machine learning produces significant improvements in repeatability, yielding a detector that is both very fast and very high quality.Comment: 35 pages, 11 figure

3-D Reconstruction of Urban Scenes from Sequences of Images

In this paper, we address the problem of the recovery of the Euclidean geometry of a scene from a sequence of images without any prior knowledge either about the parameters of the cameras, or about the motion of the camera(s). We do not require any knowledge of the absolute coordinates of some control points in the scene to achieve this goal. Using various computer vision tools, we establish correspondences between images and recover the epipolar geometry of the set of images, from which we show how to compute the complete set of perspective projection matrices for each camera position. These being known, we proceed to reconstruct the scene. This reconstruction is defined up to an unknown projective transformation (i.e. is parameterized with 15 arbitrary parameters). Next we show how to go from this reconstruction to a more constrained class of reconstructions, defined up to an unknown affine transformation (i.e. parameterized with 12 arbitrary parameters) by exploiting known geometr..

A new asymmetrical corner detector(ACD) for a semi-automatic image co-registration scheme

Co-registration of multi-sensor and multi-temporal images is essential for remote sensing applications. In the image co-registration process, automatic Ground Control Points (GCPs) selection is a key technical issue and the accuracy of GCPs localization largely accounts for the final image co-registration accuracy. In this thesis, a novel Asymmetrical Corner Detector (ACD) algorithm based on auto-correlation is presented and a semi-automatic image co-registration scheme is proposed. The ACD is designed with the consideration of the fact that asymmetrical corner points are the most common reality in remotely sensed imagery data. The ACD selects points more favourable to asymmetrical points rather than symmetrical points to avoid incorrect selection of flat points which are often highly symmetrical. The experimental results using images taken by different sensors indicate that the ACD has obtained excellent performance in terms of point localization and computation efficiency. It is more capable of selecting high quality GCPs than some well established corner detectors favourable to symmetrical corner points such as the Harris Corner Detector (Harris and Stephens, 1988). A semi-automatic image co-registration scheme is then proposed, which employs the ACD algorithm to extract evenly distributed GCPs across the overlapped area in the reference image. The scheme uses three manually selected pairs of GCPs to determine the initial transformation model and the overlapped area. Grid-control and nonmaximum suppression methods are used to secure the high quality and spread distribution of GCPs selected. It also involves the FNCC (fast normalised crosscorrelation) algorithm (Lewis, 1995) to refine the corresponding point locations in the input image and thus the GCPs are semi-automatically selected to proceed to the polynomial fitting image rectification. The performance of the proposed coregistration scheme has been demonstrated by registering multi-temporal, multi-sensor and multi-resolution images taken by Landsat TM, ETM+ and SPOT sensors. Experimental results show that consistent high registration accuracy of less than 0.7 pixels RMSE has been achieved. Keywords: Asymmetrical corner points, image co-registration, AC

Collaborative Appearance-Based Place Recognition and Improving Place Recognition Using Detection of Dynamic Objects

This dissertation makes contributions to the problem of Long-Term Appearance-Based Place Recognition. We present a framework for place recognition in a collaborative scheme and a method to reduce the impact of dynamic objects on place representations. We demonstrate our findings using a state-of-the-art place recognition approach. We begin in Part I by describing the general problem of place recognition and its importance in applications where accurate localization is crucial. We discuss feature detection and description and also explain the functioning of several place recognition frameworks. In Part II, we present a novel framework for collaboration between agents from a pure appearance-based place recognition perspective. Using this framework, multiple agents can efficiently share partial or complete knowledge about places and benefit from their teamwork. This collaborative framework allows agents with limited storage and memory capacity to become useful in environment exploration tasks (for instance, by enabling remote recognition); includes procedures to manage an agent’s memory load and distributes knowledge of places across agents; allows the reuse of knowledge from one agent to another; and increases the tolerance for failure of individual agents. Part II also defines metrics which allow us to measure the performance of a system that uses the collaborative framework. Finally, in Part III, we present an innovative method to improve the recognition of places in environments densely populated by dynamic objects. We demonstrate that we can improve the recognition performance in these environments by incorporating high- level information from dynamic objects. Tests conducted using a synthetic dataset show the benefits of our approach. The proposed method allows the system to significantly improve the recognition performance in the photo-realistic dataset while reducing storage requirements, resulting in up to 23.7 percent less storage space than the state-of-the-art approach that we have extended; smaller representations also reduced the time required to match places. In Part III, we also formulate the concept of a valid place representation and determine the quality of the observation based on dynamic objects present in the agent’s view. Of course, recognition systems that are sensitive to dynamic objects incur additional computational costs to recognize those objects. We show that this additional cost is outweighed by the benefits that incorporating dynamic object detection in the place recognition pipeline. Our findings can be used in many applications, including applications for navigation, e.g. assisting visually impaired individuals with navigating indoors, or autonomous vehicles

Robotic Goal-Based Semi-Autonomous Algorithms Improve Remote Operator Performance

The focus of this research was to determine if reliable goal-based semi-autonomous algorithms are able to improve remote operator performance or not. Two semi-autonomous algorithms were examined: visual servoing and visual dead reckoning. Visual servoing uses computer vision techniques to generate movement commands while using internal properties of the camera combined with sensor data that tell the robot its current position based on its previous position. This research shows that the semi-autonomous algorithms developed increased performance in a measurable way. An analysis of tracking algorithms for visual servoing was conducted and tracking algorithms were enhanced to make them as robust as possible. The developed algorithms were implemented on a currently fielded military robot and a human-in-the-loop experiment was conducted to measure performance

CONTRIBUTION A LA STEREOVISION OMNIDIRECTIONNELLE ET AU TRAITEMENT DES IMAGES CATADIOPTRIQUES : APPLICATION AUX SYSTEMES AUTONOMES

Computer vision and digital image processing are two disciplines aiming to endow computers with a sense of perception and image analysis, similar to that of humans. Artificial visual perception can be greatly enhanced when a large field of view is available. This thesis deals with the use of omnidirectional cameras as a mean of expanding the field of view of computer vision systems. The visual perception of depth (3D) by means of omnistereo configurations, and special processing algorithms adapted to catadioptric images, are the main subjects studied in this thesis. Firstly a survey on 3D omnidirectional vision systems is conducted. It highlights the main approaches for obtaining depth information, and provides valuable indications for the choice of the configuration according to the application requirements. Then the design of an omnistereo sensor is addressed, we present a new configuration of the proposed sensor formed by a unique catadioptric camera, dedicated to robotic applications. An experimental investigation of depth estimation accuracy was conducted to validate the new configuration.Digital images acquired by catadioptric cameras present various special geometrical proprieties, such as non-uniform resolution and severe radial distortions. The application of conventional algorithms to process such images is limited in terms of performance. For that, new algorithms adapted to the spherical geometry of catadioptric images have been developed.Gathered omnidirectional computer vision techniques were finally used in two real applications. The first concerns the integration of catadioptric cameras to a mobile robot. The second focuses on the design of a solar tracker, based on a catadioptric camera.The results confirm that the adoption of such sensors for autonomous systems offer more performance and flexibility in regards to conventional sensors.La vision par ordinateur est une discipline qui vise doter les ordinateurs d’un sens de perception et d’analyse d'image semblable à celui de l’homme. La perception visuelle artificielle peut être grandement améliorée quand un grand champ de vision est disponible. Cette thèse traite de l'utilisation des caméras omnidirectionnelles comme un moyen d'élargir le champ de vision des systèmes de vision artificielle. La perception visuelle de la profondeur (3D) par le biais de configurations omnistéréo, et les algorithmes de traitement adaptés aux images catadioptriques, sont les principaux sujets étudiés.Tout d'abord une étude des systèmes de vision omnidirectionnelle 3D est menée. Elle met en évidence les principales approches pour obtenir l’information sur la profondeur et fournit des indications précieuses sur le choix de la configuration en fonction des besoins de l'application. Ensuite, la conception d'un capteur omnistéréo est adressée ; nous présentons une nouvelle configuration du capteur proposé basé une caméra catadioptrique unique, et dédié à la robotique mobile. Des expérimentations sur la précision d’estimation de la profondeur ont été menées pour valider la nouvelle configuration. Les images catadioptriques présentent diverses propriétés géométriques particulières, telles que la résolution non-uniforme et de fortes distorsions radiales. L’application des algorithmes de traitement classiques à ce type d’images se trouve limité en termes de performances. Dans ce sens, de nouveaux algorithmes adaptés à la géométrie sphérique de ces images ont été développés.Les techniques de vision omnidirectionnelle artificielle recueillies ont été finalement exploitées dans deux applications réelles. La première concerne l’intégration des caméras catadioptriques à un robot mobile. La seconde porte sur la conception d’un suiveur solaire, à base d’une caméra catadioptrique.Les résultats obtenus confirment que l’adoption de tels capteurs pour les systèmes autonomes offre plus de performances et de flexibilité en regards aux capteurs classiques