Robust Image Matching under a Large Disparity

We present a new method for detecting point matches between two images without using any combinatorial search. Our strategy is to impose various local and non-local constraints as "soft" constraints by introducing their "confidence" measures via "mean-field approximations". The computation is a cascade of evaluating the confidence values and sorting according to them. In the end, we impose the "hard" epipolar constraint by RANSAC. We also introduce a model selection procedure to test if the image mapping can be regarded as a homography. We demonstrate the effectiveness of our method by real image examples

Development of a fast panoramic face mosaicing and recognition system

In this article, we present some development results of a system that performs mosaicing of panoramic faces. Our objective is to study the feasibility of panoramic face construction in real-time. This led us to conceive of a very simple acquisition system composed of 5 standard cameras and 5 face views taken simultaneously at different angles. Then, we chose an easily hardware-achievable algorithm: successive linear transformation, in order to compose a panoramic face of 150° from these 5 views. The method has been tested on hundreds of faces. In order to validate our system of panoramic face mosaicing, we also conducted a preliminary study on panoramic faces recognition, based on the «eigenfaces» method. Experimental results obtained show the feasibility and viability of our system. This allows us to envisage later a hardware implantation. We also are considering applying our system to other applications such as human expression categorization using movement estimation and fast 3D face reconstruction.Dans cet article, nous présentons quelques résultats sur le développement d’un système de mosaïquage de visages panoramiques. Notre objectif est d’étudier la faisabilité de construction de visages panoramiques en temps réel. Ceci nous a conduit tout d’abord à concevoir un système d’acquisition très simple, composé de 5 caméras standards qui réalise la prise de 5 vues simultanément sous différents angles. Puis, nous avons choisi un algorithme facilement implantable sur des systèmes embarqués. Cet algorithme est basé sur des transformations linéaires successives, pour composer un visage panoramique de 150 ° à partir de ces 5 vues. La méthode a été testée sur une centaine de visages. Nous avons aussi effectué une étude préliminaire sur la reconnaissance de visages panoramiques dans le but de valider notre système de mosaïquage de visages. La reconnaissance est basée sur le modèle de « visages propres ». Les résultats expérimentaux ont montré la faisabilité et la viabilité du système proposé permettant d’envisager une future implantation matérielle. Nous pensons aussi utiliser notre système de mosaïquage dans d’autres applications comme la reconstruction rapide de visages 3D et la catégorisation des expressions basée sur le mouvement

Using Linear Features for Aerial Image Sequence Mosaiking

With recent advances in sensor technology and digital image processing techniques, automatic image mosaicking has received increased attention in a variety of geospatial applications, ranging from panorama generation and video surveillance to image based rendering. The geometric transformation used to link images in a mosaic is the subject of image orientation, a fundamental photogrammetric task that represents a major research area in digital image analysis. It involves the determination of the parameters that express the location and pose of a camera at the time it captured an image. In aerial applications the typical parameters comprise two translations (along the x and y coordinates) and one rotation (rotation about the z axis). Orientation typically proceeds by extracting from an image control points, i.e. points with known coordinates. Salient points such as road intersections, and building corners are commonly used to perform this task. However, such points may contain minimal information other than their radiometric uniqueness, and, more importantly, in some areas they may be impossible to obtain (e.g. in rural and arid areas). To overcome this problem we introduce an alternative approach that uses linear features such as roads and rivers for image mosaicking. Such features are identified and matched to their counterparts in overlapping imagery. Our matching approach uses critical points (e.g. breakpoints) of linear features and the information conveyed by them (e.g. local curvature values and distance metrics) to match two such features and orient the images in which they are depicted. In this manner we orient overlapping images by comparing breakpoint representations of complete or partial linear features depicted in them. By considering broader feature metrics (instead of single points) in our matching scheme we aim to eliminate the effect of erroneous point matches in image mosaicking. Our approach does not require prior approximate parameters, which are typically an essential requirement for successful convergence of point matching schemes. Furthermore, we show that large rotation variations about the z-axis may be recovered. With the acquired orientation parameters, image sequences are mosaicked. Experiments with synthetic aerial image sequences are included in this thesis to demonstrate the performance of our approach

Weighted and filtered mutual information: A Metric for the automated creation of panoramas from views of complex scenes

To contribute a novel approach in the field of image registration and panorama creation, this algorithm foregoes any scene knowledge, requiring only modest scene overlap and an acceptable amount of entropy within each overlapping view. The weighted and filtered mutual information (WFMI) algorithm has been developed for multiple stationary, color, surveillance video camera views and relies on color gradients for feature correspondence. This is a novel extension of well-established maximization of mutual information (MMI) algorithms. Where MMI algorithms are typically applied to high altitude photography and medical imaging (scenes with relatively simple shapes and affine relationships between views), the WFMI algorithm has been designed for scenes with occluded objects and significant parallax variation between non-affine related views. Despite these typically non-affine surveillance scenarios, searching in the affine space for a homography is a practical assumption that provides computational efficiency and accurate results, even with complex scene views. The WFMI algorithm can perfectly register affine views, performs exceptionally well with near-affine related views, and in complex scene views (well beyond affine constraints) the WFMI algorithm provides an accurate estimate of the overlap regions between the views. The WFMI algorithm uses simple calculations (vector field color gradient, Laplacian filtering, and feature histograms) to generate the WFMI metric and provide the optimal affine relationship. This algorithm is unique when compared to typical MMI algorithms and modern registration algorithms because it avoids almost all a priori knowledge and calculations, while still providing an accurate or useful estimate for realistic scenes. With mutual information weighting and the Laplacian filtering operation, the WFMI algorithm overcomes the failures of typical MMI algorithms in scenes where complex or occluded shapes do not provide sufficiently large peaks in the mutual information maps to determine the overlap region. This work has currently been applied to individual video frames and it will be shown that future work could easily extend the algorithm into utilizing motion information or temporal frame registrations to enhance scenes with smaller overlap regions, lower entropy, or even more significant parallax and occlusion variations between views

Algorithms, Protocols & Systems for Remote Observation Using Networked Robotic Cameras

Emerging advances in robotic cameras, long-range wireless networking, and distributed sensors make feasible a new class of hybrid teleoperated/autonomous robotic remote "observatories" that can allow groups of peoples, via the Internet, to observe, record, and index detailed activity occurred in remote site. Equipped with robotic pan-tilt actuation mechanisms and a high-zoom lens, the camera can cover a large region with very high spatial resolution and allows for observation at a distance. High resolution motion panorama is the most nature data representation. We develop algorithms and protocols for high resolution motion panorama. We discover and prove the projection invariance and achieve real time image alignment. We propose a minimum variance based incremental frame alignment algorithm to minimize the accumulation of alignment error in incremental image alignment and ensure the quality of the panorama video over the long run. We propose a Frame Graph based panorama documentation algorithm to manage the large scale data involved in the online panorama video documentation. We propose a on-demand high resolution panorama video-streaming system that allows on-demand sharing of a high-resolution motion panorama and efficiently deals with multiple concurrent spatial-temporal user requests. In conclusion, our research work on high resolution motion panorama have significantly improve the efficiency and accuracy of image alignment, panorama video quality, data organization, and data storage and retrieving in remote observation using networked robotic cameras

Image Mosaicing by Stratified Matching

This paper focuses on the latte