Image Registration Workshop Proceedings

Automatic image registration has often been considered as a preliminary step for higher-level processing, such as object recognition or data fusion. But with the unprecedented amounts of data which are being and will continue to be generated by newly developed sensors, the very topic of automatic image registration has become and important research topic. This workshop presents a collection of very high quality work which has been grouped in four main areas: (1) theoretical aspects of image registration; (2) applications to satellite imagery; (3) applications to medical imagery; and (4) image registration for computer vision research

Feature-Based Image Registration

Image registration is the fundamental task used to match two or more partially overlapping images taken, for example, at different times, from different sensors, or from different viewpoints and stitch these images into one panoramic image comprising the whole scene. It is a fundamental image processing technique and is very useful in integrating information from different sensors, finding changes in images taken at different times, inferring three-dimensional information from stereo images, and recognizing model-based objects. Some techniques are proposed to find a geometrical transformation that relates the points of an image to their corresponding points of another image. To register two images, the coordinate transformation between a pair of images must be found. In this thesis, a feature-based method is developed to efficiently estimate an eight-parametric projective transformation model between pairs of images. The proposed approach applies wavelet transform to extract a number of feature points as the basis for registration. Each selected feature point is an edge point whose edge response is the maximum within a neighborhood. During the real matching process, we check each candidate pair in advance to see if it can possibly become a correct matching pair. Due to this checking, many unnecessary calculations involving cross-correlations can be screened in advance. Therefore, the search time for obtaining correct matching pairs is reduced significantly. Finally, based on the set of correctly matched feature point pairs, the transformation between two partially overlapping images can be decided

Automated Image Registration And Mosaicking For Multi-Sensor Images Acquired By A Miniature Unmanned Aerial Vehicle Platform

Algorithms for automatic image registration and mosaicking are developed for a miniature Unmanned Aerial Vehicle (MINI-UAV) platform, assembled by Air-O-Space International (AOSI) L.L.C.. Three cameras onboard this MINI-UAV platform acquire images in a single frame simultaneously at green (550nm), red (650 nm), and near infrared (820nm) wavelengths, but with shifting and rotational misalignment. The area-based method is employed in the developed algorithms for control point detection, which is applicable when no prominent feature details are present in image scenes. Because the three images to be registered have different spectral characteristics, region of interest determination and control point selection are the two key steps that ensure the quality of control points. Affine transformation is adopted for spatial transformation, followed by bilinear interpolation for image resampling. Mosaicking is conducted between adjacent frames after three-band co-registration. Pre-introducing the rotation makes the area-based method feasible when the rotational misalignment cannot be ignored. The algorithms are tested on three image sets collected at Stennis Space Center, Greenwood, and Oswalt in Mississippi. Manual evaluation confirms the effectiveness of the developed algorithms. The codes are converted into a software package, which is executable under the Microsoft Windows environment of personal computer platforms without the requirement of MATLAB or other special software support for commercial-off-the-shelf (COTS) product. The near real-time decision-making support is achievable with final data after its installation into the ground control station. The final products are color-infrared (CIR) composite and normalized difference vegetation index (NDVI) images, which are used in agriculture, forestry, and environmental monitoring

Fast Parametric Elastic Image Registration

We present an algorithm for fast elastic multidimensional intensity-based image registration with a parametric model of the deformation. It is fully automatic in its default mode of operation. In the case of hard real-world problems, it is capable of accepting expert hints in the form of soft landmark constraints. Much fewer landmarks are needed and the results are far superior compared to pure landmark registration. Particular attention has been paid to the factors influencing the speed of this algorithm. The B-spline deformation model is shown to be computationally more efficient than other alternatives. The algorithm has been successfully used for several two-dimensional (2-D) and three-dimensional (3-D) registration tasks in the medical domain, involving MRI, SPECT, CT, and ultrasound image modalities. We also present experiments in a controlled environment, permitting an exact evaluation of the registration accuracy. Test deformations are generated automatically using a random hierarchical fractional wavelet-based generator

Automated Complexity-Sensitive Image Fusion

To construct a complete representation of a scene with environmental obstacles such as fog, smoke, darkness, or textural homogeneity, multisensor video streams captured in diferent modalities are considered. A computational method for automatically fusing multimodal image streams into a highly informative and unified stream is proposed. The method consists of the following steps: 1. Image registration is performed to align video frames in the visible band over time, adapting to the nonplanarity of the scene by automatically subdividing the image domain into regions approximating planar patches 2. Wavelet coefficients are computed for each of the input frames in each modality 3. Corresponding regions and points are compared using spatial and temporal information across various scales 4. Decision rules based on the results of multimodal image analysis are used to combine thewavelet coefficients from different modalities 5. The combined wavelet coefficients are inverted to produce an output frame containing useful information gathered from the available modalities Experiments show that the proposed system is capable of producing fused output containing the characteristics of color visible-spectrum imagery while adding information exclusive to infrared imagery, with attractive visual and informational properties

Super Resolution of Wavelet-Encoded Images and Videos

In this dissertation, we address the multiframe super resolution reconstruction problem for wavelet-encoded images and videos. The goal of multiframe super resolution is to obtain one or more high resolution images by fusing a sequence of degraded or aliased low resolution images of the same scene. Since the low resolution images may be unaligned, a registration step is required before super resolution reconstruction. Therefore, we first explore in-band (i.e. in the wavelet-domain) image registration; then, investigate super resolution. Our motivation for analyzing the image registration and super resolution problems in the wavelet domain is the growing trend in wavelet-encoded imaging, and wavelet-encoding for image/video compression. Due to drawbacks of widely used discrete cosine transform in image and video compression, a considerable amount of literature is devoted to wavelet-based methods. However, since wavelets are shift-variant, existing methods cannot utilize wavelet subbands efficiently. In order to overcome this drawback, we establish and explore the direct relationship between the subbands under a translational shift, for image registration and super resolution. We then employ our devised in-band methodology, in a motion compensated video compression framework, to demonstrate the effective usage of wavelet subbands. Super resolution can also be used as a post-processing step in video compression in order to decrease the size of the video files to be compressed, with downsampling added as a pre-processing step. Therefore, we present a video compression scheme that utilizes super resolution to reconstruct the high frequency information lost during downsampling. In addition, super resolution is a crucial post-processing step for satellite imagery, due to the fact that it is hard to update imaging devices after a satellite is launched. Thus, we also demonstrate the usage of our devised methods in enhancing resolution of pansharpened multispectral images

Multi-resolution dental image registration based on genetic algorithm

The Automated Dental Identification System (ADIS) is a Post Mortem Dental Identification System. This thesis presents dental image registration, required for the preprocessing steps of the image comparison component of ADIS. We proposed a multi resolution dental image registration based on genetic algorithms. The main objective of this research is to develop techniques for registration of extracted subject regions of interest with corresponding reference regions of interest.;We investigated and implemented registration using two multi resolution techniques namely image sub sampling and wavelet decomposition. Multi resolution techniques help in the reduction of search data since initial registration is carried at lower levels and results are updated as the levels of resolutions increase. We adopted edges as image features that needed to be aligned. Affine transformations were selected to transform the subject dental region of interest to achieve better alignment with the reference region of interest. These transformations are known to capture complex image distortions. The similarity between subject and reference image has been computed using Oriented Hausdorff Similarity measure that is robust to severe noise and image degradations. A genetic algorithm was adopted to search for the best transformation parameters that give maximum similarity score.;Testing results show that the developed registration algorithm yielded reasonable results in accuracy for dental test cases that contained slight misalignments. The relative percentage errors between the known and estimated transformation parameters were less than 20% with a termination criterion of a ten minute time limit. Further research is needed for dental cases that contain high degree of misalignment, noise and distortions

Local Phase Coherence Measurement for Image Analysis and Processing

The ability of humans to perceive significant pattern and structure of an image is something which humans take for granted. We can recognize objects and patterns independent of changes in image contrast and illumination. In the past decades, it has been widely recognized in both biology and computer vision that phase contains critical information in characterizing the structures in images. Despite the importance of local phase information and its significant success in many computer vision and image processing applications, the coherence behavior of local phases at scale-space is not well understood. This thesis concentrates on developing an invariant image representation method based on local phase information. In particular, considerable effort is devoted to study the coherence relationship between local phases at different scales in the vicinity of image features and to develop robust methods to measure the strength of this relationship. A computational framework that computes local phase coherence (LPC) intensity with arbitrary selections in the number of coefficients, scales, as well as the scale ratios between them has been developed. Particularly, we formulate local phase prediction as an optimization problem, where the objective function computes the closeness between true local phase and the predicted phase by LPC. The proposed framework not only facilitates flexible and reliable computation of LPC, but also broadens the potentials of LPC in many applications. We demonstrate the potentials of LPC in a number of image processing applications. Firstly, we have developed a novel sharpness assessment algorithm, identified as LPC-Sharpness Index (LPC-SI), without referencing the original image. LPC-SI is tested using four subject-rated publicly-available image databases, which demonstrates competitive performance when compared with state-of-the-art algorithms. Secondly, a new fusion quality assessment algorithm has been developed to objectively assess the performance of existing fusion algorithms. Validations over our subject-rated multi-exposure multi-focus image database show good correlations between subjective ranking score and the proposed image fusion quality index. Thirdly, the invariant properties of LPC measure have been employed to solve image registration problem where inconsistency in intensity or contrast patterns are the major challenges. LPC map has been utilized to estimate image plane transformation by maximizing weighted mutual information objective function over a range of possible transformations. Finally, the disruption of phase coherence due to blurring process is employed in a multi-focus image fusion algorithm. The algorithm utilizes two activity measures, LPC as sharpness activity measure along with local energy as contrast activity measure. We show that combining these two activity measures result in notable performance improvement in achieving both maximal contrast and maximal sharpness simultaneously at each spatial location

Similarity Measure Based on Entropy and Census and Multi-Resolution Disparity Estimation Technique for Stereo Matching

Stereo matching is one of the most active research areas in the field of computer vision. Stereo matching aims to obtain 3D information by extracting correct correspondence between two images captured from different point of views. There are two research parts in stereo matching: similarity measure between correspondence points and optimization technique for dence disparity estimation. The crux of stereo matching problem in similarity measure perspective is how to deal with the inferent points ambiguity that results from the ambiguous local appearances of image points. Similarity measures in stereo matching are classified as feature-based, intensity-based or non-parametric measure. And most similarity measures in the literatures are based on pixel intensity comparison. When images are taken at different illumination conditions or different sensors used, it is very unlikely that the corresponding pixels would have the same intensity creating false correspondences if it is only based on intensity matching functions alone. Especially illumination variations between input images can cause serious degrade in the performance of stereo matching algorithms. In this situation, mutual information-based method is powerful. However, it is still ambiguous or erroneous in considering local illumination variations between images. Therefore, similarity measure to these radiometric variations are demanded and become inevitable for stereo matching. Optimization method in stereo matching can be classified into two categories: local and global optimization methods, and most state-of-the-art algorithms fall into global optimization method. Global optimization methods can greatly suppress the matching ambiguities caused by various factors such as occluded and textureless regions. However, They are usually computationally expensive due to the slow-converging optimization process. In this paper, it was proposed that a stereo matching similarity measure based on entropy and census transform and an optimization technique using dynamic programming to estimate disparity efficiently based on multi-resolution method. Proposed similarity measure is composed of entropy, Haar wavelet feature vector, and modified Census transform. In general, mutual information similarity measure based on entropy about stereo images and disparity map is a popular and powerful similarity measure which is robust to complex intensity transformation. However, it is still ambiguous or erroneous with local radiometric variations, since it only accounts for global variation between images, and does not contain spatial information. Haar wavelet response can express frequency properties of image regions and is robust to various intensity changes and bias. Therefore, entropy was utilized with Haar wavelet feature vector as geometric measure. Modified Census transform was used as another spatial similarity measure. Census transform is a well-known non-parametric measure. And it is powerful to textureless and disparity discontinuity region and robust to noisy environment. A combination of entropy with Haar wavelet feature vector and modified Census transform as similarity measure was proposed to find correspondence. It is invariant to local radiometric variations and global illumination changes, so it can be applied to find correspondence for images which undergo local as well as global radiometric variations. Proposed optimization method is a new disparity estimation technique based on dynamic programming. A method using dynamic programming with 8-direction energy aggregation to estimate accurate disparity map was applied. Using 8-direction energy aggregation, accurate disparities can be found at disparity discontinuous region and suppress a streaking phenomenon in disparity map. Finally, the multi-resolution scheme was proposed to increase efficiency while processing and disparity estimation method. A Gaussian pyramid which prevent the ailasing at low-resolution image pyramid levels was used. And the multi-resolution scheme was proposed to perform matching at every levels to find accurate disparity. This method can perform matching efficiently and make accurate disparity map. And proposed method was validated with experimental results on stereo images.제 1 장 서론...........................................1 1.1 연구 목적 및 배경...............................1 1.2관련 연구........................................3 1.3연구 내용........................................6 1.4논문의 구성......................................7 제 2 장 스테레오 시각과 스테레오 정합..................8 2.1 스테레오 시각...................................8 2.2스테레오정합....................................10 2.2.1 유사도 척도.................................10 2.2.2 최적화 방법.................................15 2.3 환경 변화에 강인한 유사도 척도.................18 2.3.1 특징 기반 유사도 척도.......................18 2.3.2 명암도 기반 유사도 척도.....................19 2.3.3 비모수 유사도 척도..........................22 제 3 장 엔트로피 및 Census 기반의 유사도 척도.........24 3.1 엔트로피 기반의 유사도 척도....................26 3.1.1 엔트로피....................................26 3.1.2 엔트로피를 이용한 MI 유사도 척도............27 3.2 제안한 Haar 웨이블렛 특징을 결합한 엔트로피 유사도척도.................................................28 3.2.1 화소 단위 엔트로피.........................28 3.2.2 Haar웨이블렛 특징을 결합한 엔트로피........35 3.3 제안한Census 변환 기반의 유사도 척도..........46 3.3.1 Census 변환................................46 3.3.2 제안한 Census 변환을 이용한 유사도 척도....49 제 4 장 8방향 동적 계획법을 이용한 변위추정..........53 4.1 동적 계획법...................................53 4.2 제안한 8방향 동적 계획법......................57 제 5 장 다해상도 기반의 스테레오 정합................67 5.1 가우시안 영상 피라미드........................67 5.2 제안한 다해상도 기반 스테레오 정합............71 제 6 장 실험 및 고찰.................................77 6.1 정합 성능 평가 방법...........................77 6.2 스테레오 정합 실험............................79 6.2.1 RDS 영상 실험..............................79 6.2.2 환경 변화가 없는 표준 영상 실험............84 6.2.3 환경 변화가 발생한 표준 영상 실험..........92 6.2.4 실제 영상 실험............................110 6.3 계산 속도....................................118 6.4 제안한 방법의 정합 성능에 대한 고찰..........121 제 7 장 결 론.....................................123 참고 문헌...........................................12