Line matching based on planar homography for stereo aerial images

© 2015 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). We propose an efficient line matching algorithm for a pair of calibrated aerial photogrammetric images, which makes use of sparse 3D points triangulated from 2D point feature correspondences to guide line matching based on planar homography. Two different strategies are applied in the proposed line matching algorithm for two different cases. When three or more points can be found coplanar with the line segment to be matched, the points are used to fit a plane and obtain an accurate planar homography. When one or two points can be found, the approximate terrain plane parallel to the line segment is utilized to compute an approximate planar homography. Six pairs of rural or urban aerial images are used to demonstrate the efficiency and validity of the proposed algorithm. Compared with line matching based on 2D point feature correspondences, the proposed method can increase the number of correctly matched line segments. In addition, compared with most line matching methods that do not use 2D point feature correspondences, the proposed method has better efficiency, although it obtains fewer matches. The C/C++ source code for the proposed algorithm is available at http://services.eng.uts.edu.au/~sdhuang/research.htm

Automatic Reassembly Method of 3D Thin-wall Fragments Based on Derivative Dynamic Time Warping

In order to address the automatic virtual reassembling of 3D thin-wall fragments, this paper proposes a 3D fragment reassembly method based on derivative dynamic time warping. Firstly, a calculation method of discrete curvature and torsion is designed to solve the difficulty of calculating curvature and torsion of discrete data points and eliminate effectively the noise interferences in the calculation process. Then, it takes curvature and torsion as the feature descriptors of the curve, searches the candidate matching line segments by the derivative dynamic time warping (DDTW) method with the feature descriptors, and records the positions of the starting and ending points of each candidate matching segment. After that, it designs a voting mechanism with the geometric invariant as the constraint information to select further the optimal matching line segments. Finally, it adopts the least squares method to estimate the rotation and transformation matrices and uses the iterative closest point (ICP) method to complete the reassembly of fragments. The experimental results show that the reassembly error is less than 1mm and that the reassembly effect is good. The method can solve the 3D curve matching in case there are partial feature defects, and can achieve the virtual restoration of the broken thin-wall fragment model quickly and effectively

Morphological Feature Extraction for Automatic Registration of Multispectral Images

The task of image registration can be divided into two major components, i.e., the extraction of control points or features from images, and the search among the extracted features for the matching pairs that represent the same feature in the images to be matched. Manual extraction of control features can be subjective and extremely time consuming, and often results in few usable points. On the other hand, automated feature extraction allows using invariant target features such as edges, corners, and line intersections as relevant landmarks for registration purposes. In this paper, we present an extension of a recently developed morphological approach for automatic extraction of landmark chips and corresponding windows in a fully unsupervised manner for the registration of multispectral images. Once a set of chip-window pairs is obtained, a (hierarchical) robust feature matching procedure, based on a multiresolution overcomplete wavelet decomposition scheme, is used for registration purposes. The proposed method is validated on a pair of remotely sensed scenes acquired by the Advanced Land Imager (ALI) multispectral instrument and the Hyperion hyperspectral instrument aboard NASA's Earth Observing-1 satellite

Automated Image Registration Using Morphological Region of Interest Feature Extraction

With the recent explosion in the amount of remotely sensed imagery and the corresponding interest in temporal change detection and modeling, image registration has become increasingly important as a necessary first step in the integration of multi-temporal and multi-sensor data for applications such as the analysis of seasonal and annual global climate changes, as well as land use/cover changes. The task of image registration can be divided into two major components: (1) the extraction of control points or features from images; and (2) the search among the extracted features for the matching pairs that represent the same feature in the images to be matched. Manual control feature extraction can be subjective and extremely time consuming, and often results in few usable points. Automated feature extraction is a solution to this problem, where desired target features are invariant, and represent evenly distributed landmarks such as edges, corners and line intersections. In this paper, we develop a novel automated registration approach based on the following steps. First, a mathematical morphology (MM)-based method is used to obtain a scale-orientation morphological profile at each image pixel. Next, a spectral dissimilarity metric such as the spectral information divergence is applied for automated extraction of landmark chips, followed by an initial approximate matching. This initial condition is then refined using a hierarchical robust feature matching (RFM) procedure. Experimental results reveal that the proposed registration technique offers a robust solution in the presence of seasonal changes and other interfering factors. Keywords-Automated image registration, multi-temporal imagery, mathematical morphology, robust feature matching

A Study on the Fingerprint Recognition Method Directional Feature Detection using Neural Networks

Fingerprint-based identification is known to be used for a very long time. Owing to their uniqueness and immutability, fingerprints are today the most widely used biometric features. Therefore, recognition using fingerprints is one of the safest methods as a way of personal identification. In this paper, a fingerprint identification method using neural networks and the direction feature vectors based on the directional image extracted from gray-scale fingerprint image without binarization and thinning is proposed. The basic idea of the above mentioned method is to track the ridge lines on the gray-scale image, by ？ailing according to the local orientation of the ridge pattern. A set of starting points are determined by superimposing a grid on the gray-scale image. A labeling strategy is adopted to examine each ridge line only once and locate the intersections between ridge lines. After the direction feature vectors are consisted of vectors by four direction labeling. Matching method used in this paper is four direction feature vectors based matching. The experiment are used total 124 feature patterns of four fingerprints, and One fingerprint image is consisted of 31 feature patterns. The results is presented excellent recognition capability of learned fingerprint images.Abstract(Korean) = 2 Abstract(English) = 3 Chapter 1 Introduction = 4 Chapter 2 Neural networks = 6 2.1 Introduction of neural networks = 6 2.2 Investigation between biological and artificial neuron = 7 2.3 Learning and structure of multilayer neural network = 10 2.4 Multilayered neural networks used experimental = 14 Chapter 3 Fingerprint recognition = 15 3.1 Direction feature vector detection = 15 3.2 Tangent direction computation = 18 3.3 Four direction labeling and pattern detection = 20 Chapter 4 Experimental results = 25 4.1 Experimental environment and method = 25 4.2 Experimental results = 29 Chapter 5 Conclusion = 40 References = 4

Image understanding and feature extraction for applications in industry and mapping

Bibliography: p. 212-220.The aim of digital photogrammetry is the automated extraction and classification of the three dimensional information of a scene from a number of images. Existing photogrammetric systems are semi-automatic requiring manual editing and control, and have very limited domains of application so that image understanding capabilities are left to the user. Among the most important steps in a fully integrated system are the extraction of features suitable for matching, the establishment of the correspondence between matching points and object classification. The following study attempts to explore the applicability of pattern recognition concepts in conjunction with existing area-based methods, feature-based techniques and other approaches used in computer vision in order to increase the level of automation and as a general alternative and addition to existing methods. As an illustration of the pattern recognition approach examples of industrial applications are given. The underlying method is then extended to the identification of objects in aerial images of urban scenes and to the location of targets in close-range photogrammetric applications. Various moment-based techniques are considered as pattern classifiers including geometric invariant moments, Legendre moments, Zernike moments and pseudo-Zernike moments. Two-dimensional Fourier transforms are also considered as pattern classifiers. The suitability of these techniques is assessed. These are then applied as object locators and as feature extractors or interest operators. Additionally the use of fractal dimension to segment natural scenes for regional classification in order to limit the search space for particular objects is considered. The pattern recognition techniques require considerable preprocessing of images. The various image processing techniques required are explained where needed. Extracted feature points are matched using relaxation based techniques in conjunction with area-based methods to 'obtain subpixel accuracy. A subpixel pattern recognition based method is also proposed and an investigation into improved area-based subpixel matching methods is undertaken. An algorithm for determining relative orientation parameters incorporating the epipolar line constraint is investigated and compared with a standard relative orientation algorithm. In conclusion a basic system that can be automated based on some novel techniques in conjunction with existing methods is described and implemented in a mapping application. This system could be largely automated with suitably powerful computers

Modelling of head movement in expression of disgust

Head movement modelling can be seen as a part of facial expression study because some expressions like disgust involves head movement. Head movement information can be acquired by video recording process. The recording process has to deal with image distortion correctable via plumb-line method. Unfortunately the linear fitting used in plumb-line requires piecewise function. The thesis aims to enhance the plumb-line-based image distortion correction using conic function coefficient evaluation replacing linear fitting. Experiments conducted shows that the proposed method handles various line orientations without having to rely on piecewise function. Besides distortion correction, an approach for expression movement tracking is needed. Optical flow-template matching is one of the techniques used for tracking. However, existing search algorithms did not discuss much on the looping technique of template matching. Moreover, tracking transient features during expression requires special process as the feature exists intermittently. The thesis aims to enhance the optical flow-template matching-based tracking method for tracking feature points during head movement by controlling the search loop and introducing anchoring to handle transient components. Experiment showed that the proposed method recorded a reduction in comparison of 40.1% over another similar method during worse case scenario. Besides reduction, the proposed method also lowered the lost point during searching when compared with existing method. Head movement modelling is not given proper attention in facial expression study hence affecting head model believability in computer graphics. The thesis aims to design head movement quantification method for head movement during disgust expression. The quantification method tracks movements of the head inclusive of the neck and named as ‘Dual Pivot Head Tracking’ (DPHT). To prove that it is perceptually better to use the proposed method, a perceptual study of expression with and without head movement was conducted. Results showed that subjects perceived disgust expression better if the proposed method is used ( -score of neck given head=14.9 vs. head given neck=3.59). To further support our proposal on the need to track head movement inclusive of the neck, experiments tracking subjects depicting disgust were conducted. A statistical two-tailed test to evaluate the existence of neck motion during head movement was done. Furthermore, visual comparison was made with a model without head movement approach. Results showed that neck motion was presence during head movement of disgust (z-score = 3.4 with p-value = 0.0006). Similarly the visual depictions showed that without the head movement inclusive of neck the rendering seemed to be incomplete. Having movement information, the thesis aims to design a temporal model of head movement during disgust expression. Neck motion, a part of head motion, plays a role during disgust expression. The thesis proposes spline-based function named Joint Cubic Bezier (JCB) to model neck motion during disgust. Experiments showed that using JCB, analysis and synthesis of neck motion during disgust expression is better than via cosine and exponential approach with angular separation score of JCB=0.986041, Exponential=0.897163 and Cosine=0.90773

Robust Character Recognition in Low-Resolution Images and Videos

Although OCR techniques work very reliably for high-resolution documents, the recognition of superimposed text in low-resolution images or videos with a complex background is still a challenge. Three major parts characterize our system for recognition of superimposed text in images and videos: localization of text regions, segmentation (binarization) of characters, and recognition. We use standard approaches to locate text regions and focus in this paper on the last two steps. Many approaches (e.g., projection profiles, k-mean clustering) do not work very well for separating characters with very small font sizes. We apply in a vertical direction a shortest-path algorithm to separate the characters in a text line. The recognition of characters is based on the curvature scale space (CSS) approach which smoothes the contour of a character with a Gaussian kernel and tracks its inflection points. A major drawback of the CSS method is its poor representation of convex segments: Convex objects cannot be represented at all due to missing inflection points. We have extended the CSS approach to generate feature points for concave and convex segments of a contour. This generic approach is not only applicable to text characters but to arbitrary objects as well. In the experimental results, we compare our approach against a pattern matching algorithm, two classification algorithms based on contour analysis, and a commercial OCR system. The overall recognition results are good enough even for the indexing of low resolution images and videos

Enhancment of dense urban digital surface models from VHR optical satellite stereo data by pre-segmentation and object detection

The generation of digital surface models (DSM) of urban areas from very high resolution (VHR) stereo satellite imagery requires advanced methods. In the classical approach of DSM generation from stereo satellite imagery, interest points are extracted and correlated between the stereo mates using an area based matching followed by a least-squares sub-pixel refinement step. After a region growing the 3D point list is triangulated to the resulting DSM. In urban areas this approach fails due to the size of the correlation window, which smoothes out the usual steep edges of buildings. Also missing correlations as for partly – in one or both of the images – occluded areas will simply be interpolated in the triangulation step. So an urban DSM generated with the classical approach results in a very smooth DSM with missing steep walls, narrow streets and courtyards. To overcome these problems algorithms from computer vision are introduced and adopted to satellite imagery. These algorithms do not work using local optimisation like the area-based matching but try to optimize a (semi-)global cost function. Analysis shows that dynamic programming approaches based on epipolar images like dynamic line warping or semiglobal matching yield the best results according to accuracy and processing time. These algorithms can also detect occlusions – areas not visible in one or both of the stereo images. Beside these also the time and memory consuming step of handling and triangulating large point lists can be omitted due to the direct operation on epipolar images and direct generation of a so called disparity image fitting exactly on the first of the stereo images. This disparity image – representing already a sort of a dense DSM – contains the distances measured in pixels in the epipolar direction (or a no-data value for a detected occlusion) for each pixel in the image. Despite the global optimization of the cost function many outliers, mismatches and erroneously detected occlusions remain, especially if only one stereo pair is available. To enhance these dense DSM – the disparity image – a pre-segmentation approach is presented in this paper. Since the disparity image is fitting exactly on the first of the two stereo partners (beforehand transformed to epipolar geometry) a direct correlation between image pixels and derived heights (the disparities) exist. This feature of the disparity image is exploited to integrate additional knowledge from the image into the DSM. This is done by segmenting the stereo image, transferring the segmentation information to the DSM and performing a statistical analysis on each of the created DSM segments. Based on this analysis and spectral information a coarse object detection and classification can be performed and in turn the DSM can be enhanced. After the description of the proposed method some results are shown and discussed