Three dimensional reconstruction of the cell cytoskeleton from stereo images

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 1998.Includes bibliographical references (leaves 80-83).Besides its primary application to robot vision, stereo vision also appears promising in the biomedical field. This study examines 3D reconstruction of the cell cytoskeleton. This application of stereo vision to electron micrographs extracts information about the interior structure of cells at the nanometer scale level. We propose two different types of stereo vision approaches: the line-segment and wavelet multiresolution methods. The former is primitive-based and the latter is a point-based approach. Structural information is stressed in both methods. Directional representation is employed to provide an ideal description for filament-type structures. In the line-segment method, line-segments are first extracted from directional representation and then matching is conducted between two line-segment sets of stereo images. A new search algorithm, matrix matching, is proposed to determine the matching globally. In the wavelet multiresolution method, a pyramidal architecture is presented. Bottom-up analysis is first performed to form two pyramids, containing wavelet decompositions and directional representations. Subsequently, top-down matching is carried out. Matching at a high level provides guidance and constraints to the matching at a lower level. Our reconstructed results reveal 3D structure and the relationships of filaments which are otherwise hard to see in the original stereo images. The method is sufficiently robust and accurate to allow the automated analysis of cell structural characteristics from electron microscopy pairs. The method may also have application to a general class of stereo images.by Yuan Cheng.S.M

Model-free Dense Stereo Reconstruction Creating Realistic 3D City Models

In this paper we describe a framework for fully automatic and model-free generation of accurate and realistic 3D city models using multiple overlapping aerial images. The underlying DSM is computed by dense image matching, using a robustified Census transform as cost function. To further reduce the noise of mismatches, we afterwards minimize a global energy functional incorporating local smoothness constraints using variational methods. Due to the convexity of the framed problem, the solution is guaranteed to converge towards the global energy minimum and can be efficiently implemented on GPU using primal-dual algorithms. The resulting point cloud is then being triangulated, local planarity constraints are exploited to reduce the number of vertices and finally a multi-view texturing is applied. The quality of the DSM and the 3D Model is evaluated on a complex urban environment, using reference data generated by laser scanning (LiDAR)

Image understanding and feature extraction for applications in industry and mapping

Bibliography: p. 212-220.The aim of digital photogrammetry is the automated extraction and classification of the three dimensional information of a scene from a number of images. Existing photogrammetric systems are semi-automatic requiring manual editing and control, and have very limited domains of application so that image understanding capabilities are left to the user. Among the most important steps in a fully integrated system are the extraction of features suitable for matching, the establishment of the correspondence between matching points and object classification. The following study attempts to explore the applicability of pattern recognition concepts in conjunction with existing area-based methods, feature-based techniques and other approaches used in computer vision in order to increase the level of automation and as a general alternative and addition to existing methods. As an illustration of the pattern recognition approach examples of industrial applications are given. The underlying method is then extended to the identification of objects in aerial images of urban scenes and to the location of targets in close-range photogrammetric applications. Various moment-based techniques are considered as pattern classifiers including geometric invariant moments, Legendre moments, Zernike moments and pseudo-Zernike moments. Two-dimensional Fourier transforms are also considered as pattern classifiers. The suitability of these techniques is assessed. These are then applied as object locators and as feature extractors or interest operators. Additionally the use of fractal dimension to segment natural scenes for regional classification in order to limit the search space for particular objects is considered. The pattern recognition techniques require considerable preprocessing of images. The various image processing techniques required are explained where needed. Extracted feature points are matched using relaxation based techniques in conjunction with area-based methods to 'obtain subpixel accuracy. A subpixel pattern recognition based method is also proposed and an investigation into improved area-based subpixel matching methods is undertaken. An algorithm for determining relative orientation parameters incorporating the epipolar line constraint is investigated and compared with a standard relative orientation algorithm. In conclusion a basic system that can be automated based on some novel techniques in conjunction with existing methods is described and implemented in a mapping application. This system could be largely automated with suitably powerful computers

Automatic face recognition using stereo images

Face recognition is an important pattern recognition problem, in the study of both natural and artificial learning problems. Compaxed to other biometrics, it is non-intrusive, non- invasive and requires no paxticipation from the subjects. As a result, it has many applications varying from human-computer-interaction to access control and law-enforcement to crowd surveillance. In typical optical image based face recognition systems, the systematic vaxiability arising from representing the three-dimensional (3D) shape of a face by a two-dimensional (21)) illumination intensity matrix is treated as random vaxiability. Multiple examples of the face displaying vaxying pose and expressions axe captured in different imaging conditions. The imaging environment, pose and expressions are strictly controlled and the images undergo rigorous normalisation and pre-processing. This may be implemented in a paxtially or a fully automated system. Although these systems report high classification accuracies (>90%), they lack versatility and tend to fail when deployed outside laboratory conditions. Recently, more sophisticated 3D face recognition systems haxnessing the depth information have emerged. These systems usually employ specialist equipment such as laser scanners and structured light projectors. Although more accurate than 2D optical image based recognition, these systems are equally difficult to implement in a non-co-operative environment. Existing face recognition systems, both 2D and 3D, detract from the main advantages of face recognition and fail to fully exploit its non-intrusive capacity. This is either because they rely too much on subject co-operation, which is not always available, or because they cannot cope with noisy data. The main objective of this work was to investigate the role of depth information in face recognition in a noisy environment. A stereo-based system, inspired by the human binocular vision, was devised using a pair of manually calibrated digital off-the-shelf cameras in a stereo setup to compute depth information. Depth values extracted from 2D intensity images using stereoscopy are extremely noisy, and as a result this approach for face recognition is rare. This was cofirmed by the results of our experimental work. Noise in the set of correspondences, camera calibration and triangulation led to inaccurate depth reconstruction, which in turn led to poor classifier accuracy for both 3D surface matching and 211) 2 depth maps. Recognition experiments axe performed on the Sheffield Dataset, consisting 692 images of 22 individuals with varying pose, illumination and expressions

Dense urban elevation models from stereo images by an affine region merging approach

The main subject of this thesis is the computation of Dense Disparity Maps from a pair of satelite or aerial stereo images from an urban scene, taken from two different viewpoints. Several steps are needed to obtain the ?nal disparity map from the pair of images. We focus here on one of these steps: how to match the points in one image with the points in the other one. This matching process is closely related to the computation of the altitudes of the objects present in the scene. Indeed, the precision we can obtain in these altitude values is directly proportional to the precision in the matching process. This precision in the altitude is also inversely proportional to the distance between both viewpoints where the images are taken(baseline). The matching process is a widely studied ?eld in the Computer Vision Community and several methods and algorithms have been developed so far ([31, 27, 49]). Most of them consider a big base- line con?guration, which increases the performance in the altitude and also simpli?es the matching process. However, this assumption presents a major drawback with objects that are occluded in one image but appear in the other one. The bigger the baseline is, the more objects are occluded in one image and are not occluded in the other one. Recently, a different approach in which the images are taken with a very small baseline started to be analyzed ([19, 20]). This approach has the advantage of eliminating most of the ambiguities presented when one object occluded in one image is not occluded in the other one. Indeed, if we consider that we have a very small baseline, the occlusions presented in both images are almost the same. Now, this con?guration obviously decreases the precision in the ?nal altitude. In order to continue obtaining highly accurate altitude values, the precision in the matching process must be im- proved. The methods developed so far which consider the small baseline approach, compute altitude values with a high precision at some points, but leave the rest of them with no altitude values at all, generating a non-dense disparity map. Based on the fact that piecewise-a?ne models are reasonable for the elevation in urban areas, we propose a new method to interpolate and denoise those non-dense disparity maps. Under lambertian illumination hypothesis 1 , it is reasonable to assume that homogeneous regions in the graylevel image, correspond to the same a?ne elevation model. In other words, the borders between the piecewise a?ne elevation model are included to a large extent within contrasted graylevel borders. Hence, it is reasonable to look for an piecewise a?ne ?t to the elevation model where the borders between regions are taken from a graylevel segmenation of the image We present a region-merging algorithm that starts with an over-segmentation of the gray-level im- age. The disparity values at each region are approximated by an a?ne model, and a meaningfulness measure of the ?t is assigned to each of them. Using this meaningfulness as a merging order, the method iterates until no new merge is possible, according to a merging criterion which is also based on the meaningfulness of each pair of neighboring regions. In the last step, the algorithm performs a validation of the ?nal regions using again the meaningfulness of the ?t. The regions validated in this last step are those for which the a?ne model is a good approximation. The region-merging algorithm presented in this work can be seen as an attempt to incorporate a semantical meaning to real scenes: we have developed a validation method to determine whether the data within a region is well approximated by an a?ne model or not. Hence, we could analyze more complex models, de?ning a suitable validation criterion for each of them. In this way, we can search for the model that best explains a given data set in terms of its meaningfulness

Miniaturized embedded stereo vision system (MESVS)

Stereo vision is one of the fundamental problems of computer vision. It is also one of the oldest and heavily investigated areas of 3D vision. Recent advances of stereo matching methodologies and availability of high performance and efficient algorithms along with availability of fast and affordable hardware technology, have allowed researchers to develop several stereo vision systems capable of operating at real-time. Although a multitude of such systems exist in the literature, the majority of them concentrates only on raw performance and quality rather than factors such as dimension, and power requirement, which are of significant importance in the embedded settings. In this thesis a new miniaturized embedded stereo vision system (MESVS) is presented, which is miniaturized to fit within a package of 5x5cm, is power efficient, and cost-effective. Furthermore, through application of embedded programming techniques and careful optimization, MESVS achieves the real-time performance of 20 frames per second. This work discusses the various challenges involved regarding design and implementation of this system and the measures taken to tackle them