Accurate 3D shape and displacement measurement using a scanning electron microscope

With the current development of nano-technology, there exists an increasing demand for three-dimensional shape and deformation measurements at this reduced-length scale in the field of materials research. Images acquired by \ud Scanning Electron Microscope (SEM) systems coupled with analysis by Digital Image Correlation (DIC) is an interesting combination for development of a high magnification measurement system. However, a SEM is designed for visualization, not for metrological studies, and the application of DIC to the micro- or nano-scale with such a system faces the challenges of calibrating the imaging system and correcting the spatially-varying and \ud time-varying distortions in order to obtain accurate measurements. Moreover, the SEM provides only a single sensor and recovering 3D information is not possible with the classical stereo-vision approach. But the specimen being mounted on the mobile SEM stage, images can be acquired from multiple viewpoints and 3D reconstruction is possible using the principle of videogrammetry for recovering the unknown rigid-body motions undergone by \ud the specimen.\ud The dissertation emphasizes the new calibration methodology that has been developed because it is a major contribution for the accuracy of 3D shape and deformation measurements at reduced-length scale. It proves that, unlike previous works, image drift and distortion must be taken into account if accurate measurements are to be made with such a system. Necessary background and required theoretical knowledge for the 3D shape measurement using videogrammetry and for in-plane and out-of-plane deformation measurement are presented in details as well. In order to validate our work and demonstrate in particular the obtained measurement accuracy, experimental results resulting from different applications are presented throughout the different chapters. At last, a software gathering different computer vision applications has been developed.\ud Avec le développement actuel des nano-technologies, la demande en matière d'étude du comportement des matériaux à des échelles micro ou nanoscopique ne cesse d'augmenter. Pour la mesure de forme ou de déformations tridimensionnelles à ces échelles de grandeur,l'acquisition d'images à partir d'un Microscope électronique à Balayage (MEB) couplée à l'analyse par corrélation d'images numériques s'est avérée une technique intéressante. \ud Cependant, un MEB est un outil conçu essentiellement pour de la visualisation et son utilisation pour des mesures tridimensionnelles précises pose un certain nombre de difficultés comme par exemple le calibrage du système et la \ud correction des fortes distorsions (spatiales et temporelles) présentes dans les images. De plus, le MEB ne possède qu'un seul capteur et les informations tridimensionnelles souhaitées ne peuvent pas être obtenues par une approche classique de type stéréovision. Cependant, l'échantillon à analyser étant monté sur un support orientable, des images peuvent être acquises sous différents points de vue, ce qui permet une reconstruction tridimensionnelle en utilisant le principe de vidéogrammétrie pour retrouver à partir des seules images les mouvements inconnus du porte-échantillon.\ud La thèse met l'accent sur la nouvelle technique de calibrage et de correction des distorsions développée car c'est une contribution majeure pour la précision de la mesure de forme et de déformations 3D aux échelles de \ud grandeur étudiées. Elle prouve que, contrairement aux travaux précédents, la prise en compte de la dérive temporelle et des distorsions spatiales d'images \ud est indispensable pour obtenir une précision de mesure suffisante. Les principes permettant la mesure de forme par vidéogrammétrie et le calcul de déformations 2D et 3D sont aussi présentés en détails. Enfin, et dans le but de valider nos travaux et démontrer en particulier la précision de mesure obtenue, des résultats expérimentaux issus de différentes applications sont présentés.\ud \ud \u

Unsupervised segmentation of road images. A multicriteria approach

This paper presents a region-based segmentation algorithm which can be applied to various problems since it does not requir e a priori knowledge concerning the kind of processed images . This algorithm, based on a split and merge method, gives reliable results both on homogeneous grey level images and on textured images . First, images are divided into rectangular sectors . The splitting algorithm works independently on each sector, and uses a homogeneity criterion based only on grey levels . The mergin g is then achieved through assigning labels to each region obtained by the splitting step, using extracted feature measurements . We modeled exploited fields (data field and label field) by Markov Random Fields (MRF), the segmentation is then optimall y determined using the Iterated Conditional Modes (ICM) . Input data of the merging step are regions obtained by the splitting step and their corresponding features vector. The originality of this algorithm is that texture coefficients are directly computed from these regions . These regions will be elementary sites for the Markov relaxation process . Thus, a region- based segmentation algorith m using texture and grey level is obtained . Results from various images types are presented .Nous présentons ici un algorithme de segmentation en régions pouvant s'appliquer à des problèmes très variés car il ne tient compte d'aucune information a priori sur le type d'images traitées. Il donne de bons résultats aussi bien sur des images possédant des objets homogènes au sens des niveaux de gris que sur des images possédant des régions texturées. C'est un algorithme de type division-fusion. Lors d'une première étape, l'image est découpée en fenêtres, selon une grille. L'algorithme de division travaille alors indépendamment sur chaque fenêtre, et utilise un critère d'homogénéité basé uniquement sur les niveaux de gris. La texture de chacune des régions ainsi obtenues est alors calculée. A chaque région sera associé un vecteur de caractéristiques comprenant des paramètres de luminance, et des paramètres de texture. Les régions ainsi définies jouent alors le rôle de sites élémentaires pour le processus de fusion. Celui-ci est fondé sur la modélisation des champs exploités (champ d'observations et champ d'étiquettes) par des champs de Markov. Nous montrerons les résultats de segmentation obtenus sur divers types d'images

Optical flow estimation using steered-L1 norm

Motion is a very important part of understanding the visual picture of the surrounding environment. In image processing it involves the estimation of displacements for image points in an image sequence. In this context dense optical flow estimation is concerned with the computation of pixel displacements in a sequence of images, therefore it has been used widely in the field of image processing and computer vision. A lot of research was dedicated to enable an accurate and fast motion computation in image sequences. Despite the recent advances in the computation of optical flow, there is still room for improvements and optical flow algorithms still suffer from several issues, such as motion discontinuities, occlusion handling, and robustness to illumination changes. This thesis includes an investigation for the topic of optical flow and its applications. It addresses several issues in the computation of dense optical flow and proposes solutions. Specifically, this thesis is divided into two main parts dedicated to address two main areas of interest in optical flow. In the first part, image registration using optical flow is investigated. Both local and global image registration has been used for image registration. An image registration based on an improved version of the combined Local-global method of optical flow computation is proposed. A bi-lateral filter was used in this optical flow method to improve the edge preserving performance. It is shown that image registration via this method gives more robust results compared to the local and the global optical flow methods previously investigated. The second part of this thesis encompasses the main contribution of this research which is an improved total variation L1 norm. A smoothness term is used in the optical flow energy function to regularise this function. The L1 is a plausible choice for such a term because of its performance in preserving edges, however this term is known to be isotropic and hence decreases the penalisation near motion boundaries in all directions. The proposed improved L1 (termed here as the steered-L1 norm) smoothness term demonstrates similar performance across motion boundaries but improves the penalisation performance along such boundaries

Analysis of motion in scale space

This work includes some new aspects of motion estimation by the optic flow method in scale spaces. The usual techniques for motion estimation are limited to the application of coarse to fine strategies. The coarse to fine strategies can be successful only if there is enough information in every scale. In this work we investigate the motion estimation in the scale space more basically. The wavelet choice for scale space decomposition of image sequences is discussed in the first part of this work. We make use of the continuous wavelet transform with rotationally symmetric wavelets. Bandpass decomposed sequences allow the replacement of the structure tensor by the phase invariant energy operator. The structure tensor is computationally more expensive because of its spatial or spatio-temporal averaging. The energy operator needs in general no further averaging. The numerical accuracy of the motion estimation with the energy operator is compared to the results of usual techniques, based on the structure tensor. The comparison tests are performed on synthetic and real life sequences. Another practical contribution is the accuracy measurement for motion estimation by adaptive smoothed tensor fields. The adaptive smoothing relies on nonlinear anisotropic diffusion with discontinuity and curvature preservation. We reached an accuracy gain under properly chosen parameters for the diffusion filter. A theoretical contribution from mathematical point of view is a new discontinuity and curvature preserving regularization for motion estimation. The convergence of solutions for the isotropic case of the nonlocal partial differential equation is shown. For large displacements between two consecutive frames the optic flow method is systematically corrupted because of the violence of the sampling theorem. We developed a new method for motion analysis by scale decomposition, which allows to circumvent the systematic corruption without using the coarse to fine strategy. The underlying assumption is, that in a certain neighborhood the grey value undergoes the same displacement. If this is fulfilled, then the same optic flow should be measured in all scales. If there arise inconsistencies in a pixel across the scale space, so they can be detected and the scales containing this inconsistencies are not taken into account

A blackboard-based system for learning to identify images from feature data

A blackboard-based system which learns recognition rules for objects from a set of training examples, and then identifies and locates these objects in test images, is presented. The system is designed to use data from a feature matcher developed at R.S.R.E. Malvern which finds the best matches for a set of feature patterns in an image. The feature patterns are selected to correspond to typical object parts which occur with relatively consistent spatial relationships and are sufficient to distinguish the objects to be identified from one another. The learning element of the system develops two separate sets of rules, one to identify possible object instances and the other to attach probabilities to them. The search for possible object instances is exhaustive; its scale is not great enough for pruning to be necessary. Separate probabilities are established empirically for all combinations of features which could represent object instances. As accurate probabilities cannot be obtained from a set of preselected training examples, they are updated by feedback from the recognition process. The incorporation of rule induction and feedback into the blackboard system is achieved by treating the induced rules as data to be held on a secondary blackboard. The single recognition knowledge source effectively contains empty rules which this data can be slotted into, allowing it to be used to recognise any number of objects - there is no need to develop a separate knowledge source for each object. Additional object-specific background information to aid identification can be added by the user in the form of background checks to be carried out on candidate objects. The system has been tested using synthetic data, and successfully identified combinations of geometric shapes (squares, triangles etc.). Limited tests on photographs of vehicles travelling along a main road were also performed successfully

An effective trajectory-based algorithm for ball detection and tracking with application to the analysis of broadcast sports video

Ph.DDOCTOR OF PHILOSOPH