Variational methods and its applications to computer vision

Many computer vision applications such as image segmentation can be formulated in a ''variational'' way as energy minimization problems. Unfortunately, the computational task of minimizing these energies is usually difficult as it generally involves non convex functions in a space with thousands of dimensions and often the associated combinatorial problems are NP-hard to solve. Furthermore, they are ill-posed inverse problems and therefore are extremely sensitive to perturbations (e.g. noise). For this reason in order to compute a physically reliable approximation from given noisy data, it is necessary to incorporate into the mathematical model appropriate regularizations that require complex computations. The main aim of this work is to describe variational segmentation methods that are particularly effective for curvilinear structures. Due to their complex geometry, classical regularization techniques cannot be adopted because they lead to the loss of most of low contrasted details. In contrast, the proposed method not only better preserves curvilinear structures, but also reconnects some parts that may have been disconnected by noise. Moreover, it can be easily extensible to graphs and successfully applied to different types of data such as medical imagery (i.e. vessels, hearth coronaries etc), material samples (i.e. concrete) and satellite signals (i.e. streets, rivers etc.). In particular, we will show results and performances about an implementation targeting new generation of High Performance Computing (HPC) architectures where different types of coprocessors cooperate. The involved dataset consists of approximately 200 images of cracks, captured in three different tunnels by a robotic machine designed for the European ROBO-SPECT project.Open Acces

Advances in Simultaneous Localization and Mapping in Confined Underwater Environments Using Sonar and Optical Imaging.

This thesis reports on the incorporation of surface information into a probabilistic simultaneous localization and mapping (SLAM) framework used on an autonomous underwater vehicle (AUV) designed for underwater inspection. AUVs operating in cluttered underwater environments, such as ship hulls or dams, are commonly equipped with Doppler-based sensors, which---in addition to navigation---provide a sparse representation of the environment in the form of a three-dimensional (3D) point cloud. The goal of this thesis is to develop perceptual algorithms that take full advantage of these sparse observations for correcting navigational drift and building a model of the environment. In particular, we focus on three objectives. First, we introduce a novel representation of this 3D point cloud as collections of planar features arranged in a factor graph. This factor graph representation probabalistically infers the spatial arrangement of each planar segment and can effectively model smooth surfaces (such as a ship hull). Second, we show how this technique can produce 3D models that serve as input to our pipeline that produces the first-ever 3D photomosaics using a two-dimensional (2D) imaging sonar. Finally, we propose a model-assisted bundle adjustment (BA) framework that allows for robust registration between surfaces observed from a Doppler sensor and visual features detected from optical images. Throughout this thesis, we show methods that produce 3D photomosaics using a combination of triangular meshes (derived from our SLAM framework or given a-priori), optical images, and sonar images. Overall, the contributions of this thesis greatly increase the accuracy, reliability, and utility of in-water ship hull inspection with AUVs despite the challenges they face in underwater environments. We provide results using the Hovering Autonomous Underwater Vehicle (HAUV) for autonomous ship hull inspection, which serves as the primary testbed for the algorithms presented in this thesis. The sensor payload of the HAUV consists primarily of: a Doppler velocity log (DVL) for underwater navigation and ranging, monocular and stereo cameras, and---for some applications---an imaging sonar.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120750/1/paulozog_1.pd

Machine vision techniques for inspection of dry-fibre composite preforms in the aerospace industry

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis presents the results of a three year investigation into machine vision techniques for in-process automated inspection of dry-fibre composite preforms. Efficient texture analysis based techniques have been developed, tested, and implemented in a prototype robotic assembly cell. Industrial constraints have been considered in the development of all the algorithms described. A single channel texture analysis model is described which can successfully segment images containing only a few textures. The model is based on convolution of the image with small kernels optimised for the task, and is elegant in the sense that it is computationally simple and easily realisable in low cost hardware. A new convolution kernel optimisation algorithm is described. It is demonstrated that convolution kernels can also be optimised to perform as edge operators in simple textured images. A novel boundary refinement algorithm is described which reduces the inspection errors inherent in texture based boundary estimates. The algorithm takes the form of a local search, using the texture estimate as a guiding template, and selects edge points by maximising a merit function. Optimum parameters for the merit function are obtained using multiple training images in conjunction with simple function optimisation algorithms.This study is funded by the Engineering and Physical Sciences Research Council (EPSRC) and Dowty Aerospace Propellers Ltd

Artificial vision by thermography : calving prediction and defect detection in carbon fiber reinforced polymer

La vision par ordinateur est un domaine qui consiste à extraire ou identifier une ou plusieurs informations à partir d’une ou plusieurs images dans le but soit d’automatiser une tache, soit de fournir une aide à la décision. Avec l’augmentation de la capacité de calcul des ordinateurs, la vulgarisation et la diversification des moyens d’imagerie tant dans la vie quotidienne, que dans le milieu industriel,ce domaine a subi une évolution rapide lors de dernières décennies. Parmi les différentes modalités d’imagerie pour lesquels il est possible d’utiliser la vision artificielle cette thèse se concentre sur l’imagerie infrarouge. Plus particulièrement sur l’imagerie infrarouge pour les longueurs d’ondes comprises dans les bandes moyennes et longues. Cette thèse se porte sur deux applications industrielles radicalement différentes. Dans la première partie de cette thèse, nous présentons une application de la vision artificielle pour la détection du moment de vêlage en milieux industriel pour des vaches Holstein. Plus précisément l’objectif de cette recherche est de déterminer le moment de vêlage en n’utilisant que des données comportementales de l’animal. À cette fin, nous avons acquis des données en continu sur différents animaux pendant plusieurs mois. Parmi les nombreux défis présentés par cette application l’un d’entre eux concerne l’acquisition des données. En effet, les caméras que nous avons utilisées sont basées sur des capteurs bolométriques, lesquels sont sensibles à un grand nombre de variables. Ces variables peuvent être classées en quatre catégories : intrinsèque, environnemental, radiométrique et géométrique. Un autre défit important de cette recherche concerne le traitement des données. Outre le fait que les données acquises utilisent une dynamique plus élevée que les images naturelles ce qui complique le traitement des données ; l’identification de schéma récurrent dans les images et la reconnaissance automatique de ces derniers grâce à l’apprentissage automatique représente aussi un défi majeur. Nous avons proposé une solution à ce problème. Dans le reste de cette thèse nous nous sommes penchés sur la problématique de la détection de défaut dans les matériaux, en utilisant la technique de la thermographie pulsée. La thermographie pulsée est une méthode très populaire grâce à sa simplicité, la possibilité d’être utilisée avec un grand nombre de matériaux, ainsi que son faible coût. Néanmoins, cette méthode est connue pour produire des données bruitées. La cause principale de cette réputation vient des diverses sources de distorsion auquel les cameras thermiques sont sensibles. Dans cette thèse, nous avons choisi d’explorer deux axes. Le premier concerne l’amélioration des méthodes de traitement de données existantes. Dans le second axe, nous proposons plusieurs méthodes pour améliorer la détection de défauts. Chaque méthode est comparée à plusieurs méthodes constituant l’état de l’art du domaine.Abstract Computer vision is a field which consists in extracting or identifying one or more information from one or more images in order either to automate a task or to provide decision support. With the increase in the computing capacity of computers, the popularization and diversification of imaging means, both in industry, as well as in everyone’s life, this field has undergone a rapid development in recent decades. Among the different imaging modalities for which it is possible to use artificial vision, this thesis focuses on infrared imaging. More particularly on infrared imagery for wavelengths included in the medium and long bands. This thesis focuses on two radically different industrial applications. In the first part of this thesis, we present an application of artificial vision for the detection of the calving moment in industrial environments for Holstein cows. More precisely, the objective of this research is to determine the time of calving using only physiological data from the animal. To this end, we continuously acquired data on different animals over several days. Among the many challenges presented by this application, one of them concerns data acquisition. Indeed, the cameras we used are based on bolometric sensors, which are sensitive to a large number of variables. These variables can be classified into four categories: intrinsic, environmental, radiometric and geometric. Another important challenge in this research concerns the processing of data. Besides the fact that the acquired data uses a higher dynamic range than the natural images which complicates the processing of the data; Identifying recurring patterns in images and automatically recognizing them through machine learning is a major challenge. We have proposed a solution to this problem. In the rest of this thesis we have focused on the problem of defect detection in materials, using the technique of pulsed thermography. Pulse thermography is a very popular method due toits simplicity, the possibility of being used with a large number of materials, as well as its low cost. However, this method is known to produce noisy data. The main cause of this reputation comes from the various sources of distortion to which thermal cameras are sensitive. In this thesis, we have chosen to explore two axes. The first concerns the improvement of existing data processing methods. In the second axis, we propose several methods to improve fault detection. Each method is compared to several methods constituting the state of the art in the field

Robust techniques and applications in fuzzy clustering

This dissertation addresses issues central to frizzy classification. The issue of sensitivity to noise and outliers of least squares minimization based clustering techniques, such as Fuzzy c-Means (FCM) and its variants is addressed. In this work, two novel and robust clustering schemes are presented and analyzed in detail. They approach the problem of robustness from different perspectives. The first scheme scales down the FCM memberships of data points based on the distance of the points from the cluster centers. Scaling done on outliers reduces their membership in true clusters. This scheme, known as the Mega-clustering, defines a conceptual mega-cluster which is a collective cluster of all data points but views outliers and good points differently (as opposed to the concept of Dave\u27s Noise cluster). The scheme is presented and validated with experiments and similarities with Noise Clustering (NC) are also presented. The other scheme is based on the feasible solution algorithm that implements the Least Trimmed Squares (LTS) estimator. The LTS estimator is known to be resistant to noise and has a high breakdown point. The feasible solution approach also guarantees convergence of the solution set to a global optima. Experiments show the practicability of the proposed schemes in terms of computational requirements and in the attractiveness of their simplistic frameworks. The issue of validation of clustering results has often received less attention than clustering itself. Fuzzy and non-fuzzy cluster validation schemes are reviewed and a novel methodology for cluster validity using a test for random position hypothesis is developed. The random position hypothesis is tested against an alternative clustered hypothesis on every cluster produced by the partitioning algorithm. The Hopkins statistic is used as a basis to accept or reject the random position hypothesis, which is also the null hypothesis in this case. The Hopkins statistic is known to be a fair estimator of randomness in a data set. The concept is borrowed from the clustering tendency domain and its applicability to validating clusters is shown here. A unique feature selection procedure for use with large molecular conformational datasets with high dimensionality is also developed. The intelligent feature extraction scheme not only helps in reducing dimensionality of the feature space but also helps in eliminating contentious issues such as the ones associated with labeling of symmetric atoms in the molecule. The feature vector is converted to a proximity matrix, and is used as an input to the relational fuzzy clustering (FRC) algorithm with very promising results. Results are also validated using several cluster validity measures from literature. Another application of fuzzy clustering considered here is image segmentation. Image analysis on extremely noisy images is carried out as a precursor to the development of an automated real time condition state monitoring system for underground pipelines. A two-stage FCM with intelligent feature selection is implemented as the segmentation procedure and results on a test image are presented. A conceptual framework for automated condition state assessment is also developed

Condition Assessment of Concrete Bridge Decks Using Ground and Airborne Infrared Thermography

Applications of nondestructive testing (NDT) technologies have shown promise in assessing the condition of existing concrete bridges. Infrared thermography (IRT) has gradually gained wider acceptance as a NDT and evaluation tool in the civil engineering field. The high capability of IRT in detecting subsurface delamination, commercial availability of infrared cameras, lower cost compared with other technologies, speed of data collection, and remote sensing are some of the expected benefits of applying this technique in bridge deck inspection practices. The research conducted in this thesis aims at developing a rational condition assessment system for concrete bridge decks based on IRT technology, and automating its analysis process in order to add this invaluable technique to the bridge inspector’s tool box. Ground penetrating radar (GPR) has also been vastly recognized as a NDT technique capable of evaluating the potential of active corrosion. Therefore, integrating IRT and GPR results in this research provides more precise assessments of bridge deck conditions. In addition, the research aims to establish a unique link between NDT technologies and inspector findings by developing a novel bridge deck condition rating index (BDCI). The proposed procedure captures the integrated results of IRT and GPR techniques, along with visual inspection judgements, thus overcoming the inherent scientific uncertainties of this process. Finally, the research aims to explore the potential application of unmanned aerial vehicle (UAV) infrared thermography for detecting hidden defects in concrete bridge decks. The NDT work in this thesis was conducted on full-scale deteriorated reinforced concrete bridge decks located in Montreal, Quebec and London, Ontario. The proposed models have been validated through various case studies. IRT, either from the ground or by utilizing a UAV with high-resolution thermal infrared imagery, was found to be an appropriate technology for inspecting and precisely detecting subsurface anomalies in concrete bridge decks. The proposed analysis produced thermal mosaic maps from the individual IR images. The k-means clustering classification technique was utilized to segment the mosaics and identify objective thresholds and, hence, to delineate different categories of delamination severity in the entire bridge decks. The proposed integration methodology of NDT technologies and visual inspection results provided more reliable BDCI. The information that was sought to identify the parameters affecting the integration process was gathered from bridge engineers with extensive experience and intuition. The analysis process utilized the fuzzy set theory to account for uncertainties and imprecision in the measurements of bridge deck defects detected by IRT and GPR testing along with bridge inspector observations. The developed system and models should stimulate wider acceptance of IRT as a rapid, systematic and cost-effective evaluation technique for detecting bridge deck delaminations. The proposed combination of IRT and GPR results should expand their correlative use in bridge deck inspection. Integrating the proposed BDCI procedure with existing bridge management systems can provide a detailed and timely picture of bridge health, thus helping transportation agencies in identifying critical deficiencies at various service life stages. Consequently, this can yield sizeable reductions in bridge inspection costs, effective allocation of limited maintenance and repair funds, and promote the safety, mobility, longevity, and reliability of our highway transportation assets