Machine vision techniques for inspection of dry-fibre composite preforms in the aerospace industry

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis presents the results of a three year investigation into machine vision techniques for in-process automated inspection of dry-fibre composite preforms. Efficient texture analysis based techniques have been developed, tested, and implemented in a prototype robotic assembly cell. Industrial constraints have been considered in the development of all the algorithms described. A single channel texture analysis model is described which can successfully segment images containing only a few textures. The model is based on convolution of the image with small kernels optimised for the task, and is elegant in the sense that it is computationally simple and easily realisable in low cost hardware. A new convolution kernel optimisation algorithm is described. It is demonstrated that convolution kernels can also be optimised to perform as edge operators in simple textured images. A novel boundary refinement algorithm is described which reduces the inspection errors inherent in texture based boundary estimates. The algorithm takes the form of a local search, using the texture estimate as a guiding template, and selects edge points by maximising a merit function. Optimum parameters for the merit function are obtained using multiple training images in conjunction with simple function optimisation algorithms.This study is funded by the Engineering and Physical Sciences Research Council (EPSRC) and Dowty Aerospace Propellers Ltd

A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation

Recent work has shown that optical flow estimation can be formulated as a supervised learning task and can be successfully solved with convolutional networks. Training of the so-called FlowNet was enabled by a large synthetically generated dataset. The present paper extends the concept of optical flow estimation via convolutional networks to disparity and scene flow estimation. To this end, we propose three synthetic stereo video datasets with sufficient realism, variation, and size to successfully train large networks. Our datasets are the first large-scale datasets to enable training and evaluating scene flow methods. Besides the datasets, we present a convolutional network for real-time disparity estimation that provides state-of-the-art results. By combining a flow and disparity estimation network and training it jointly, we demonstrate the first scene flow estimation with a convolutional network.Comment: Includes supplementary materia

Image segmentation in the wavelet domain using N-cut framework

We introduce a wavelet domain image segmentation algorithm based on Normalized Cut (NCut) framework in this thesis. By employing the NCut algorithm we solve the perceptual grouping problem of image segmentation which aims at the extraction of the global impression of an image. We capitalize on the reduced set of data to be processed and statistical features derived from the wavelet-transformed images to solve graph partitioning more efficiently than before. Five orientation histograms are computed to evaluate similarity/dissimilarity measure of local structure. We use properties of the wavelet transform filtering to capture edge information in vertical, horizontal and diagonal orientations. This approach allows for direct processing of compressed data and results in faster implementation of NCut framework than that in the spatial domain and also decent quality of segmentation of natural scene images

Pixel classification methods for identifying and quantifying leaf surface injury from digital images

Plants exposed to stress due to pollution, disease or nutrient deficiency often develop visible symptoms on leaves such as spots, colour changes and necrotic regions. Early symptom detection is important for precision agriculture, environmental monitoring using bio-indicators and quality assessment of leafy vegetables. Leaf injury is usually assessed by visual inspection, which is labour-intensive and to a consid- erable extent subjective. In this study, methods for classifying individual pixels as healthy or injured from images of clover leaves exposed to the air pollutant ozone were tested and compared. RGB images of the leaves were acquired under controlled conditions in a laboratory using a standard digital SLR camera. Different feature vectors were extracted from the images by including different colour and texture (spa- tial) information. Four approaches to classification were evaluated: (1) Fit to a Pattern Multivariate Image Analysis (FPM) combined with T2 statistics (FPM-T2) or (2) Residual Sum of Squares statistics (FPM-RSS), (3) linear discriminant analysis (LDA) and (4) K-means clustering. The predicted leaf pixel classifications were trained from and compared to manually segmented images to evaluate classification performance. The LDA classifier outperformed the three other approaches in pixel identification with significantly higher accuracy, precision, true positive rate and F-score and significantly lower false positive rate and computation time. A feature vector of single pixel colour channel intensities was sufficient for capturing the information relevant for pixel identification. Including neighbourhood pixel information in the feature vector did not improve performance, but significantly increased the computation time. The LDA classifier was robust with 95% mean accuracy, 83% mean true positive rate and 2% mean false positive rate, indicating that it has potential for real-time applications.Opstad Kruse, OM.; Prats Montalbán, JM.; Indahl, UG.; Kvaal, K.; Ferrer Riquelme, AJ.; Futsaether, CM. (2014). Pixel classification methods for identifying and quantifying leaf surface injury from digital images. Computers and Electronics in Agriculture. 108:155-165. doi:10.1016/j.compag.2014.07.010S15516510

Making Tactile Textures with Predefined Affective Properties

A process for the design and manufacture of 3D tactile textures with predefined affective properties was developed. Twenty four tactile textures were manufactured. Texture measures from the domain of machine vision were used to characterize the digital representations of the tactile textures. To obtain affective ratings, the textures were touched, unseen, by 107 participants who scored them against natural, warm, elegant, rough, simple, and like, on a semantic differential scale. The texture measures were correlated with the participants' affective ratings using a novel feature subset evaluation method and a partial least squares genetic algorithm. Six measures were identified that are significantly correlated with human responses and are unlikely to have occurred by chance. Regression equations were used to select 48 new tactile textures that had been synthesized using mixing algorithms and which were likely to score highly against the six adjectives when touched by participants. The new textures were manufactured and rated by participants. It was found that the regression equations gave excellent predictive ability. The principal contribution of the work is the demonstration of a process, using machine vision methods and rapid prototyping, which can be used to make new tactile textures with predefined affective properties

Explaining Classifiers using Adversarial Perturbations on the Perceptual Ball

We present a simple regularization of adversarial perturbations based upon the perceptual loss. While the resulting perturbations remain imperceptible to the human eye, they differ from existing adversarial perturbations in that they are semi-sparse alterations that highlight objects and regions of interest while leaving the background unaltered. As a semantically meaningful adverse perturbations, it forms a bridge between counterfactual explanations and adversarial perturbations in the space of images. We evaluate our approach on several standard explainability benchmarks, namely, weak localization, insertion deletion, and the pointing game demonstrating that perceptually regularized counterfactuals are an effective explanation for image-based classifiers.Comment: CVPR 202