Deep CNN-Based Automated Optical Inspection for Aerospace Components

ABSTRACT The defect detection problem is of outmost importance in high-tech industries such as aerospace manufacturing and is widely employed using automated industrial quality control systems. In the aerospace manufacturing industry, composite materials are extensively applied as structural components in civilian and military aircraft. To ensure the quality of the product and high reliability, manual inspection and traditional automatic optical inspection have been employed to identify the defects throughout production and maintenance. These inspection techniques have several limitations such as tedious, time- consuming, inconsistent, subjective, labor intensive, expensive, etc. To make the operation effective and efficient, modern automated optical inspection needs to be preferred. In this dissertation work, automatic defect detection techniques are tested on three levels using a novel aerospace composite materials image dataset (ACMID). First, classical machine learning models, namely, Support Vector Machine and Random Forest, are employed for both datasets. Second, deep CNN-based models, such as improved ResNet50 and MobileNetV2 architectures are trained on ACMID datasets. Third, an efficient defect detection technique that combines the features of deep learning and classical machine learning model is proposed for ACMID dataset. To assess the aerospace composite components, all the models are trained and tested on ACMID datasets with distinct sizes. In addition, this work investigates the scenario when defective and non-defective samples are scarce and imbalanced. To overcome the problems of imbalanced and scarce datasets, oversampling techniques and data augmentation using improved deep convolutional generative adversarial networks (DCGAN) are considered. Furthermore, the proposed models are also validated using one of the benchmark steel surface defects (SSD) dataset

Visual Anomaly Detection via Dual-Attention Transformer and Discriminative Flow

In this paper, we introduce the novel state-of-the-art Dual-attention Transformer and Discriminative Flow (DADF) framework for visual anomaly detection. Based on only normal knowledge, visual anomaly detection has wide applications in industrial scenarios and has attracted significant attention. However, most existing methods fail to meet the requirements. In contrast, the proposed DTDF presents a new paradigm: it firstly leverages a pre-trained network to acquire multi-scale prior embeddings, followed by the development of a vision Transformer with dual attention mechanisms, namely self-attention and memorial-attention, to achieve two-level reconstruction for prior embeddings with the sequential and normality association. Additionally, we propose using normalizing flow to establish discriminative likelihood for the joint distribution of prior and reconstructions at each scale. The DADF achieves 98.3/98.4 of image/pixel AUROC on Mvtec AD; 83.7 of image AUROC and 67.4 of pixel sPRO on Mvtec LOCO AD benchmarks, demonstrating the effectiveness of our proposed approach.Comment: Submission to IEEE Transactions On Industrial Informatic

Automated Quantitative Analyses of Fatigue-Induced Surface Damage by Deep Learning

The digitization of materials is the prerequisite for accelerating product development. However, technologically, this is only beneficial when reliability is maintained. This requires comprehension of the microstructure-driven fatigue damage mechanisms across scales. A substantial fraction of the lifetime for high performance materials is attributed to surface damage accumulation at the microstructural scale (e.g., extrusions and micro crack formation). Although, its modeling is impeded by a lack of comprehensive understanding of the related mechanisms. This makes statistical validation at the same scale by micromechanical experimentation a fundamental requirement. Hence, a large quantity of processed experimental data, which can only be acquired by automated experiments and data analyses, is crucial. Surface damage evolution is often accessed by imaging and subsequent image post-processing. In this work, we evaluated deep learning (DL) methodologies for semantic segmentation and different image processing approaches for quantitative slip trace characterization. Due to limited annotated data, a U-Net architecture was utilized. Three data sets of damage locations observed in scanning electron microscope (SEM) images of ferritic steel, martensitic steel, and copper specimens were prepared. In order to allow the developed models to cope with material-specific damage morphology and imaging-induced variance, a customized augmentation pipeline for the input images was developed. Material domain generalizability of ferritic steel and conjunct material trained models were tested successfully. Multiple image processing routines to detect slip trace orientation (STO) from the DL segmented extrusion areas were implemented and assessed. In conclusion, generalization to multiple materials has been achieved for the DL methodology, suggesting that extending it well beyond fatigue damage is feasible

Automated Quality Control in Manufacturing Production Lines: A Robust Technique to Perform Product Quality Inspection

Quality control (QC) in manufacturing processes is critical to ensuring consumers receive products with proper functionality and reliability. Faulty products can lead to additional costs for the manufacturer and damage trust in a brand. A growing trend in QC is the use of machine vision (MV) systems because of their noncontact inspection, high repeatability, and efficiency. This thesis presents a robust MV system developed to perform comparative dimensional inspection on diversely shaped samples. Perimeter, area, rectangularity, and circularity are determined in the dimensional inspection algorithm for a base item and test items. A score determined with the four obtained parameter values provides the likeness between the base item and a test item. Additionally, a surface defect inspection is offered capable of identifying scratches, dents, and markings. The dimensional and surface inspections are used in a QC industrial case study. The case study examines the existing QC system for an electric motor manufacturer and proposes the developed QC system to increase product inspection count and efficiency while maintaining accuracy and reliability. Finally, the QC system is integrated in a simulated product inspection line consisting of a robotic arm and conveyor belts. The simulated product inspection line could identify the correct defect in all tested items and demonstrated the system’s automation capabilities

Usage of convolutional neural networks for identifying marine mammal individuals

Identifying marine mammals is a common practice performed by whale-watching crew members. Typically, an experienced marine ecologist is the one who can identify not just the taxa, but also the individual. This process is however always done in the aftermath of data sampling, where the goal is to use photo identification and match the dorsal fins of individuals spotted at the different spatio-temporal scales. This dissertation provides the pipeline and addresses the chal lenges in the usage of Convolutional Neural Networks (CNNs) to discriminate marine mammal individuals, in this case (pilot whales) based on the dorsal fins. The dissertation uses as input the 1138 images dataset containing over 856 individuals, and through three experiments addresses the issues when discriminating such a high number of classes. In the first experiment, the dissertation studies the role of synthetic data augmentation in boosting model performance. In second, the dissertation benchmarks the existing state-of-the-art convolutional neural network architectures. In third, the dissertation focuses on discriminating other features from dorsal fins to identify indi viduals (scratches, nicks, roundness, wideness). The dissertation outlines the issues and proposes the guidelines for the next effort in discriminating marine mammal individuals.A identificação de mamíferos marinhos é uma prática comum realizada pelos tripulantes de ob servação de baleias. Normalmente, o ecologista marinho experiente é aquele que pode identificar não apenas os táxons, mas também o indivíduo. Este processo, é no entanto feito sempre após a amostragem de dados, onde o objetivo é usar a identificação por foto e combinar as barbatanas dorsais dos indivíduos localizados nas diferentes escalas espaço-temporais. Esta dissertação fornece o pipeline e aborda os desafios do uso de Redes Neurais Convolucionais (CNNs) para discriminar indivíduos de mamíferos marinhos, neste caso (baleias-piloto) com base nas barbatanas dorsais. A dissertação usa como input um dataset de 1138 imagens que contêm 856 indivíduos, e através de três experiências aborda os problemas de discriminar um número tão elevado de classes. Na primeira experiência, a dissertação estuda o papel do aumento de dados sintéticos no melhora mento do desempenho do modelo. Na segunda experiência, a dissertação avalia arquiteturas de redes neurais convolucionais de última geração existentes. Na terceira experiência, a dissertação foca-se em discrimar outras caraterísticas das barbatanas dorsais para identificar indivíduos (ar ranhões, cortes, redondeza, amplitude). A dissertação descreve os problemas e propõe as diretrizes para o próximo esforço em discriminar indivíduos de mamíferos marinhos

Texture and Colour in Image Analysis

Research in colour and texture has experienced major changes in the last few years. This book presents some recent advances in the field, specifically in the theory and applications of colour texture analysis. This volume also features benchmarks, comparative evaluations and reviews

Toward Flare-Free Images: A Survey

Lens flare is a common image artifact that can significantly degrade image quality and affect the performance of computer vision systems due to a strong light source pointing at the camera. This survey provides a comprehensive overview of the multifaceted domain of lens flare, encompassing its underlying physics, influencing factors, types, and characteristics. It delves into the complex optics of flare formation, arising from factors like internal reflection, scattering, diffraction, and dispersion within the camera lens system. The diverse categories of flare are explored, including scattering, reflective, glare, orb, and starburst types. Key properties such as shape, color, and localization are analyzed. The numerous factors impacting flare appearance are discussed, spanning light source attributes, lens features, camera settings, and scene content. The survey extensively covers the wide range of methods proposed for flare removal, including hardware optimization strategies, classical image processing techniques, and learning-based methods using deep learning. It not only describes pioneering flare datasets created for training and evaluation purposes but also how they were created. Commonly employed performance metrics such as PSNR, SSIM, and LPIPS are explored. Challenges posed by flare's complex and data-dependent characteristics are highlighted. The survey provides insights into best practices, limitations, and promising future directions for flare removal research. Reviewing the state-of-the-art enables an in-depth understanding of the inherent complexities of the flare phenomenon and the capabilities of existing solutions. This can inform and inspire new innovations for handling lens flare artifacts and improving visual quality across various applications

Crack Analyser: a novel image-based NDT approach for measuring crack severity ​

openIn Europa, le infrastrutture civili e di trasporto necessitano di una manutenzione efficace e proattiva per garantire il continuo funzionamento in sicurezza durante l'intero loro ciclo di vita. I paesi europei devono ogni anno stanziare enormi risorse per mantenere il loro livello di funzionalità. Ciò fa sorgere la necessità urgente di adottare approcci di ispezione di monitoraggio più rapidi e affidabili per aiutare ad affrontare questi problemi. Il deterioramento delle strutture è più spesso anticipato dalla formazione di fessure sulla superficie del calcestruzzo. La presenza di fessurazioni può essere sintomo di diverse problematiche quali dilatazioni e ritiri dovuti a sbalzi di temperatura, assestamenti della struttura, copertura impropria fornita in fase di getto, corrosione delle armature in acciaio, carichi pesanti applicati, vibrazioni insufficienti al momento della posa del calcestruzzo o perdite d'acqua per ritiro superficiale del calcestruzzo. Diventa quindi di primaria importanza l'identificazione, la misurazione e il monitoraggio delle fessurazioni sulla superficie del calcestruzzo. I principali metodi di ispezione attualmente adottati si basano su strumenti manuali e righelli: un’attività lunga e ingombrante, soggetta a errori e scarsamente oggettiva sull'analisi quantitativa perché fortemente dipendente dall'esperienza dell'operatore. Secondo la norma UNI EN 1992-1-1:2005, la larghezza massima delle fessure del calcestruzzo ammessa per una generica classe di rischio è di 0,3 mm. Per questo motivo, per misurare in modo accurato e affidabile la dimensione della fessura, è necessario l’impiego di strumenti di misura con caratteristiche metrologiche adeguate (es. precisione e accuratezza almeno un ordine inferiore al valore da misurare). In caso contrario, la severità della fessura potrebbe essere classificata erroneamente. Questo lavoro di tesi propone un nuovo approccio automatico, basato su immagini, in grado di localizzare e misurare fessure su superfici in calcestruzzo rispettando il vincolo metrologico imposto dalla norma UNI EN 1992-1-1:2005. Utilizzando una sola immagine, il metodo sviluppato è in grado di localizzare e misurare automaticamente e rapidamente la larghezza e la lunghezza di una fessura su una superficie. Il sistema di misura sviluppato sfrutta una singola telecamera operante nel campo del visibile per acquisire un'immagine digitalizzata della superficie da ispezionare. Il componente software del sistema riceve in input la singola immagine che inquadra la crepa e fornisce in output un'immagine aumentata dove viene evidenziata la crepa e la sua larghezza e lunghezza media/max. La misura della larghezza della fessura viene eseguita perpendicolarmente alla linea centrale della fessura con una precisione sub-pixel. Il sistema di misurazione è stato implementato su uno smartphone per eseguire ispezioni manuali da parte dell'operatore e su sistemi integrati per l'ispezione remota con robot o velivoli senza pilota (UAV)). Le strategie sviluppate possono essere facilmente estese a qualsiasi altro contesto in cui sia richiesto un controllo di qualità superficiale mirato all'identificazione e misura di eventuali danni o difettosità. ​Europe’s ageing transport infrastructure needs effective and proactive maintenance in order to continue its safe operation during the entire life cycle; European countries have to allocate huge resources for maintaining their service-ability level. This give rise to the necessity of an urgent need to adopt faster and more reliable monitoring inspection approaches to help tackling these issues. The deterioration of structures is most often foreseen by the formation of cracks on concrete surface. The presence of cracks can be a symptom of various problems like expansion and shrinks due to temperature differences, settlement of the structure, improper cover provided during concreting, corrosion of reinforcement steel, heavy load applied, insufficient vibration at the time of laying the concrete or loss of water from concrete surface shrinkage, therefore the identification, measurement and monitoring of cracks on the concrete surface becomes of primary importance. The main currently adopted inspection methods rely on visual marking and rulers, long and cumbersome activity, prone to errors and poorly objective on quantitative analysis because it strongly depends on operator experience. According to UNI EN 1992-1-1:2005 standard , the maximum admitted concrete crack width is 0.3 mm. For this reason, to accurately and reliably measure the target dimension, it is necessary to employ measurement instruments with suitable metrological characteristics (e.g. precision and accuracy at least one order lower than the value to be measured). Otherwise, the crack severity could be misclassified. This thesis work proposes a novel automatic image-based approach able to locate and measure cracks on concrete surfaces respecting the metrological constraint imposed by UNI EN 1992-1-1:2005 standard. Using only one image, the developed method is able to automatically and rapidly locate and measure the average width and length of a crack in an existing concrete structure. The measurement system developed exploits a single camera working in the visible range to acquire a digitized image of the structure being inspected. The software component of the system receives as input the single image framing the crack and gives as output an augmented image where the crack is highlighted as well as its average/max width and length. The measure of the crack width is performed perpendicularly to the crack central line with sub-pixel accuracy. The measurement system has been deployed on a smartphone for operator-based manual inspections as well on embedded systems for remote inspection with robots or Unmanned Aerial Vehicles (UAVs). The strategies developed can be easily extended from concrete inspection applications to any other context where a surface quality control targeted to the identification of eventual damages/defects is required. The activity was triggered by an explicit need within the EnDurCrete project. ​INGEGNERIA INDUSTRIALEembargoed_20220321Giulietti, Nicol