Simulation-Based Data Augmentation for the Quality Inspection of Structural Adhesive with Deep Learning

UIDB/00066/2020 POCI-01-0247-FEDER-034072The advent of Industry 4.0 has shown the tremendous transformative potential of combining artificial intelligence, cyber-physical systems and Internet of Things concepts in industrial settings. Despite this, data availability is still a major roadblock for the successful adoption of data-driven solutions, particularly concerning deep learning approaches in manufacturing. Specifically in the quality control domain, annotated defect data can often be costly, time-consuming and inefficient to obtain, potentially compromising the viability of deep learning approaches due to data scarcity. In this context, we propose a novel method for generating annotated synthetic training data for automated quality inspections of structural adhesive applications, validated in an industrial cell for automotive parts. Our approach greatly reduces the cost of training deep learning models for this task, while simultaneously improving their performance in a scarce manufacturing data context with imbalanced training sets by 3.1% ([email protected]). Additional results can be seen at https://ricardosperes.github.io/simulation-synth-adhesive/.publishersversionpublishe

Generative adversarial networks for data augmentation in structural adhesive inspection

UIDB/- 00066/2020 POCI-01-0247-FEDER-034072The technological advances brought forth by the Industry 4.0 paradigm have renewed the disruptive potential of artificial intelligence in the manufacturing sector, building the data-driven era on top of concepts such as Cyber-Physical Systems and the Internet of Things. However, data availability remains a major challenge for the success of these solutions, particularly concerning those based on deep learning approaches. Specifically in the quality inspection of structural adhesive applications, found commonly in the automotive domain, defect data with sufficient variety, volume and quality is generally costly, time-consuming and inefficient to obtain, jeopardizing the viability of such approaches due to data scarcity. To mitigate this, we propose a novel approach to generate synthetic training data for this application, leveraging recent breakthroughs in training generative adversarial networks with limited data to improve the performance of automated inspection methods based on deep learning, especially for imbalanced datasets. Preliminary results in a real automotive pilot cell show promise in this direction, with the approach being able to generate realistic adhesive bead images and consequently object detection models showing improved mean average precision at different thresholds when trained on the augmented dataset. For reproducibility purposes, the model weights, configurations and data encompassed in this study are made publicly available.publishersversionpublishe

Deep Learning in the Automotive Industry: Applications and Tools

Deep Learning refers to a set of machine learning techniques that utilize neural networks with many hidden layers for tasks, such as image classification, speech recognition, language understanding. Deep learning has been proven to be very effective in these domains and is pervasively used by many Internet services. In this paper, we describe different automotive uses cases for deep learning in particular in the domain of computer vision. We surveys the current state-of-the-art in libraries, tools and infrastructures (e.\,g.\ GPUs and clouds) for implementing, training and deploying deep neural networks. We particularly focus on convolutional neural networks and computer vision use cases, such as the visual inspection process in manufacturing plants and the analysis of social media data. To train neural networks, curated and labeled datasets are essential. In particular, both the availability and scope of such datasets is typically very limited. A main contribution of this paper is the creation of an automotive dataset, that allows us to learn and automatically recognize different vehicle properties. We describe an end-to-end deep learning application utilizing a mobile app for data collection and process support, and an Amazon-based cloud backend for storage and training. For training we evaluate the use of cloud and on-premises infrastructures (including multiple GPUs) in conjunction with different neural network architectures and frameworks. We assess both the training times as well as the accuracy of the classifier. Finally, we demonstrate the effectiveness of the trained classifier in a real world setting during manufacturing process.Comment: 10 page

Special Session on Industry 4.0

No abstract available

Synthetic Data-Enhanced Deep Learning For Quality Control Of Automated Welding Processes

Automotive production systems are designed to produce large quantities in high quality and short throughput times and are therefore organized as line production. This places high quality requirements on the joining processes in automotive body shops, in which automated, robot-guided welding is a key process. The quality of these thermal joining processes depends on various physical and chemical influencing factors, whose interactions cannot be explicitly modelled. This leads to enormous quality assurance efforts in several quality control loops, which may include visual inspections, non-destructive testing of samples to assess the internal structure and destructive testing of samples for the assessment of mechanical properties such as tensile strength. Due to the increasing availability of data in automated processes and the complexity of welding processes, the application of Deep Learning has a great potential to reduce quality control efforts in automotive body shops. Using Deep Learning to leverage process data and accurately predict quality parameters in welding processes is investigated in research, yet model training requires a large, balanced and annotated dataset, whose generation is time and cost intensive, particularly for production data. However, there are generative AI methods such as Generative Adversarial Networks (GANs) that are able to generate synthetic data and thus offer the potential to generate a large amount of annotated production data with relatively little effort. This paper presents a systematic approach to evaluate the potential of incorporating synthetic data in a real-world production dataset to improve quality control using Deep Learning. The approach is validated for the analysis of real-world ultrasound images of resistance spot welding (RSW) processes from the automotive industry. Different Deep Learning architectures to generate synthetic data are compared. Results show that adding synthetic data to the training dataset can improve the accuracy of Deep Learning models for quality monitoring in welding processes