Process monitoring by deep neural networks in directed energy deposition : CNN-based detection, segmentation, and statistical analysis of melt pools

The complex interaction between laser and material in Laser Wire Direct Energy Deposition (LW-DED) Additive Manufacturing (AM) benefits from process monitoring methods to ensure process stability and final part quality. Understanding the relationship between process parameters and melt pool geometrical characteristics can be used to effectively monitor and in-process control the process, as the melt pool geometrical characteristics serve as crucial indicators of process stability and quality. This study presents a novel in-situ monitoring approach for LW-DED, utilizing process images for melt pool segmentation, melt pool geometrical characteristics estimation, process stability assessment, and bead geometry prediction. The segmentation of melt pool objects was successfully accomplished using Convolutional Neural Networks (CNN)-based models, enabling the prediction of essential parameters such as melt pool area, height, width, center of area, and the center point of the bounding box enclosing the melt pool. Multiple models were compared regarding the accuracy and processing speed using a controlled central composite design and random experiments. We used an Inconel alloy 625 wire and two distinct substrate materials for deposition, a coaxial laser welding head with a 3 kW fiber laser, and an off-axis welding camera for monitoring. Among the CNN architectures evaluated, YOLOv8l demonstrated superior accuracy with mean Average Precision (mAP) values of 0.925 and 0.853 for Stainless Steel (SS) and low carbon steel (S355) substrates, respectively. Additionally, YOLOv8s exhibited a notable processing speed of over 114 frames per second, which indicates its suitability for real-time process control. Furthermore, the results indicate a significant correlation between process parameters and melt pool geometry variables. Notably, a clear correlation was established between melt pool characteristics and bead geometries obtained through microscopic examinations, including penetration depth and heat-affected zone. The analysis revealed a significant correlation for the bead area and width parameters. In relation to the bead height, while the correlation exhibited a lower magnitude compared to bead area and width, it remained responsive. In addition, the tensor masks derived from the developed models have proven to be highly effective in accurately predicting bead geometries. The results demonstrate the effectiveness of YOLO-based algorithms for detecting and segmenting the melt pool. Statistical analysis confirms the significance of stabilized process data and the accuracy of melt pool geometric models. We demonstrate that integrating advanced monitoring and control techniques using artificial intelligence methods like CNN can facilitate process stability and quality control.Peer reviewe

Digital twin of construction crane and realization of the physical to virtual connection

Digital twin is an integrated multi-physics representation of a complex physical entity. This article constructs the digital twin of the construction crane, proposes a framework for the construction of the tower crane digital twin, and realizes the connection from physical to virtual in the concept of digital twin. The main contributions are divided into three parts: development of tower crane monitoring dataset, tower crane detection and tower crane operation mode recognition. By using labellmg to annotate more than 20,000 tower crane images in 583 tower crane videos, a tower crane image recognition dataset and a tower crane operating mode dataset are established. Yolov5x algorithm is selected in the tower crane detection. Edge extraction is used to improve the quality of the raw dataset and distance-intersection-over union non-maximum suppression is used to replace the traditional non-maximum suppression part in the Yolov5x algorithm to improve the detect accuracy when some tower cranes are overlapping. The final test set detection accuracy rate is 93.85%. After comparing the LSTM and CNN algorithms, 3DResNet algorithm is selected for tower crane operational mode recognition. The raw dataset is augmented by rotating the image by ±10° and ±20°, and the augmented dataset enlarges five times. Using these methods, the final recognition accuracy of tower crane operation mode reaches 87%. These models can be installed on the cctv to monitor the running status of the tower crane in real time and transfer relevant information to the virtual model. The tower crane in the virtual space completes the action of the physical tower crane, thereby realizing the physical-to-virtual mapping in the digital twin

Development of an intelligent computer vision system for identification, characterization and analysis of yarn quality

Doctoral program in Electronics and Computers Engineering (specializing in Control, Automation and Robotics)Portugal é um país com uma forte tradição da indústria têxtil e, mesmo no contexto de mudanças no mercado, as empresas portuguesas continuam a perseguir o nível de excelência que lhes permite sobreviver num mundo mais complexo e exigente. Na indústria têxtil, a qualidade do produto final está diretamente relacionada com a qualidade do fio e, portanto, é essencial fazer uma avaliação precisa das características de acordo com certos parâmetros predefinidos. Há uma evolução dos dispositivos que avaliam a qualidade do fio, no entanto, ainda têm limitações, como alto custo, dimensão e peso, assim como resolução e precisão limitadas na determinação de certos parâmetros do fio. O objetivo principal desta tese é desenvolver algoritmos de deep learning para identificar e caracterizar a pilosidade do fio, além de criar algoritmos para caracterizar e analisar outros parâmetros de qualidade do fio usando visão computacional. A estratégia foi, em primeiro lugar, projetar um protótipo mecatrónico que permitisse a captura direta de imagens ou vídeos de alta qualidade do enrolamento do fio, e também uma análise e classificação das pilosidades do fio. O protótipo permite obter outras características inerentes à análise da qualidade do fio, como: massa linear, diâmetro, volume, direção da torção, passo da torção, desvio médio de massa, coeficiente de variação, coeficiente de pilosidade, desvio médio de pilosidade e desvio padrão. Esta tese de doutoramento introduz, como uma das principais contribuições, uma nova abordagem de deep learning utilizando um algoritmo otimizado baseado no YOLOv5s6 (You only look once) para caracterizar diferentes tipos de pilosidade do fio. Os resultados mostram que o algoritmo proposto melhora significativamente o desempenho do modelo, com um aumento de 5-6% na métrica mAP0.5 (mean average precision at 0.5 intersection over union (IoU)) e um aumento de 11-12% na métrica mAP0.5:0.95 em comparação com o algoritmo YOLOv5s6 padrão. A abordagem melhora efetivamente todas as métricas analisadas para a caracterização da pilosidade do fio. A implementação bem-sucedida deste trabalho pode aumentar a eficiência produtiva da indústria têxtil e contribuir para a criação de produtos de alto valor acrescentado.Portugal has a strong tradition in the textile industry, even under the context of market and demands changes, the Portuguese companies continue pursuing the excellence level that makes them survive in a more complex and challenging world. In the textile industry, the quality of the final product is directly related to the quality of the yarn and therefore it is essential to make an accurate assessment of the yarn characteristics, according to certain pre-established parameters. There is a significant evolution of the devices that evaluate the quality of the yarn, however, these devices still have several limitations such as high cost, large dimension, and weight, as well as limited resolution and precision in determining certain parameters of the yarn. The main goal of this thesis is to develop deep learning algorithms to identify and characterize yarn hairiness, as well as to create algorithms for characterizing and analyzing other yarn quality parameters using computer vision. The strategy was, first, to design a mechatronic prototype that allows for the direct capture of high-quality images or videos of yarn winding, and also for the analysis and classification of yarn hairiness. It also allows to obtain other characteristics inherent to the analysis of the yarn quality, such as: linear mass, diameter, volume, twist direction, twist step, average mass deviation, coefficient of variation, hairiness coefficient, average hairiness deviation and standard deviation. This thesis introduces, as one of the main achievements, a novel deep learning approach using an optimized algorithm based on YOLOv5s6 (You only look once) to characterize different types of yarn hairiness. The results show that the proposed algorithm significantly improves the model performance, with a 5-6% increase in mAP0.5 (mean average precision at 0.5 intersection over union (IoU)) metric and an 11-12% increase in the mAP0.5:0.95 metric compared to the standard YOLOv5s6 algorithm. The approach effectively enhances all analyzed metrics for yarn hairiness characterization. The successful implementation of this work can increase the productive efficiency of the textile industry and contribute to the development of high added value products

Flexible Automation and Intelligent Manufacturing: The Human-Data-Technology Nexus

This is an open access book. It gathers the first volume of the proceedings of the 31st edition of the International Conference on Flexible Automation and Intelligent Manufacturing, FAIM 2022, held on June 19 – 23, 2022, in Detroit, Michigan, USA. Covering four thematic areas including Manufacturing Processes, Machine Tools, Manufacturing Systems, and Enabling Technologies, it reports on advanced manufacturing processes, and innovative materials for 3D printing, applications of machine learning, artificial intelligence and mixed reality in various production sectors, as well as important issues in human-robot collaboration, including methods for improving safety. Contributions also cover strategies to improve quality control, supply chain management and training in the manufacturing industry, and methods supporting circular supply chain and sustainable manufacturing. All in all, this book provides academicians, engineers and professionals with extensive information on both scientific and industrial advances in the converging fields of manufacturing, production, and automation

Flexible Automation and Intelligent Manufacturing: The Human-Data-Technology Nexus

This is an open access book. It gathers the first volume of the proceedings of the 31st edition of the International Conference on Flexible Automation and Intelligent Manufacturing, FAIM 2022, held on June 19 – 23, 2022, in Detroit, Michigan, USA. Covering four thematic areas including Manufacturing Processes, Machine Tools, Manufacturing Systems, and Enabling Technologies, it reports on advanced manufacturing processes, and innovative materials for 3D printing, applications of machine learning, artificial intelligence and mixed reality in various production sectors, as well as important issues in human-robot collaboration, including methods for improving safety. Contributions also cover strategies to improve quality control, supply chain management and training in the manufacturing industry, and methods supporting circular supply chain and sustainable manufacturing. All in all, this book provides academicians, engineers and professionals with extensive information on both scientific and industrial advances in the converging fields of manufacturing, production, and automation

Semi-automatic liquid filling system using NodeMCU as an integrated Iot Learning tool

Computer programming and IoT are the key skills required in Industrial Revolution 4.0 (IR4.0). The industry demand is very high and therefore related students in this field should grasp adequate knowledge and skill in college or university prior to employment. However, learning technology related subject without applying it to an actual hardware can pose difficulty to relate the theoretical knowledge to problems in real application. It is proven that learning through hands-on activities is more effective and promotes deeper understanding of the subject matter (He et al. in Integrating Internet of Things (IoT) into STEM undergraduate education: Case study of a modern technology infused courseware for embedded system course. Erie, PA, USA, pp 1–9 (2016)). Thus, to fulfill the learning requirement, an integrated learning tool that combines learning of computer programming and IoT control for an industrial liquid filling system model is developed and tested. The integrated learning tool uses NodeMCU, Blynk app and smartphone to enable the IoT application. The system set-up is pre-designed for semi-automation liquid filling process to enhance hands-on learning experience but can be easily programmed for full automation. Overall, it is a user and cost friendly learning tool that can be developed by academic staff to aid learning of IoT and computer programming in related education levels and field

Proceedings of the Irish Machine Vision and Image Processing Conference 2025

The Organising Committee extends a warm welcome to all speakers and delegates of the 2025 Irish Machine Vision and Image Processing Conference (IMVIP 2025). This year it is hosted at the Ulster University under the organisation of the School of Computing, Engineering and Intelligent Systems and the Intelligent Systems Research Centre. The IMVIP Conference is Ireland’s primary meeting for those researching in the fields of machine vision and image processing. The conference has been running since 1997 and provides a forum for the exchange of ideas and the presentation of research conducted both in Ireland and worldwide. IMVIP is a single track conference consisting of high quality previously unpublished contributed papers focussing on both theoretical research and practical experiences in all areas. After a rigorous review process, 24 papers were selected for oral presentation and a further 14 for poster presentation; we wish to sincerely thank the members of the Programme Committee for generously giving their time, effort and expertise in reviewing the submissions. Continuing the tradition of inviting high-profile speakers to IMVIP, we are delighted to have three high-profile speakers give keynote talks: Cornelia Fermuller from University of Maryland, with a talk entitled "Learning to Play: AI Technologies for Supporting Violin Instruction and Motor Skill Acquisition", Abulele Mditshwa from Amazon Web Services with a talk entitled "Leveraging AWS Rekognition for Advanced Computer Vision Applications: Industry Use Cases and Best Practices" and Oisin Mac Aodha from University of Edinburgh with a talk entitled "Learning from Visual Data in the Wild". IMVIP 2025 is run in association with the Irish Pattern Recognition and Classification Society (IPRCS), a member of the International Association for Pattern Recognition (IAPR) and the International Federation of Classification Societies (IFCS)

Proceedings of Abstracts, School of Physics, Engineering and Computer Science Research Conference 2022

© 2022 The Author(s). This is an open-access work distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. For further details please see https://creativecommons.org/licenses/by/4.0/. Plenary by Prof. Timothy Foat, ‘Indoor dispersion at Dstl and its recent application to COVID-19 transmission’ is © Crown copyright (2022), Dstl. This material is licensed under the terms of the Open Government Licence except where otherwise stated. To view this licence, visit http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected] present proceedings record the abstracts submitted and accepted for presentation at SPECS 2022, the second edition of the School of Physics, Engineering and Computer Science Research Conference that took place online, the 12th April 2022

Ablation Study on Convolutional Neural Network-Transformer Fusion

This paper presents an ablation study on combining Convolutional Neural Networks and Vision Transformers for image classification. Features extracted from both architectures are reduced using dimensionality reduction techniques, Principal Component Analysis and Uniform Manifold Approximation and Projection, and fused using multiple fusion strategies. Support Vector Machine, \textit{k}-Nearest Neighbour, and Random Forest are used for image classification. Results highlight that Uniform Manifold Approximation is more effective for Convolutional Neural Network features, while Principal Component Analysis better suits the Vision Transformer in this study. The best performance was achieved using ResNet50 and Vision Transformer features, reduced with Uniform Manifold Approximation and Principal Component Analysis, fused via concatenation, and classified with \textit{k}-Nearest Neighbour

Review of advanced road materials, structures, equipment, and detection technologies

As a vital and integral component of transportation infrastructure, pavement has a direct and tangible impact on socio-economic sustainability. In recent years, an influx of groundbreaking and state-of-the-art materials, structures, equipment, and detection technologies related to road engineering have continually and progressively emerged, reshaping the landscape of pavement systems. There is a pressing and growing need for a timely summarization of the current research status and a clear identification of future research directions in these advanced and evolving technologies. Therefore, Journal of Road Engineering has undertaken the significant initiative of introducing a comprehensive review paper with the overarching theme of “advanced road materials, structures, equipment, and detection technologies”. This extensive and insightful review meticulously gathers and synthesizes research findings from 39 distinguished scholars, all of whom are affiliated with 19 renowned universities or research institutions specializing in the diverse and multidimensional field of highway engineering. It covers the current state and anticipates future development directions in the four major and interconnected domains of road engineering: advanced road materials, advanced road structures and performance evaluation, advanced road construction equipment and technology, and advanced road detection and assessment technologies