Image-based Decision Support Systems: Technical Concepts, Design Knowledge, and Applications for Sustainability

Unstructured data accounts for 80-90% of all data generated, with image data contributing its largest portion. In recent years, the field of computer vision, fueled by deep learning techniques, has made significant advances in exploiting this data to generate value. However, often computer vision models are not sufficient for value creation. In these cases, image-based decision support systems (IB-DSSs), i.e., decision support systems that rely on images and computer vision, can be used to create value by combining human and artificial intelligence. Despite its potential, there is only little work on IB-DSSs so far. In this thesis, we develop technical foundations and design knowledge for IBDSSs and demonstrate the possible positive effect of IB-DSSs on environmental sustainability. The theoretical contributions of this work are based on and evaluated in a series of artifacts in practical use cases: First, we use technical experiments to demonstrate the feasibility of innovative approaches to exploit images for IBDSSs. We show the feasibility of deep-learning-based computer vision and identify future research opportunities based on one of our practical use cases. Building on this, we develop and evaluate a novel approach for combining human and artificial intelligence for value creation from image data. Second, we develop design knowledge that can serve as a blueprint for future IB-DSSs. We perform two design science research studies to formulate generalizable principles for purposeful design — one for IB-DSSs and one for the subclass of image-mining-based decision support systems (IM-DSSs). While IB-DSSs can provide decision support based on single images, IM-DSSs are suitable when large amounts of image data are available and required for decision-making. Third, we demonstrate the viability of applying IBDSSs to enhance environmental sustainability by performing life cycle assessments for two practical use cases — one in which the IB-DSS enables a prolonged product lifetime and one in which the IB-DSS facilitates an improvement of manufacturing processes. We hope this thesis will contribute to expand the use and effectiveness of imagebased decision support systems in practice and will provide directions for future research

Condition monitoring of tool performance using a machine learning-based on-machine vision system during face milling of Inconel 718

The superior properties of Inconel 718 necessitate its use in manufacturing more than 50% of aircraft engine structural components, including high-pressure compressor blades, casings, and discs. However, literature attributed the synergistic impact of these properties and process parameters as the primary cause of wear complexity, notably affecting the performance of PVD-coated carbide inserts during CNC milling of Inconel 718. Features stemming from the wear complexity include uncontrolled wear mechanisms, failure modes, and a rapid flank wear rate, serving as significant indicators of sub-optimal cutting conditions. In trying to diagnose tool wear, previous Tool Condition Monitoring (TCM) techniques could not decipher, explore, and synthesise the diverse features essential for the predictive control of tool performance in challenging CNC machining conditions. Therefore, the successful implementation of advanced feature engineering and Machine Learning (ML) models in Machine Vision-based TCM (MVTCM) offers a proactive approach in predicting and controlling the performance of PVD-coated carbide tools in challenging CNC machining domains. The hypothesis of this study encompassed three aspects. The first aspect focused on the study of tool wear complexity by characterizing the dominant wear mechanisms, failure modes, and flank wear depth (VB) during face milling of Inconel 718. These features were correlated with the process parameters to establish a coherent tool wear dataset for training the feature engineering and ML models. The second aspect involved the development of feature engineering and ML models, including the multi-sectional singular value decomposition (SVD), a YOLOv3 Tool Wear Detection Model (YOLOv3-TWDM), a multi-layer perceptron neural network (MLPNN), and an inductive-reasoning algorithm. The final aspect pertained to the development of a volatile MV-TCM system’s design, which was integrated with feature engineering and ML techniques to create an enhanced ML-based MV-TCM system. The system was vigorously validated by conducting an online experiment, where the predicted were compared with the actual wear measurements. Furthermore, the inductive reasoning algorithm was devised to regulate process parameters for in-process control of flank wear evolution. The findings demonstrate that the Diverse Feature Synthesis Vector devised in this research was superior in representing the complex flank wear morphology as compared to some data reported by relevant literature, where geometric and fractal features were used to predict VB progression online. In addition, the ML-based MV-TCM system successfully utilized the DFSV to predict and control VB rate during face milling of Inconel 718. The system achieved higher predictive efficiency than image processing-based MV-TCM systems applied in the previous studies, with an offline validation RMSE of 45.5µm, R2 of 96.52%, and MAPE of 2.36%, as well as an online validation RMSE of 29.09µm, R2 of 97%, and MAPE of 3.52%. Additionally, the system employed a multi-stage optimization strategy that regulated process parameters at different VB levels to minimize the magnitudes of flank wear and chipping. This strategy extended tool life by 63.63% (relative to the conventional method) and 56.52% (relative to the GKRR soft-computing technique). Therefore, this research demonstrates the significance of applying ML-based MV-TCM system for predictive control of tool wear evolution during CNC milling of Inconel 718

Condition monitoring of tool performance using a machine learning-based on-machine vision system during face milling of Inconel 718

The superior properties of Inconel 718 necessitate its use in manufacturing more than 50% of aircraft engine structural components, including high-pressure compressor blades, casings, and discs. However, literature attributed the synergistic impact of these properties and process parameters as the primary cause of wear complexity, notably affecting the performance of PVD-coated carbide inserts during CNC milling of Inconel 718. Features stemming from the wear complexity include uncontrolled wear mechanisms, failure modes, and a rapid flank wear rate, serving as significant indicators of sub-optimal cutting conditions. In trying to diagnose tool wear, previous Tool Condition Monitoring (TCM) techniques could not decipher, explore, and synthesise the diverse features essential for the predictive control of tool performance in challenging CNC machining conditions. Therefore, the successful implementation of advanced feature engineering and Machine Learning (ML) models in Machine Vision-based TCM (MVTCM) offers a proactive approach in predicting and controlling the performance of PVD-coated carbide tools in challenging CNC machining domains. The hypothesis of this study encompassed three aspects. The first aspect focused on the study of tool wear complexity by characterizing the dominant wear mechanisms, failure modes, and flank wear depth (VB) during face milling of Inconel 718. These features were correlated with the process parameters to establish a coherent tool wear dataset for training the feature engineering and ML models. The second aspect involved the development of feature engineering and ML models, including the multi-sectional singular value decomposition (SVD), a YOLOv3 Tool Wear Detection Model (YOLOv3-TWDM), a multi-layer perceptron neural network (MLPNN), and an inductive-reasoning algorithm. The final aspect pertained to the development of a volatile MV-TCM system’s design, which was integrated with feature engineering and ML techniques to create an enhanced ML-based MV-TCM system. The system was vigorously validated by conducting an online experiment, where the predicted were compared with the actual wear measurements. Furthermore, the inductive reasoning algorithm was devised to regulate process parameters for in-process control of flank wear evolution. The findings demonstrate that the Diverse Feature Synthesis Vector devised in this research was superior in representing the complex flank wear morphology as compared to some data reported by relevant literature, where geometric and fractal features were used to predict VB progression online. In addition, the ML-based MV-TCM system successfully utilized the DFSV to predict and control VB rate during face milling of Inconel 718. The system achieved higher predictive efficiency than image processing-based MV-TCM systems applied in the previous studies, with an offline validation RMSE of 45.5µm, R2 of 96.52%, and MAPE of 2.36%, as well as an online validation RMSE of 29.09µm, R2 of 97%, and MAPE of 3.52%. Additionally, the system employed a multi-stage optimization strategy that regulated process parameters at different VB levels to minimize the magnitudes of flank wear and chipping. This strategy extended tool life by 63.63% (relative to the conventional method) and 56.52% (relative to the GKRR soft-computing technique). Therefore, this research demonstrates the significance of applying ML-based MV-TCM system for predictive control of tool wear evolution during CNC milling of Inconel 718

Automated drill bit forensics : enhancing efficiency and accuracy through image processing and machine learning

In recent years, the automation of drilling has garnered considerable attention within both the upstream oil and gas companies and the drilling research community. Drill bit forensics, being integral to the enhancement of efficiencies and profits in the oil and gas industries, promises heightened drilling efficiency, augmented consistency, and a refined comprehension of bit damage mechanisms through automation. Nevertheless, the adoption rate of drilling automation remains sluggish, largely due to the intricate nature of drilling operations. At present, the conventional inspection and grading of bit damage by human operators is labor-intensive and susceptible to human biases. This underscores the imperative for an automated system in drill bit forensics, which would aid drilling operators and specialists in processing and analyzing bit damage data. In this dissertation, a novel systematic framework is introduced, amalgamating computer vision and machine learning techniques with domain-specific knowledge of drill bits. This framework streamlines the evaluation process from identifying various drill bit components, quantifying and categorizing cutter damage, collating positional data, to ultimately forecasting the primary causes of damage. The methodologies devised are applied to visual data of drill bits, encompassing images and videos from hundreds of different bit runs. This work delves into several innovative contributions: (1) The industry's first bit detection model that segments distinct parts of the bit; (2) A pioneering proposition to utilize video data of drill bits to expedite the automation of bit forensics; (3) A comprehensive workflow tailored for diverse bit data sources; (4) An adaptable analytical methodology for discerning the root causes of bit damage. The outcomes underscore the potential of an automated system in drill bit forensics to bolster the precision and uniformity of drill bit assessments, offering invaluable insights into drilling operations. This groundbreaking methodology lays the foundation for further advancements in the realm of automated drill bit forensics, targeting the enhancement of the overall efficacy and cost-efficiency of drilling operations.Mechanical Engineerin

Artificial cognitive architecture with self-learning and self-optimization capabilities. Case studies in micromachining processes

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura : 22-09-201