Overview: Computer vision and machine learning for microstructural characterization and analysis

The characterization and analysis of microstructure is the foundation of microstructural science, connecting the materials structure to its composition, process history, and properties. Microstructural quantification traditionally involves a human deciding a priori what to measure and then devising a purpose-built method for doing so. However, recent advances in data science, including computer vision (CV) and machine learning (ML) offer new approaches to extracting information from microstructural images. This overview surveys CV approaches to numerically encode the visual information contained in a microstructural image, which then provides input to supervised or unsupervised ML algorithms that find associations and trends in the high-dimensional image representation. CV/ML systems for microstructural characterization and analysis span the taxonomy of image analysis tasks, including image classification, semantic segmentation, object detection, and instance segmentation. These tools enable new approaches to microstructural analysis, including the development of new, rich visual metrics and the discovery of processing-microstructure-property relationships.Comment: submitted to Materials and Metallurgical Transactions

Structural Material Property Tailoring Using Deep Neural Networks

Advances in robotics, artificial intelligence, and machine learning are ushering in a new age of automation, as machines match or outperform human performance. Machine intelligence can enable businesses to improve performance by reducing errors, improving sensitivity, quality and speed, and in some cases achieving outcomes that go beyond current resource capabilities. Relevant applications include new product architecture design, rapid material characterization, and life-cycle management tied with a digital strategy that will enable efficient development of products from cradle to grave. In addition, there are also challenges to overcome that must be addressed through a major, sustained research effort that is based solidly on both inferential and computational principles applied to design tailoring of functionally optimized structures. Current applications of structural materials in the aerospace industry demand the highest quality control of material microstructure, especially for advanced rotational turbomachinery in aircraft engines in order to have the best tailored material property. In this paper, deep convolutional neural networks were developed to accurately predict processing-structure-property relations from materials microstructures images, surpassing current best practices and modeling efforts. The models automatically learn critical features, without the need for manual specification and/or subjective and expensive image analysis. Further, in combination with generative deep learning models, a framework is proposed to enable rapid material design space exploration and property identification and optimization. The implementation must take account of real-time decision cycles and the trade-offs between speed and accuracy

Deep Learning Based Reliability Models For High Dimensional Data

The reliability estimation of products has crucial applications in various industries, particularly in current competitive markets, as it has high economic impacts. Hence, reliability analysis and failure prediction are receiving increasing attention. Reliability models based on lifetime data have been developed for different modern applications. These models are able to predict failure by incorporating the influence of covariates on time-to-failure. The covariates are factors that affect the subjects’ lifetime. Modern technologies generate covariates which can be utilized to improve failure time prediction. However, there are several challenges to incorporate the covariates into reliability models. First, the covariates generally are high dimensional and topologically complex. Second, the existing reliability models are not efficient in modeling the effect on the complex covariates on failure time. Third, failure time information may not be available for all covariates, as collecting such information is a costly and time-consuming process. To overcome the first challenge, we propose a statistical approach to model the complex data. The proposed model generalizes penalized logistic regression to capture the spatial properties of the data. An efficient parameter estimation method is developed to make the model practical in case of large sample sizes. To tackle the second challenge, a deep learning-based reliability model is proposed. The model can capture the complex effect of the data on failure time. A novel loss function based on the partial likelihood function is developed to train the deep learning model. Furthermore, to overcome the third difficulty, we proposed a transfer learning-based reliability model to estimate failure time based on the failure time of similar covariates. The proposed model is based on a two-level autoencoder to minimize the distribution distance of covariates. A new parameter estimation method is developed to estimate the parameter of the proposed two-level autoencoder model. Various simulation studies are conducted to demonstrate the proposed models. The results show that the proposed models outperformed the traditional statistical and reliability models. Moreover, physical experiments on advanced high strength steel are designed to demonstrate the proposed model. As microstructure images of the steels affect the failure time of the steel, the images are considered as covariates. The results show that the proposed models predict the failure time and hazard function of the materials more accurately than existing reliability models

Automated Segmentation of Large Image Datasets using Artificial Intelligence for Microstructure Characterisation, Damage Analysis and High-Throughput Modelling Input

Many properties of commonly used materials are driven by their microstructure, which can be influenced by the composition and manufacturing processes. To optimise future materials, understanding the microstructure is critically important. Here, we present two novel approaches based on artificial intelligence that allow the segmentation of the phases of a microstructure for which simple numerical approaches, such as thresholding, are not applicable: One is based on the nnU-Net neural network, and the other on generative adversarial networks (GAN). Using large panoramic scanning electron microscopy images of dual-phase steels as a case study, we demonstrate how both methods effectively segment intricate microstructural details, including martensite, ferrite, and damage sites, for subsequent analysis. Either method shows substantial generalizability across a range of image sizes and conditions, including heat-treated microstructures with different phase configurations. The nnU-Net excels in mapping large image areas. Conversely, the GAN-based method performs reliably on smaller images, providing greater step-by-step control and flexibility over the segmentation process. This study highlights the benefits of segmented microstructural data for various purposes, such as calculating phase fractions, modelling material behaviour through finite element simulation, and conducting geometrical analyses of damage sites and the local properties of their surrounding microstructure.Comment: 37 pages, 24 figure

Fabrication, characterization of high-entropy alloys and deep learning-based inspection in metal additive manufacturing

Alloying has been used to confer desirable properties to materials. It typically involves the addition of small amounts of secondary elements to a primary element. In the past decade, however, a new alloying strategy that involves the combination of multiple principal elements in high concentrations to create new materials called high- entropy alloys (HEAs) has been in vogue. In the first part, the investigation focused on the fabrication process and property assessment of the additive manufactured HEA to broaden its engineering applications. Additive manufacturing (AM) is based on manufacturing philosophy through the layer-by-layer method and accomplish the near net-shaped components fabrication. Attempt was made to coat AlCoCrFeNi HEA on an AISI 304 stainless steel substrate to integrate their properties, however, it failed due to the cracks at the interface. The implementation of an intermediate layer improved the bond and eliminated the cracks. Next, an AlCoCrFeNiTi0.5 HEA coating was fabricated on the Ti6Al4V substrate, and its isothermal oxidation behavior was studied. The HEA coating effectively improved the Ti6Al4V substrate\u27s oxidation resistance due to the formation of continuous protective oxides. In the second part, research efforts were made on the deep learning-based quality inspection of additive manufactured products. The traditional inspection process has relied on manual recognition, which could suffer from low efficiency and potential bias. A neural-network approach was developed toward robust real-world AM anomaly detection. The results indicate the promising application of the neural network in the AM industry --Abstract, page iv

Microstructure quality control of steels using deep learning

In quality control, microstructures are investigated rigorously to ensure structural integrity, exclude the presence of critical volume defects, and validate the formation of the target microstructure. For quenched, hierarchically-structured steels, the morphology of the bainitic and martensitic microstructures are of major concern to guarantee the reliability of the material under service conditions. Therefore, industries conduct small sample-size inspections of materials cross-sections through metallographers to validate the needle morphology of such microstructures. We demonstrate round-robin test results revealing that this visual grading is afflicted by pronounced subjectivity despite the thorough training of personnel. Instead, we propose a deep learning image classification approach that distinguishes steels based on their microstructure type and classifies their needle length alluding to the ISO 643 grain size assessment standard. This classification approach facilitates the reliable, objective, and automated classification of hierarchically structured steels. Specifically, an accuracy of 96% and roughly 91% is attained for the distinction of martensite/bainite subtypes and needle length, respectively. This is achieved on an image dataset that contains significant variance and labeling noise as it is acquired over more than ten years from multiple plants, alloys, etchant applications, and light optical microscopes by many metallographers (raters). Interpretability analysis gives insights into the decision-making of these models and allows for estimating their generalization capability