Artificial Intelligence in Civil Infrastructure Health Monitoring—historical Perspectives, Current Trends, and Future Visions

Over the past 2 decades, the use of artificial intelligence (AI) has exponentially increased toward complete automation of structural inspection and assessment tasks. This trend will continue to rise in image processing as unmanned aerial systems (UAS) and the internet of things (IoT) markets are expected to expand at a compound annual growth rate of 57.5% and 26%, respectively, from 2021 to 2028. This paper aims to catalog the milestone development work, summarize the current research trends, and envision a few future research directions in the innovative application of AI in civil infrastructure health monitoring. A blow-by-blow account of the major technology progression in this research field is provided in a chronological order. Detailed applications, key contributions, and performance measures of each milestone publication are presented. Representative technologies are detailed to demonstrate current research trends. A road map for future research is outlined to address contemporary issues such as explainable and physics-informed AI. This paper will provide readers with a lucid memoir of the historical progress, a good sense of the current trends, and a clear vision for future research

Use of data mining techniques to explain the primary factors influencing water sensitivity of asphalt mixtures

The water sensitivity of asphalt mixtures affects the durability of the pavements, and it depends on several parameters related to its composition (aggregates and binder) and the production and application processes. One of the main parameters used in the European Standards to measure the water sensitivity of asphalt mixtures is the indirect tensile strength ratio (ITSR). Therefore, this work aims to obtain a predictive model of ITSR of asphalt mixtures using several parameters that affect water sensitivity and assess their relative importance. The database used to develop the model comprises thirteen parameters collected from one hundred sixty different asphalt mixtures. Data Mining techniques were applied to process the data using Multiple Regression, Artificial Neural Networks, and Support Vector Machines (SVM). The different metrics analysed showed that SVM is the best predictive model of the ITSR (mean absolute deviation of 0.116, root mean square error of 0.150 and Pearson correlation coefficient of 0.667). The application of a sensitivity analysis indicates that the binder content is the parameter that most influences the water sensitivity of asphalt mixtures (26%). However, this property depends simultaneously on other factors such as the characteristics of the coarse and fine aggregates (24.9%), asphalt binder characteristics (19.3%) and the use of additives (10%).Acknowledgements This work was partly financed by FCT/MCTES through national funds (PIDDAC) under the R & D Unit Institute for Sustainability and Innovation in Structural Engineering (ISISE) , under reference UIDB/04029/2020

Unmasking Clever Hans Predictors and Assessing What Machines Really Learn

Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly "intelligent" behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem solving behaviors. Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for. Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication

Artificial Intelligence-Based Prediction of Permeable Pavement Surface Infiltration Rates

Permeable pavements are a type of low impact development technology that is an alternative to conventional asphalt pavements. These pavements are used to address urban stormwater runoff concerns through infiltration and storage. Overtime, sediments carried by stormwater runoff degrade the performance of these pavements and can eventually diminish the infiltration capacity to the point where no infiltration takes place. The objective of this research is to develop a data-driven model to predict the infiltration rate of permeable pavements. Four permeable concrete lab specimens were constructed and subjected to clogging cycles while obtaining surface images and infiltration data. An artificial neural network was created to investigate the relationship between the images of the pavement surface and its associated surface infiltration rate. Results indicated that image parameters do change significantly as pavements clog and are suitable as inputs to predict surface infiltration rate, although model variability needs to be addressed

BCI applications based on artificial intelligence oriented to deep learning techniques

A Brain-Computer Interface, BCI, can decode the brain signals corresponding to the intentions of individuals who have lost neuromuscular connection, to reestablish communication to control external devices. To this aim, BCI acquires brain signals as Electroencephalography (EEG) or Electrocorticography (ECoG), uses signal processing techniques and extracts features to train classifiers for providing proper control instructions. BCI development has increased in the last decades, improving its performance through the use of different signal processing techniques for feature extraction and artificial intelligence approaches for classification, such as deep learning-oriented classifiers. All of these can assure more accurate assistive systems but also can enable an analysis of the learning process of signal characteristics for the classification task. Initially, this work proposes the use of a priori knowledge and a correlation measure to select the most discriminative ECoG signal electrodes. Then, signals are processed using spatial filtering and three different types of temporal filtering, followed by a classifier made of stacked autoencoders and a softmax layer to discriminate between ECoG signals from two types of visual stimuli. Results show that the average accuracy obtained is 97% (+/- 0.02%), which is similar to state-of-the-art techniques, nevertheless, this method uses minimal prior physiological and an automated statistical technique to select some electrodes to train the classifier. Also, this work presents classifier analysis, figuring out which are the most relevant signal features useful for visual stimuli classification. The features and physiological information such as the brain areas involved are compared. Finally, this research uses Convolutional Neural Networks (CNN) or Convnets to classify 5 categories of motor tasks EEG signals. Movement-related cortical potentials (MRCPs) are used as a priori information to improve the processing of time-frequency representation of EEG signals. Results show an increase of more than 25% in average accuracy compared to a state-of-the-art method that uses the same database. In addition, an analysis of CNN or ConvNets filters and feature maps is done to and the most relevant signal characteristics that can help classify the five types of motor tasks.DoctoradoDoctor en Ingeniería Eléctrica y Electrónic

Artificial Intelligence in Materials Modeling and Design

In recent decades, the use of artificial intelligence (AI) techniques in the field of materials modeling has received significant attention owing to their excellent ability to analyze a vast amount of data and reveal correlations between several complex interrelated phenomena. In this review paper, we summarize recent advances in the applications of AI techniques for numerical modeling of different types of materials. AI techniques such as machine learning and deep learning show great advantages and potential for predicting important mechanical properties of materials and reveal how changes in certain principal parameters affect the overall behavior of engineering materials. Furthermore, in this review, we show that the application of AI techniques can significantly help to improve the design and optimize the properties of future advanced engineering materials. Finally, a perspective on the challenges and prospects of the applications of AI techniques for material modeling is presented

Leveraging Computer Vision for Applications in Biomedicine and Geoscience

Skin cancer is one of the most common types of cancer and is usually classified as either non-melanoma and melanoma skin cancer. Melanoma skin cancer accounts for about half of all skin cancer-related deaths. The 5-year survival rate is 99% when the cancer is detected early but drops to 25% once it becomes metastatic. In other words, the key to preventing death is early detection. Foraminifera are microscopic single-celled organisms that exist in marine environments and are classified as living a benthic or planktic lifestyle. In total, roughly 50,000 species are known to have existed, of which about 9,000 are still living today. Foraminifera are important proxies for reconstructing past ocean and climate conditions and as bio-indicators of anthropogenic pollution. Since the 1800s, the identification and counting of foraminifera have been performed manually. The process is resource-intensive. In this dissertation, we leverage recent advances in computer vision, driven by breakthroughs in deep learning methodologies and scale-space theory, to make progress towards both early detection of melanoma skin cancer and automation of the identification and counting of microscopic foraminifera. First, we investigate the use of hyperspectral images in skin cancer detection by performing a critical review of relevant, peer-reviewed research. Second, we present a novel scale-space methodology for detecting changes in hyperspectral images. Third, we develop a deep learning model for classifying microscopic foraminifera. Finally, we present a deep learning model for instance segmentation of microscopic foraminifera. The works presented in this dissertation are valuable contributions in the fields of biomedicine and geoscience, more specifically, towards the challenges of early detection of melanoma skin cancer and automation of the identification, counting, and picking of microscopic foraminifera

Machine Learning Prediction of Mechanical and Durability Properties of Recycled Aggregates Concrete

Whilst recycled aggregate (RA) can alleviate the environmental footprint of concrete production and the landfilling of colossal amounts of demolition waste, there need for robust predictive tools for its effects on mechanical and durability properties. In this thesis, state-of-the-art machine learning (ML) models were deployed to predict properties of recycled aggregate concrete (RAC). A systematic review was performed to analyze pertinent ML techniques previously applied in the concrete technology field. Accordingly, three different ML methods were selected to determine the compressive strength of RAC and perform mixture proportioning optimization. Furthermore, a gradient boosting regression tree was used to study the effects of RA and several types of binders on the carbonation depth of RAC. The ML models developed in this study demonstrated robust performance to predict diverse properties of RAC

Deep Learning-Guided Prediction of Material’s Microstructures and Applications to Advanced Manufacturing

Material microstructure prediction based on processing conditions is very useful in advanced manufacturing. Trial-and-error experiments are very time-consuming to exhaust numerous combinations of processing parameters and characterize the resulting microstructures. To accelerate process development and optimization, researchers have explored microstructure prediction methods, including physical-based modeling and feature-based machine learning. Nevertheless, they both have limitations. Physical-based modeling consumes too much computational power. And in feature-based machine learning, low-dimensional microstructural features are manually extracted to represent high-dimensional microstructures, which leads to information loss. In this dissertation, a deep learning-guided microstructure prediction framework is established. It uses a conditional generative adversarial network (CGAN) to regress microstructures against numerical processing parameters. After training, the algorithm grasps the mapping between microstructures and processing parameters and can infer the microstructure according to an unseen processing parameter value. This CGAN-enabled approach consumes low computational power for prediction and does not require manual feature extraction. A regression-based conditional Wasserstein generative adversarial network (RCWGAN) is developed, and its microstructure prediction capability is demonstrated on a synthetic micrograph dataset. Several important hyperparameters, including loss function, model depth, number of training epochs, and size of the training set, are systematically studied and optimized. After optimization, prediction accuracy in various microstructural features is over 92%. Then the RCWGAN is validated on a scanning electron microscopy (SEM) micrograph dataset obtained from laser-sintered alumina. Data augmentation is applied to ensure an adequate number of training samples. Different regularization technologies are studied. It is found that gradient penalty can preserve the most details in the generated microstructure. After training, the RCWGAN is able to predict the microstructure as a function of laser power. In-situ microstructure monitoring using the RCWGAN is proposed and demonstrated. Obtaining microstructure information during fabrication could enable accurate microstructure control. It opens the possibility of fabricating a new kind of materials with novel functionalities. The RCWGAN is integrated into a laser sintering system equipped with a camera to demonstrate this novel application. Surface-emission brightness is captured by the camera during the laser sintering process and fed to the RCWGAN for online microstructure prediction. After training, the RCWGAN learns the mapping between surface-emission brightness and microstructures and can make prediction in seconds. The prediction accuracy is over 95% in terms of average grain size