Machine Learning-Based Constitutive Modelling for Granular Materials

As a material second only to liquids in nature, granular materials are widely used in hydraulic structures, roads, bridges etc. Dam-building granular materials are complex systems of pore structures and continuously graded rock particles. An accurate description of their mechanical properties is essential for the safety analysis of ultra-high rockfill dams. At the microscopic scale, granular materials are discrete elementary systems aggregated by complex internal interactions, and their microscopic mechanical structure and statistical characteristics influence the macroscopic mechanical properties; at the macroscopic scale, especially in engineering-scale computational analysis, granular materials are often regarded as continuous media and their constitutive relationship are described using non-linear or elastic-plastic theories. Yet, there is no unified theory to characterise all their constitutive properties. Constitutive modelling stands as a pivotal topic within mechanical calculations. Establishing an accurate description of the relationship between deformation and constitutive response serves as the foundation for Boundary Value Problem analysis. With the growing prominence of machine learning techniques in the data-driven realm, they are expected to enhance constitutive modelling and potentially surpass classical models based on simplifying assumptions. More and more endeavours have been dedicated to integrating machine learning into mechanical calculations and assessing its efficacy. This PhD thesis focuses on the use of machine learning techniques to investigate the feasibility of developing a constitutive model for granular materials and applying it in boundary value problem calculations. The main areas of research include the following aspects:1. In Chapter 2, we introduce a deep learning model designed to reproduce the macroscopic mechanical response of granular materials across various particle size distributions (PSDs) and initial states, considering different loading conditions. We start by extracting stress-strain data from massive DEM simulations and then proceed to capture the mechanical behaviour of these granular materials through the Long Short-Term Memory networks. The work contains three central issues: LSTM cell customisation, granular materials stress-strain sampling, and loading history pasteurisation. The validation results demonstrate that this deep learning model achieves good generalisation and a high level of prediction accuracy when tested on the true triaxial loading dataset.For the different loading and unloading paths in the conventional triaxial simulation of the DEM, an Active Learning approach is introduced to guide the sampling (Chapter 3). Based on the positive correlation between the prediction error and the uncertainty given by activate learning method, the strain paths are evaluated without DEM simulations, from which the worst predicted paths are selected for sampling. To prevent data redundancy, points in the vicinity of one selected point will not be selected for the current resampling round. The model was trained on single-cyclic loading datasets and performed quite well under multiple-cyclic loading paths.In order to circumvent the reliance on phenomenological assumptions in boundary value problem analysis, a computational framework coupled with FEM and neural network (FEM-NN) is proposed (Chapter 4). Building on the work in Chapter 2 and 3, we further introduce FEM-DEM multiscale simulations by employing the Random Gaussian Process to generate macroscopic random loading paths to be applied to the macro-scale model. A large amount of stress-strain data on the integration points is collected. Part of them are subsequently, used to train the neural network. Material loading histories represented by encoded variables. Active learning is employed here again to assess the informativeness of the data points, according to which the points are resampled from the massive database. Two examples are provided to demonstrate the effectiveness of the implemented framework which provides considerable improvements in computational efficiency and the ability to reproduce the mechanical response of granular materials at the macroscopic scale.4. In Chapter 5 the trained network-based constitutive model is embedded into the explicit FEM solver. In implicit FEM solvers for non-linear static problems, a global equilibrium solution is typically obtained via Newton-Raphson iteration. However, the non-linear iterations may not converge when the predicted tangential matrix is not accurate enough. Therefore, the explicit FEM solver is employed to circumvent non-linear iteration. The network is trained and investigated on data generated from two constitutive models (IME model and CSUH model) separately. The trained network is able to reproduce almost exactly the ground truth results at the macroscopic level. However, the error accumulation problem resulting from a large number of steps is an-other challenge to the prediction accuracy and robustness of the data-driven model. A check-and-revision method is proposed to iteratively optimise the model by expanding the training range and improving the network generalisation.5. Chapter 6 focuses on evaluating the capacity and performance of a network-based material cell with physics extension against boundary-value problems. The proposed material cell aims to reproduce constitutive relationships learned from datasets generated by random loading paths following random Gaussian Process. The material cell demonstrates its effectiveness across three progressively complex constitutive models by incorporating physics-based basis functions as prior/assumptions. An adaptive linear transformation is introduced to mitigate the error caused by magnitude gaps between strain increments in training sets and finite element simulations. The mate- rial cell successfully reproduces constitutive relationships in FEM simulations, and its performance is comprehensively evaluated by comparing two different material cells: the sequentially trained gated recurrent unit (GRU)-based material cell and the one-to-one trained deep network-based material cell. The GRU-based material cell can be trained without prior knowledge about the internal variables. Consequently, this enables us to directly derive the constitutive model using stress-strain data obtained from experiments.6. A universal constitutive model has been introduced, combining the recurrent machine learning structure with traditional constitutive models in Chapter 7. A dramatic drop in prediction accuracy emerges when the input strain exceeds the training space because of the poor generalisation ability of the purely data-driven method. Therefore, we introduce the widely accepted elasticity theory, yielding, hardening and plastic flow as physical constraints to build a machine learning-based universal constitutive model. These constraints serve as priors/assumptions for the machine learning model. During the sample preparation stage, they alleviate the stringent demands for the completeness of data sampling. In the model calculations, they guide the model to make predictions, even for unseen loading paths. The proposed model has been calibrated and tested with FEM-DEM datasets

Machine Learning for Financial Prediction Under Regime Change Using Technical Analysis: A Systematic Review

Recent crises, recessions and bubbles have stressed the non-stationary nature and the presence of drastic structural changes in the financial domain. The most recent literature suggests the use of conventional machine learning and statistical approaches in this context. Unfortunately, several of these techniques are unable or slow to adapt to changes in the price-generation process. This study aims to survey the relevant literature on Machine Learning for financial prediction under regime change employing a systematic approach. It reviews key papers with a special emphasis on technical analysis. The study discusses the growing number of contributions that are bridging the gap between two separate communities, one focused on data stream learning and the other on economic research. However, it also makes apparent that we are still in an early stage. The range of machine learning algorithms that have been tested in this domain is very wide, but the results of the study do not suggest that currently there is a specific technique that is clearly dominant

Blood Glucose Level Prediction: A Graph-based Explainable Method with Federated Learning

In the UK, approximately 400,000 people with type 1 diabetes (T1D) rely on insulin delivery due to insufficient pancreatic insulin production. Managing blood glucose (BG) levels is crucial, with continuous glucose monitoring (CGM) playing a key role. CGM, tracking BG every 5 minutes, enables effective blood glucose level prediction (BGLP) by considering factors like carbohydrate intake and insulin delivery. Recent research has focused on developing sequential models for BGLP using historical BG data, incorporating additional attributes such as carbohydrate intake, insulin delivery, and time. These methods have shown notable success in BGLP, with some providing temporal explanations. However, they often lack clear correlations between attributes and their impact on BGLP. Additionally, some methods raise privacy concerns by aggregating participant data to learn population patterns. Addressing these limitations, we introduced a graph attentive memory (GAM) model, combining a graph attention network (GAT) with a gated recurrent unit (GRU). GAT applies graph attention to model attribute correlations, offering transparent, dynamic attribute relationships. Attention weights dynamically gauge attribute significance over time. To ensure privacy, we employed federated learning (FL), facilitating secure population pattern analysis. Our method was validated using the OhioT1DM'18 and OhioT1DM'20 datasets from 12 participants, focusing on 6 key attributes. We demonstrated our model's stability and effectiveness through hyperparameter impact analysis

Evolving developmental, recurrent and convolutional neural networks for deliberate motion planning in sparse reward tasks

Motion planning algorithms have seen a diverse set of approaches in a variety of disciplines. In the domain of artificial evolutionary systems, motion planning has been included in models to achieve sophisticated deliberate behaviours. These algorithms rely on fixed rules or little evolutionary influence which compels behaviours to conform within those specific policies, rather than allowing the model to establish its own specialised behaviour. In order to further these models, the constraints imposed by planning algorithms must be removed to grant greater evolutionary control over behaviours. That is the focus of this thesis. An examination of prevailing neuroevolution methods led to the use of two distinct approaches, NEAT and HyperNEAT. Both were used to gain an understanding of the components necessary to create neuroevolution planning. The findings accumulated in the formation of a novel convolutional neural network architecture with a recurrent convolution process. The architecture’s goal was to iteratively disperse local activations to greater regions of the feature space. Experimentation showed significantly improved robustness over contemporary neuroevolution techniques as well as an efficiency increase over a static rule set. Greater evolutionary responsibility is given to the model with multiple network combinations; all of which continually demonstrated the necessary behaviours. In comparison, these behaviours were shown to be difficult to achieve in a state-of-the-art deep convolutional network. Finally, the unique use of recurrent convolution is relocated to a larger convolutional architecture on an established benchmarking platform. Performance improvements are seen on a number of domains which illustrates that this recurrent mechanism can be exploited in alternative areas outside of planning. By presenting a viable neuroevolution method for motion planning a potential emerges for further systems to adopt and examine the capability of this work in prospective domains, as well as further avenues of experimentation in convolutional architectures

A Comparative Study on Statistical and Machine Learning Forecasting Methods for an FMCG Company

Demand forecasting has been an area of study among scholars and businessmen ever since the start of the industrial revolution and has only gained focus in recent years with the advancements in AI. Accurate forecasts are no longer a luxury, but a necessity to have for effective decisions made in planning production and marketing. Many aspects of the business depend on demand, and this is particularly true for the Fast-Moving Consumer Goods industry where the high volume and demand volatility poses a challenge for planners to generate accurate forecasts as consumer demand complexity rises. Inaccurate demand forecasts lead to multiple issues such as high holding costs on excess inventory, shortages on certain SKUs in the market leading to sales loss and a significant impact on both top line and bottom line for the business. Researchers have attempted to look at the performance of statistical time series models in comparison to machine learning methods to evaluate their robustness, computational time and power. In this paper, a comparative study was conducted using statistical and machine learning techniques to generate an accurate forecast using shipment data of an FMCG company. Naïve method was used as a benchmark to evaluate performance of other forecasting techniques, and was compared to exponential smoothing, ARIMA, KNN, Facebook Prophet and LSTM using past 3 years shipments. Methodology followed was CRISP-DM from data exploration, pre-processing and transformation before applying different forecasting algorithms and evaluation. Moreover, secondary goals behind this paper include understanding associations between SKUs through market basket analysis, and clustering using KNN based on brand, customer, order quantity and value to propose a product segmentation strategy. The results of both clustering and forecasting models are then evaluated to choose the optimal forecasting technique, and a visual representation of the forecast and exploratory analysis conducted is displayed using R

Insects have the capacity for subjective experience

To what degree are non-human animals conscious? We propose that the most meaningful way to approach this question is from the perspective of functional neurobiology. Here we focus on subjective experience, which is a basic awareness of the world without further reflection on that awareness. This is considered the most basic form of consciousness. Tellingly, this capacity is supported by the integrated midbrain and basal ganglia structures, which are among the oldest and most highly conserved brain systems in vertebrates. A reasonable inference is that the capacity for subjective experience is both widespread and evolutionarily old within the vertebrate lineage. We argue that the insect brain supports functions analogous to those of the vertebrate midbrain and hence that insects may also have a capacity for subjective experience. We discuss the features of neural systems which can and cannot be expected to support this capacity as well as the relationship between our arguments based on neurobiological mechanism and our approach to the “hard problem” of conscious experience