Continual Learning of Multi-modal Dynamics with External Memory

We study the problem of fitting a model to a dynamical environment when new modes of behavior emerge sequentially. The learning model is aware when a new mode appears, but it does not have access to the true modes of individual training sequences. We devise a novel continual learning method that maintains a descriptor of the mode of an encountered sequence in a neural episodic memory. We employ a Dirichlet Process prior on the attention weights of the memory to foster efficient storage of the mode descriptors. Our method performs continual learning by transferring knowledge across tasks by retrieving the descriptors of similar modes of past tasks to the mode of a current sequence and feeding this descriptor into its transition kernel as control input. We observe the continual learning performance of our method to compare favorably to the mainstream parameter transfer approach

Damage classification after the 2009 L'Aquila earthquake using multinomial logistic regression and neural networks

Post-earthquake surveys represent a fundamental tool for managing the emergency phase after a strong earthquake. In Italy, the evaluation of the post-earthquake functionality of ordinary buildings is based on the AeDES forms (Agibilitá e Danno nell'Emergenza Sismica, or equivalently, Rapid Post-Earthquake Damage evaluation forms). This form includes information on the building and records of the observed damage classified according to type and intensity in 60 subclasses. Based on the observed damage and expert judgment, the buildings are clustered into six risk classes, from A to F. The assigned class is used to calculate the maximum economic reimbursement owed for the reconstruction or repair of the building. However, often the cluster assignment is not entirely objective due to the inherent responsibility associated with a less conservative assessment. This paper uses the data from the 2009 L'Aquila earthquake to develop classification models based on multinomial logistic regression (MLR) and artificial neural networks (ANN) calibrated with data theoretically less influenced by personal biases. The proposed models, particularly the MLR, are intended to support the decision-making of the evaluation team in future updates of the AeDES forms. This approach cannot substitute expert evaluation, which is always necessary for complex scenarios but may mitigate the impact of subjectivity and can provide an indication of the expected outcome of the survey

Mitigation of Catastrophic Interference in Neural Networks and Ensembles using a Fixed Expansion Layer

Catastrophic forgetting (also known in the literature as catastrophic interference) is the phenomenon by which learning systems exhibit a severe exponential loss of learned information when exposed to relatively small amounts of new training data. This loss of information is not caused by constraints due to the lack of resources available to the learning system, but rather is caused by representational overlap within the learning system and by side-effects of the training methods used. Catastrophic forgetting in auto-associative pattern recognition is a well-studied attribute of most parameterized supervised learning systems. A variation of this phenomenon, in the context of feedforward neural networks, arises when non-stationary inputs lead to loss of previously learned mappings. The majority of the schemes proposed in the literature for mitigating catastrophic forgetting are not data-driven, but rather rely on storage of prior representations of the learning system. We introduce the Fixed Expansion Layer (FEL) feedforward neural network that embeds an expansion layer which sparsely encodes the information contained within the hidden layer, in order to help mitigate forgetting of prior learned representations. The fixed expansion layer approach is generally applicable to feedforward neural networks, as demonstrated by the application of the FEL technique to a recurrent neural network algorithm built on top of a standard feedforward neural network. Additionally, we investigate a novel framework for training ensembles of FEL networks, based on exploiting an information-theoretic measure of diversity between FEL learners, to further control undesired plasticity. The proposed methodology is demonstrated on a several tasks, clearly emphasizing its advantages over existing techniques. The architecture proposed can be applied to address a range of computational intelligence tasks, including classification problems, regression problems and system control

Predictive modelling of global solar radiation with artificial intelligence approaches using MODIS satellites and atmospheric reanalysis data for Australia

Global solar radiation (GSR) prediction is a prerequisite task for agricultural management and agronomic decisions, including photovoltaic (PV) power generation, biofuel exploration and several other bio-physical applications. Since short-term variabilities in the GSR incorporate stochastic and intermittent behaviours (such as periodic fluctuations, jumps and trends) due to the dynamicity of atmospheric variables, GSR predictions, as required for solar energy generation, is a challenging endeavour to satisfactorily predict the solar generated electricity in a PV system. Additionally, the solar radiation data, as required for solar energy monitoring purposes, are not available in all geographic locations due to the absence of meteorological stations and this is especially true for remote and regional solar powered sites. To surmount these challenges, the universally (and freely available) atmospheric gridded datasets (e.g., reanalysis and satellite variables) integrated into solar radiation predictive models to generate reliable GSR predictions can be considered as a viable medium for future solar energy exploration, utilisation and management. Hence, this doctoral thesis aims to design and evaluate novel Artificial Intelligence (AI; Machine Learning and Deep Learning) based predictive models for GSR predictions, using the European Centre for Medium Range Weather Forecasting (ECMWF) Interim-ERA reanalysis and Moderate Resolution Imaging Spectroradiometer (MODIS) Satellite variables enriched with ground-based weather station datasets for the prediction of both long-term (i.e., monthly averaged daily) as well as the short-term (i.e., daily and half-hourly) GSR. The focus of the study region is Queensland, the sunshine state, as well as a number of major solar cities in Australia where solar energy utilisation is actively being promoted by the Australian State and Federal Government agencies. Firstly, the Artificial Neural Networks (ANN), a widely used Machine Learning model is implemented to predict daily GSR at five different cities in Australia using ECMWF Reanalysis fields obtained from the European Centre for Medium Range Weather Forecasting repository. Secondly, the Self-Adaptive Differential Evolutionary Extreme Learning Machine (i.e., SaDE-ELM) is also proposed for monthly averaged daily GSR prediction trained with ECMWF reanalysis and MODIS satellite data from the Moderate Resolution Imaging Spectroradiometer. Thirdly, a three-phase Support Vector Regression (SVR; Machine Learning) model is developed to predict monthly averaged daily GSR prediction where the MODIS data are used to train and evaluate the model and the Particle Swarm Algorithm (PSO) is used as an input selection algorithm. The PSO selected inputs are further transformed into wavelet subseries via non-decimated Discrete Wavelet Transform to unveil the embedded features leading to a hybrid PSO-W-SVR model, seen to outperform the comparative hybrid models. Fourthly, to improve the accuracy of conventional techniques adopted for GSR prediction, Deep Learning (DL) approach based on Deep Belief Network (DBN) and Deep Neural Network (DNN) algorithms are developed to predict the monthly averaged daily GSR prediction using MODIS-based dataset. Finally, the Convolutional Neural Network (CNN) integrated with a Long Short-Term Memory Network (LSTM) model is used to construct a hybrid CLSTM model which is tested to predict the half-hourly GSR values over multiple time-step horizons (i.e., 1-Day, 1-Week, 2-Week, and 1-Month periods). Here, several statistical, Machine Learning and Deep Learning models are adopted to benchmark the proposed DNN and CLSTM models against conventional models (ANN, SaDE-ELM, SVR, DBN). In this doctoral research thesis, a Global Sensitivity Analysis method that attempts to utilise the Gaussian Emulation Machine (GEM-SA) algorithm is employed for a sensitivity analysis of the model predictors. Sensitivity analysis of selected predictors ascertains that the variables: aerosol, cloud, and water vapour parameters used as input parameters for GSR prediction play a significant role and the most important predictors are seen to vary with the geographic location of the tested study site. A suite of alternative models are also developed to evaluate the input datasets classified into El Niño, La Niña and the positive and negative phases of the Indian Ocean Dipole moment. This considers the impact of synoptic-scale climate phenomenon on long-term GSR predictions. A seasonal analysis of models applied at the tested study sites showed that proposed predictive models are an ideal tool over several other comparative models used for GSR prediction. This study also ascertains that an Artificial Intelligence based predictive model integrated with ECMWF reanalysis and MODIS satellite data incorporating physical interactions of the GSR (and its variability) with the other important atmospheric variables can be considered to be an efficient method to predict GSR. In terms of their practical use, the models developed can be used to assist with solar energy modelling and monitoring in solar-rich sites that have diverse climatic conditions, to further support cleaner energy utilization. The outcomes of this doctoral research program are expected to lead to new applications of Artificial Intelligence based predictive tools for GSR prediction, as these tools are able to capture the non-linear relationships between the predictor and the target variable (GSR). The Artificial Intelligence models can therefore assist climate adaptation and energy policymakers to devise new energy management devices not only for Australia but also globally, to enable optimal management of solar energy resources and promote renewable energy to combat current issues of climate change. Additionally, the proposed predictive models may also be applied to other renewable energy areas such as wind, drought, streamflow, flood and electricity demand for prediction

Dynamic data driven investigation of petrophysical and geomechanical properties for reservoir formation evaluation

Petrophysical and geomechanical properties of the formation such as Young’s modulus, bulk modulus, shear modulus, Poisson’s ratio, and porosity provide characteristic description of the hydrocarbon reservoir. It is well-established that static geomechanical properties are good representatives of reservoir formations; however, they are non-continuous along the wellbore, expensive and determining these properties may lead to formation damage. Dynamic geomechanical formation properties from acoustic measurements offer a continuous and non-destructive means to provide a characteristic description of the reservoir formation. In the absence of reliable acoustic measurements of the formation, such as sonic logs, the estimation of the dynamic geomechanical properties becomes challenging. Several techniques like empirical, analytical and intelligent systems have been used to approximate the property estimates. These techniques can also be used to approximate acoustic measurements thus enable dynamic estimation of geomechanical properties. This study intends to explore methodologies and models to dynamically estimate geomechanical properties in the absence of some or all acoustic measurements of the formation. The present work focused on developing empirical and intelligent systems like artificial neural networks (ANN), Gaussian processes (GP), and recurrent neural networks (RNN) to determine the dynamic geomechanical properties. The developed models serve as a cost-effective, reliable, efficient, and robust methods, offering dyanmic geomechanical analysis of the formation. This thesis has five main contributions: (a) a new data-driven empirical model of estimating static Young’s modulus from dynamic Young’s modulus, (b) a new data-driven ANN model for sonic well log prediction, (c) a new data-driven GP model for shear wave transit time prediction, (d) a new dynamic data-driven RNN model for sonic well log reproduction, and (e) an assessment on the ANN as a reliable sonic logging tool

Algorithms for cancer genome data analysis - Learning techniques for ITH modeling and gene fusion classification

L'abstract è presente nell'allegato / the abstract is in the attachmen

Comprehensive review of vision-based fall detection systems

Vision-based fall detection systems have experienced fast development over the last years. To determine the course of its evolution and help new researchers, the main audience of this paper, a comprehensive revision of all published articles in the main scientific databases regarding this area during the last five years has been made. After a selection process, detailed in the Materials and Methods Section, eighty-one systems were thoroughly reviewed. Their characterization and classification techniques were analyzed and categorized. Their performance data were also studied, and comparisons were made to determine which classifying methods best work in this field. The evolution of artificial vision technology, very positively influenced by the incorporation of artificial neural networks, has allowed fall characterization to become more resistant to noise resultant from illumination phenomena or occlusion. The classification has also taken advantage of these networks, and the field starts using robots to make these systems mobile. However, datasets used to train them lack real-world data, raising doubts about their performances facing real elderly falls. In addition, there is no evidence of strong connections between the elderly and the communities of researchers