Neural networks in geophysical applications

Neural networks are increasingly popular in geophysics. Because they are universal approximators, these tools can approximate any continuous function with an arbitrary precision. Hence, they may yield important contributions to finding solutions to a variety of geophysical applications. However, knowledge of many methods and techniques recently developed to increase the performance and to facilitate the use of neural networks does not seem to be widespread in the geophysical community. Therefore, the power of these tools has not yet been explored to their full extent. In this paper, techniques are described for faster training, better overall performance, i.e., generalization,and the automatic estimation of network size and architecture

The Use of the Random Number Generator and Artificial Intelligence Analysis for Dimensionality Reduction of Follicular Lymphoma Transcriptomic Data

Follicular lymphoma (FL) is one of the most frequent subtypes of non-Hodgkin lymphomas. This research predicted the prognosis of 184 untreated follicular lymphoma patients (LLMPP GSE16131 series), using gene expression data and artificial intelligence (AI) neural networks. A new strategy based on the random number generation was used to create 120 different and independent multilayer perceptron (MLP) solutions, and 22,215 gene probes were ranked according to their averaged normalized importance for predicting the overall survival. After dimensionality reduction, the final neural network architecture included (1) newly identified predictor genes related to cell adhesion and migration, cell signaling, and metabolism (EPB41L4B, MOCOS, SPIN2A, BTD, SRGAP3, CTNS, PRB1, L1CAM, and CEP57); (2) the international prognostic index (IPI); and (3) other relevant immuno-oncology, immune microenvironment, and checkpoint markers (CD163, CSF1R, FOXP3, PDCD1, TNFRSF14 (HVEM), and IL10). The performance of this neural network was good, with an area under the curve (AUC) of 0.89. A comparison with other machine learning techniques (C5 tree, logistic regression, Bayesian network, discriminant analysis, KNN algorithms, LSVM, random trees, SVM, tree-AS, XGBoost linear, XGBoost tree, CHAID, Quest, C&R tree, random forest, and neural network) was also made. In conclusion, the overall survival of follicular lymphoma was predicted with a neural network with high accuracy

Mitigation of Catastrophic Interference in Neural Networks and Ensembles using a Fixed Expansion Layer

Catastrophic forgetting (also known in the literature as catastrophic interference) is the phenomenon by which learning systems exhibit a severe exponential loss of learned information when exposed to relatively small amounts of new training data. This loss of information is not caused by constraints due to the lack of resources available to the learning system, but rather is caused by representational overlap within the learning system and by side-effects of the training methods used. Catastrophic forgetting in auto-associative pattern recognition is a well-studied attribute of most parameterized supervised learning systems. A variation of this phenomenon, in the context of feedforward neural networks, arises when non-stationary inputs lead to loss of previously learned mappings. The majority of the schemes proposed in the literature for mitigating catastrophic forgetting are not data-driven, but rather rely on storage of prior representations of the learning system. We introduce the Fixed Expansion Layer (FEL) feedforward neural network that embeds an expansion layer which sparsely encodes the information contained within the hidden layer, in order to help mitigate forgetting of prior learned representations. The fixed expansion layer approach is generally applicable to feedforward neural networks, as demonstrated by the application of the FEL technique to a recurrent neural network algorithm built on top of a standard feedforward neural network. Additionally, we investigate a novel framework for training ensembles of FEL networks, based on exploiting an information-theoretic measure of diversity between FEL learners, to further control undesired plasticity. The proposed methodology is demonstrated on a several tasks, clearly emphasizing its advantages over existing techniques. The architecture proposed can be applied to address a range of computational intelligence tasks, including classification problems, regression problems and system control