21 research outputs found
Evolving the Curve
Evolutionary algorithms are used to generate personal contact networks, modelling human populations, that are most likely to match a given epidemic profile. The Susceptible-Infected-Removed (SIR) model is used and also expanded upon to allow for an extended period of infection, termed the SIIR model. The networks generated for each of these models are thoroughly evaluated for their ability to match nine different epidemic profiles. The addition of the SIIR model showed that the model of infection has an impact on the networks generated. For the SIR and SIIR models, these differences were relatively minor in most cases.Natural Sciences and Engineering Research Council of Canad
Training Physics-Informed Neural Networks via Multi-Task Optimization for Traffic Density Prediction
Physics-informed neural networks (PINNs) are a newly emerging research
frontier in machine learning, which incorporate certain physical laws that
govern a given data set, e.g., those described by partial differential
equations (PDEs), into the training of the neural network (NN) based on such a
data set. In PINNs, the NN acts as the solution approximator for the PDE while
the PDE acts as the prior knowledge to guide the NN training, leading to the
desired generalization performance of the NN when facing the limited
availability of training data. However, training PINNs is a non-trivial task
largely due to the complexity of the loss composed of both NN and physical law
parts. In this work, we propose a new PINN training framework based on the
multi-task optimization (MTO) paradigm. Under this framework, multiple
auxiliary tasks are created and solved together with the given (main) task,
where the useful knowledge from solving one task is transferred in an adaptive
mode to assist in solving some other tasks, aiming to uplift the performance of
solving the main task. We implement the proposed framework and apply it to
train the PINN for addressing the traffic density prediction problem.
Experimental results demonstrate that our proposed training framework leads to
significant performance improvement in comparison to the traditional way of
training the PINN.Comment: accepted by the 2023 IEEE International Joint Conference on Neural
Networks (IJCNN 2023
Evolving Self-taught Neural Networks: The Baldwin Effect and the Emergence of Intelligence
The so-called Baldwin Effect generally says how learning,
as a form of ontogenetic adaptation, can influence the process of
phylogenetic adaptation, or evolution. This idea has also been taken
into computation in which evolution and learning are used as computational
metaphors, including evolving neural networks. This paper
presents a technique called evolving self-taught neural networks –
neural networks that can teach themselves without external supervision
or reward. The self-taught neural network is intrinsically motivated.
Moreover, the self-taught neural network is the product of the
interplay between evolution and learning. We simulate a multi-agent
system in which neural networks are used to control autonomous
agents. These agents have to forage for resources and compete for
their own survival. Experimental results show that the interaction between
evolution and the ability to teach oneself in self-taught neural
networks outperform evolution and self-teaching alone. More
specifically, the emergence of an intelligent foraging strategy is also
demonstrated through that interaction. Indications for future work on
evolving neural networks are also presented
Learning to extrapolate using continued fractions: Predicting the critical temperature of superconductor materials
In Artificial Intelligence we often seek to identify an unknown target
function of many variables giving a limited set of instances
with where is a
domain of interest. We refer to as the training set and the final quest is
to identify the mathematical model that approximates this target function for
new ; with the set with (i.e. thus testing the model generalisation). However, for some
applications, the main interest is approximating well the unknown function on a
larger domain that contains . In cases involving the design of new
structures, for instance, we may be interested in maximizing ; thus, the
model derived from alone should also generalize well in for samples
with values of larger than the largest observed in . In that sense, the
AI system would provide important information that could guide the design
process, e.g., using the learned model as a surrogate function to design new
lab experiments.
We introduce a method for multivariate regression based on iterative fitting
of a continued fraction by incorporating additive spline models. We compared it
with established methods such as AdaBoost, Kernel Ridge, Linear Regression,
Lasso Lars, Linear Support Vector Regression, Multi-Layer Perceptrons, Random
Forests, Stochastic Gradient Descent and XGBoost. We tested the performance on
the important problem of predicting the critical temperature of superconductors
based on physical-chemical characteristics.Comment: Submitted to IEEE Transactions on Artificial Intelligence (TAI
Parallel Exploration via Negatively Correlated Search
Effective exploration is a key to successful search. The recently proposed
Negatively Correlated Search (NCS) tries to achieve this by parallel
exploration, where a set of search processes are driven to be negatively
correlated so that different promising areas of the search space can be visited
simultaneously. Various applications have verified the advantages of such novel
search behaviors. Nevertheless, the mathematical understandings are still
lacking as the previous NCS was mostly devised by intuition. In this paper, a
more principled NCS is presented, explaining that the parallel exploration is
equivalent to the explicit maximization of both the population diversity and
the population solution qualities, and can be optimally obtained by partially
gradient descending both models with respect to each search process. For
empirical assessments, the reinforcement learning tasks that largely demand
exploration ability is considered. The new NCS is applied to the popular
reinforcement learning problems, i.e., playing Atari games, to directly train a
deep convolution network with 1.7 million connection weights in the
environments with uncertain and delayed rewards. Empirical results show that
the significant advantages of NCS over the compared state-of-the-art methods
can be highly owed to the effective parallel exploration ability
Byzantine-Resilient Learning Beyond Gradients: Distributing Evolutionary Search
Modern machine learning (ML) models are capable of impressive performances.
However, their prowess is not due only to the improvements in their
architecture and training algorithms but also to a drastic increase in
computational power used to train them.
Such a drastic increase led to a growing interest in distributed ML, which in
turn made worker failures and adversarial attacks an increasingly pressing
concern. While distributed byzantine resilient algorithms have been proposed in
a differentiable setting, none exist in a gradient-free setting.
The goal of this work is to address this shortcoming. For that, we introduce
a more general definition of byzantine-resilience in ML - the
\textit{model-consensus}, that extends the definition of the classical
distributed consensus. We then leverage this definition to show that a general
class of gradient-free ML algorithms - ()-Evolutionary Search - can
be combined with classical distributed consensus algorithms to generate
gradient-free byzantine-resilient distributed learning algorithms. We provide
proofs and pseudo-code for two specific cases - the Total Order Broadcast and
proof-of-work leader election.Comment: 10 pages, 4 listings, 2 theorem
Improving land cover classification using genetic programming for feature construction
Batista, J. E., Cabral, A. I. R., Vasconcelos, M. J. P., Vanneschi, L., & Silva, S. (2021). Improving land cover classification using genetic programming for feature construction. Remote Sensing, 13(9), [1623]. https://doi.org/10.3390/rs13091623Genetic programming (GP) is a powerful machine learning (ML) algorithm that can produce readable white-box models. Although successfully used for solving an array of problems in different scientific areas, GP is still not well known in the field of remote sensing. The M3GP algorithm, a variant of the standard GP algorithm, performs feature construction by evolving hyperfeatures from the original ones. In this work, we use the M3GP algorithm on several sets of satellite images over different countries to create hyperfeatures from satellite bands to improve the classification of land cover types. We add the evolved hyperfeatures to the reference datasets and observe a significant improvement of the performance of three state-of-the-art ML algorithms (decision trees, random forests, and XGBoost) on multiclass classifications and no significant effect on the binary classifications. We show that adding the M3GP hyperfeatures to the reference datasets brings better results than adding the well-known spectral indices NDVI, NDWI, and NBR. We also compare the performance of the M3GP hyperfeatures in the binary classification problems with those created by other feature construction methods such as FFX and EFS.publishersversionpublishe