Convergence under dynamical thresholds with delays

Necessary and sufficient conditions are obtained for the existence of a globally asymptotically stable equilibrium of a class of delay differential equations modeling the action of a neuron with dynamical threshold effects

Learning in Single Hidden Layer Feedforward Network Models: Backpropagation in a Real World Application

Leaming in neural networks has attracted considerable interest in recent years. Our focus is on learning in single hidden layer feedforward networks which is posed as a search in the network parameter space for a network that minimizes an additive error function of statistically independent examples. In this contribution, we review first the class of single hidden layer feedforward networks and characterize the learning process in such networks from a statistical point of view. Then we describe the backpropagation procedure, the leading case of gradient descent learning algorithms for the class of networks considered here, as well as an efficient heuristic modification. Finally, we analyse the applicability of these learning methods to the problem of predicting interregional telecommunication flows. Particular emphasis is laid on the engineering judgment, first, in choosing appropriate values for the tunable parameters, second, on the decision whether to train the network by epoch or by pattern (random approximation), and, third, on the overfitting problem. In addition, the analysis shows that the neural network model whether using either epoch-based or pattern-based stochastic approximation outperforms the classical regression approach to modelling telecommunication flows. (authors' abstract)Series: Discussion Papers of the Institute for Economic Geography and GIScienc

Estimating neural signal dynamics in the human brain

Although brain imaging methods are highly effective for localizing the effects of neural activation throughout the human brain in terms of the blood oxygenation level dependent (BOLD) response, there is currently no way to estimate the underlying neural signal dynamics in generating the BOLD response in each local activation region (except for processes slower than the BOLD time course). Knowledge of the neural signal is critical if spatial mapping is to progress to the analysis of dynamic information flow through the cortical networks as the brain performs its tasks. We introduce an analytic approach that provides a new level of conceptualization and specificity in the study of brain processing by non-invasive methods. This technique allows us to use brain imaging methods to determine the dynamics of local neural population responses to their native temporal resolution throughout the human brain, with relatively narrow confidence intervals on many response properties. The ability to characterize local neural dynamics in the human brain represents a significant enhancement of brain imaging capabilities, with potential applications ranging from general cognitive studies to assessment of neuropathologies

Lifelong learning of human actions with deep neural network self-organization

Lifelong learning is fundamental in autonomous robotics for the acquisition and fine-tuning of knowledge through experience. However, conventional deep neural models for action recognition from videos do not account for lifelong learning but rather learn a batch of training data with a predefined number of action classes and samples. Thus, there is the need to develop learning systems with the ability to incrementally process available perceptual cues and to adapt their responses over time. We propose a self-organizing neural architecture for incrementally learning to classify human actions from video sequences. The architecture comprises growing self-organizing networks equipped with recurrent neurons for processing time-varying patterns. We use a set of hierarchically arranged recurrent networks for the unsupervised learning of action representations with increasingly large spatiotemporal receptive fields. Lifelong learning is achieved in terms of prediction-driven neural dynamics in which the growth and the adaptation of the recurrent networks are driven by their capability to reconstruct temporally ordered input sequences. Experimental results on a classification task using two action benchmark datasets show that our model is competitive with state-of-the-art methods for batch learning also when a significant number of sample labels are missing or corrupted during training sessions. Additional experiments show the ability of our model to adapt to non-stationary input avoiding catastrophic interference

A Methodology for the Development of Recurrent Networks for Sequence Processing Tasks

Artificial neural networks are increasingly being used for dealing with real world applications. Many of these (e.g. speech recognition) are based on an ability to perform sequence processing. A class of artificial neural networks, known as recurrent networks, have architectures which incorporate feedback connections. This in turn allows the development of a memory mechanism to allow sequence processing to occur. A large number of recurrent network models have been developed, together with modifications of existing architectures and learning rules. However there has been comparatively little effort made to compare the performance of these models relative to each other. Such comparative studies would show differences in performance between networks and allow an examination of what features of a network give rise to desirable behaviours such as faster learning and superior generalisation ability. This thesis describes the results of a number of existing comparative studies and the results of new research. Three different recurrent networks, both in their original form and with modifications, are tested with four different sequence processing tasks. The results of this research clearly show that recurrent networks vary widely in terms of their performance and lead to a methodology based on the following conclusions: </br

Persistence in complex systems

Persistence is an important characteristic of many complex systems in nature, related to how long the system remains at a certain state before changing to a different one. The study of complex systems' persistence involves different definitions and uses different techniques, depending on whether short-term or long-term persistence is considered. In this paper we discuss the most important definitions, concepts, methods, literature and latest results on persistence in complex systems. Firstly, the most used definitions of persistence in short-term and long-term cases are presented. The most relevant methods to characterize persistence are then discussed in both cases. A complete literature review is also carried out. We also present and discuss some relevant results on persistence, and give empirical evidence of performance in different detailed case studies, for both short-term and long-term persistence. A perspective on the future of persistence concludes the work.This research has been partially supported by the project PID2020-115454GB-C21 of the Spanish Ministry of Science and Innovation (MICINN). This research has also been partially supported by Comunidad de Madrid, PROMINT-CM project (grant ref: P2018/EMT-4366). J. Del Ser would like to thank the Basque Government for its funding support through the EMAITEK and ELKARTEK programs (3KIA project, KK-2020/00049), as well as the consolidated research group MATHMODE (ref. T1294-19). GCV work is supported by the European Research Council (ERC) under the ERC-CoG-2014 SEDAL Consolidator grant (grant agreement 647423)

Persistence in complex systems

Persistence is an important characteristic of many complex systems in nature, related to how long the system remains at a certain state before changing to a different one. The study of complex systems’ persistence involves different definitions and uses different techniques, depending on whether short-term or long-term persistence is considered. In this paper we discuss the most important definitions, concepts, methods, literature and latest results on persistence in complex systems. Firstly, the most used definitions of persistence in short-term and long-term cases are presented. The most relevant methods to characterize persistence are then discussed in both cases. A complete literature review is also carried out. We also present and discuss some relevant results on persistence, and give empirical evidence of performance in different detailed case studies, for both short-term and long-term persistence. A perspective on the future of persistence concludes the work.This research has been partially supported by the project PID2020-115454GB-C21 of the Spanish Ministry of Science and Innovation (MICINN). This research has also been partially supported by Comunidad de Madrid, PROMINT-CM project (grant ref: P2018/EMT-4366). J. Del Ser would like to thank the Basque Government for its funding support through the EMAITEK and ELKARTEK programs (3KIA project, KK-2020/00049), as well as the consolidated research group MATHMODE (ref. T1294-19). GCV work is supported by the European Research Council (ERC) under the ERC-CoG-2014 SEDAL Consolidator grant (grant agreement 647423)