Multiplex visibility graphs to investigate recurrent neural network dynamics

Source at https://doi.org/10.1038/srep44037 .A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods

Automated sleep state classification of wide-field calcium imaging data via multiplex visibility graphs and deep learning

BACKGROUND: Wide-field calcium imaging (WFCI) allows for monitoring of cortex-wide neural dynamics in mice. When applied to the study of sleep, WFCI data are manually scored into the sleep states of wakefulness, non-REM (NREM) and REM by use of adjunct EEG and EMG recordings. However, this process is time-consuming and often suffers from low inter- and intra-rater reliability and invasiveness. Therefore, an automated sleep state classification method that operates on WFCI data alone is needed. NEW METHOD: A hybrid, two-step method is proposed. In the first step, spatial-temporal WFCI data is mapped to multiplex visibility graphs (MVGs). Subsequently, a two-dimensional convolutional neural network (2D CNN) is employed on the MVGs to be classified as wakefulness, NREM and REM. RESULTS: Sleep states were classified with an accuracy of 84% and Cohen\u27s κ of 0.67. The method was also effectively applied on a binary classification of wakefulness/sleep (accuracy=0.82, κ = 0.62) and a four-class wakefulness/sleep/anesthesia/movement classification (accuracy=0.74, κ = 0.66). Gradient-weighted class activation maps revealed that the CNN focused on short- and long-term temporal connections of MVGs in a sleep state-specific manner. Sleep state classification performance when using individual brain regions was highest for the posterior area of the cortex and when cortex-wide activity was considered. COMPARISON WITH EXISTING METHOD: On a 3-hour WFCI recording, the MVG-CNN achieved a κ of 0.65, comparable to a κ of 0.60 corresponding to the human EEG/EMG-based scoring. CONCLUSIONS: The hybrid MVG-CNN method accurately classifies sleep states from WFCI data and will enable future sleep-focused studies with WFCI

Теорія складних мереж та передвісники фінансових крахів

Based on the network paradigm of complexity in the work, a systematic analysis of the dynamics of the largest stock markets in the world and cryptocurrency market has been carried out. According to the algorithms of the visibility graph and recurrence plot, the daily values of stock and crypto indices are converted into a networks and multiplex networks, the spectral and topological properties of which are sensitive to the critical and crisis phenomena of the studied complex systems. This work is the first to investigate the network properties of the crypto index CCI30 and the multiplex network of key cryptocurrencies. It is shown that some of the spectral and topological characteristics can serve as measures of the complexity of the stock and crypto market, and their specific behaviour in the pre-crisis period is used as indicators-precursors of critical phenomena.Виходячи з мережевої парадигми складності роботи, а проведено систематичний аналіз динаміки найбільших фондових ринків світу та ринку криптовалют. Відповідно до алгоритмів графіку видимості та графіку повторень, добові значення фондових та криптоіндексів перетворюються на мережі та мультиплексні мережі, спектральні та топологічні властивості яких чутливі до критичних та кризових явищ досліджуваних складних систем. Ця робота є першою, яка досліджує мережеві властивості криптоіндексу CCI30 та мультиплексної мережі ключових криптовалют. Показано, що деякі спектральні та топологічні характеристики можуть служити мірами складності фондового та крипто-ринку, а їх специфічна поведінка в докризовий період використовується як індикатори-попередники критичних явищ

Visibility graph analysis of intraspinal pressure signal predicts functional outcome in spinal cord injured patients.

To guide management of patients with acute spinal cord injuries, we developed intraspinal pressure monitoring from the injury site. Here, we examine the complex fluctuations in the intraspinal pressure signal using network theory. We analyzed 7,097 hours of intraspinal pressure data from 58 patients with severe cord injuries. Intraspinal pressure signals were split into hourly windows. Each window was mapped into a visibility graph as follows: Vertical bars were drawn at 0.1 Hz representing signal amplitudes. Each bar produced a node, thus totalling 360 nodes per graph. Two nodes were linked with an edge if the straight line through the nodes did not intersect a bar. We computed several topological metrics for each graph including diameter, modularity, eccentricity and small-worldness. Patients were followed up for 20 months on average. Our data show that the topological structure of intraspinal pressure visibility graphs is highly sensitive to pathological events at the injury site including cord compression (high intraspinal pressure), ischemia (low spinal cord perfusion pressure) and deranged autoregulation (high spinal pressure reactivity index). These pathological changes correlate with long graph diameter, high modularity, high eccentricity and reduced small-worldness. In a multivariate logistic regression model, age, neurological status on admission and average node eccentricity were independent predictors of neurological improvement. We conclude that analysis of intraspinal pressure fluctuations after spinal cord injury as graphs, rather than time series, captures clinically important information. Our novel technique may be applied to other signals recorded from injured CNS e.g intracranial pressure, tissue metabolite and oxygen levels

Deep Randomized Neural Networks

Randomized Neural Networks explore the behavior of neural systems where the majority of connections are fixed, either in a stochastic or a deterministic fashion. Typical examples of such systems consist of multi-layered neural network architectures where the connections to the hidden layer(s) are left untrained after initialization. Limiting the training algorithms to operate on a reduced set of weights inherently characterizes the class of Randomized Neural Networks with a number of intriguing features. Among them, the extreme efficiency of the resulting learning processes is undoubtedly a striking advantage with respect to fully trained architectures. Besides, despite the involved simplifications, randomized neural systems possess remarkable properties both in practice, achieving state-of-the-art results in multiple domains, and theoretically, allowing to analyze intrinsic properties of neural architectures (e.g. before training of the hidden layers' connections). In recent years, the study of Randomized Neural Networks has been extended towards deep architectures, opening new research directions to the design of effective yet extremely efficient deep learning models in vectorial as well as in more complex data domains. This chapter surveys all the major aspects regarding the design and analysis of Randomized Neural Networks, and some of the key results with respect to their approximation capabilities. In particular, we first introduce the fundamentals of randomized neural models in the context of feed-forward networks (i.e., Random Vector Functional Link and equivalent models) and convolutional filters, before moving to the case of recurrent systems (i.e., Reservoir Computing networks). For both, we focus specifically on recent results in the domain of deep randomized systems, and (for recurrent models) their application to structured domains

Persistence in complex systems

Persistence is an important characteristic of many complex systems in nature, related to how long the system remains at a certain state before changing to a different one. The study of complex systems' persistence involves different definitions and uses different techniques, depending on whether short-term or long-term persistence is considered. In this paper we discuss the most important definitions, concepts, methods, literature and latest results on persistence in complex systems. Firstly, the most used definitions of persistence in short-term and long-term cases are presented. The most relevant methods to characterize persistence are then discussed in both cases. A complete literature review is also carried out. We also present and discuss some relevant results on persistence, and give empirical evidence of performance in different detailed case studies, for both short-term and long-term persistence. A perspective on the future of persistence concludes the work.This research has been partially supported by the project PID2020-115454GB-C21 of the Spanish Ministry of Science and Innovation (MICINN). This research has also been partially supported by Comunidad de Madrid, PROMINT-CM project (grant ref: P2018/EMT-4366). J. Del Ser would like to thank the Basque Government for its funding support through the EMAITEK and ELKARTEK programs (3KIA project, KK-2020/00049), as well as the consolidated research group MATHMODE (ref. T1294-19). GCV work is supported by the European Research Council (ERC) under the ERC-CoG-2014 SEDAL Consolidator grant (grant agreement 647423)

Persistence in complex systems

Persistence is an important characteristic of many complex systems in nature, related to how long the system remains at a certain state before changing to a different one. The study of complex systems’ persistence involves different definitions and uses different techniques, depending on whether short-term or long-term persistence is considered. In this paper we discuss the most important definitions, concepts, methods, literature and latest results on persistence in complex systems. Firstly, the most used definitions of persistence in short-term and long-term cases are presented. The most relevant methods to characterize persistence are then discussed in both cases. A complete literature review is also carried out. We also present and discuss some relevant results on persistence, and give empirical evidence of performance in different detailed case studies, for both short-term and long-term persistence. A perspective on the future of persistence concludes the work.This research has been partially supported by the project PID2020-115454GB-C21 of the Spanish Ministry of Science and Innovation (MICINN). This research has also been partially supported by Comunidad de Madrid, PROMINT-CM project (grant ref: P2018/EMT-4366). J. Del Ser would like to thank the Basque Government for its funding support through the EMAITEK and ELKARTEK programs (3KIA project, KK-2020/00049), as well as the consolidated research group MATHMODE (ref. T1294-19). GCV work is supported by the European Research Council (ERC) under the ERC-CoG-2014 SEDAL Consolidator grant (grant agreement 647423)