Perturbation theory and singular perturbations for input-to-state multistable systems on manifolds

We consider the notion of Input-to-State Multistability, which generalizes ISS to nonlinear systems evolving on Riemannian manifolds and possessing a finite number of compact, globally attractive, invariant sets, which in addition satisfy a specific condition of acyclicity. We prove that a parameterized family of dynamical systems whose solutions converge to those of a limiting system inherits such Input-to-State Multistability property from the limiting system in a semi-global practical fashion. A similar result is also established for singular perturbation models whose boundary-layer subsystem is uniformly asymptotically stable and whose reduced subsystem is Input-to-State Multistable. Known results in the theory of perturbations, singular perturbations, averaging, and highly oscillatory control systems, are here generalized to the multistable setting by replacing the classical asymptotic stability requirement of a single invariant set with attractivity and acyclicity of a decomposable invariant one

Nonlinear resonance and excitability in interconnected systems

Engineering design amounts to develop components and interconnect them to obtain a desired behaviour. While in the context of equilibrium dynamics there is a well-developed theory that can account for robustness and optimality in this process, we still lack a corresponding methodology for nonequilibrium dynamics and in particular oscillatory behaviours. With the aim of fostering such a theory, this thesis studies two basic interconnections in the contexts of nonlinear resonance and excitability, two phenomena with the potential of encompassing a large number of applications. The first interconnection is considered in the context of vibration absorption. It corresponds to coupling two Duffing oscillators, the prototypical example of nonlinear resonator. Of primary interest is the frequency response of the system, which quantifies the behaviour in presence of harmonic forces. The analysis focuses on how isolated families of solutions appear and merge with a main one. Using singularity theory it is possible to organise these solutions in the space of parameters and delimit their presence through numerical methods. The second interconnection studied in this dissertation appears in the context of excitable circuits. Combining a fast excitable system and a slower oscillatory system that share a similar structure naturally leads to bursting. The resulting system has a slow-fast structure that can be leveraged in the analysis. The first step of this analysis is a novel slow-fast model of bistability between a rest state and a spiking attractor. Following this, the analysis moves to the complete interconnection, and in particular on how it can generate different patterns of bursting activity

Noise-induced escape in an excitable system

We consider the stochastic dynamics of escape in an excitable system, the FitzHugh-Nagumo (FHN) neuronal model, for different classes of excitability. We discuss, first, the threshold structure of the FHN model as an example of a system without a saddle state. We then develop a nonlinear (nonlocal) stability approach based on the theory of large fluctuations, including a finite-noise correction, to describe noise-induced escape in the excitable regime. We show that the threshold structure is revealed via patterns of most probable (optimal) fluctuational paths. The approach allows us to estimate the escape rate and the exit location distribution. We compare the responses of a monostable resonator and monostable integrator to stochastic input signals and to a mixture of periodic and stochastic stimuli. Unlike the commonly used local analysis of the stable state, our nonlocal approach based on optimal paths yields results that are in good agreement with direct numerical simulations of the Langevin equation

Samoorganizacija u spregnutim ekscitabilnim sistemima: sadejstvo višestrukih vremenskih skala i šuma

The dynamics of complex systems typically involves multiple spatial and temporal scales, while emergent phenomena are often associated with critical transitions in which a small parameter variation causes a sudden shift to a qualitatively different regime. In the vicinity of such transitions, complex systems are highly sensitive to external perturbations, potentially resulting in dynamical switching between different (meta)stable states. Such behavior is typical for many biological systems consisting of coupled excitable units. In neuronal systems, for instance, self-organization is influenced by the interplay between noise from diverse sources and a multi-timescale structure arising from both local and coupling dynamics. The present thesis is devoted to several types of self-organized dynamics in coupled stochastic excitable systems with multiple timescale dynamics. The excitable behavior of single units is well understood, in terms of both the nonlinear threshold-like response to external perturbations and the characteristic non-monotonous response to noise, embodied by different resonant phenomena. However, the excitable behavior of coupled systems, as a new paradigm of emergent dynamics, involves a number of fundamental open problems, including how interactions modify local dynamics resulting in excitable behavior at the level of the coupled system, and how the interplay of multiscale dynamics and noise gives rise to switching dynamics and resonant phenomena. This thesis comprises a systematic approach to addressing these issues, consisting of three complementary lines of research. In particular, within the first line of research, we have extended the notion of excitability to coupled systems, considering the examples of a small motif of locally excitable units and a population of stochastic neuronal maps. In the case of the motif, we have classified different types of excitable responses and, by applying elements of singular perturbation theory, identified what determines the motif’s threshold-like response. Regarding the neuronal population, we have established the concept of macroscopic excitability whereby an entire population of excitable units acts like an excitable element itself. To examine the stability viiand bifurcations of the macroscopic excitability state, as well as the associated stimulusresponse relationship, we have derived the first effective mean-field model for the collective dynamics of coupled stochastic maps.Dinamika kompleksnih sistema se tipiˇcno odigrava na nekoliko prostornih i vremenskih skala, pri ˇcemu su emergentni fenomeni ˇcesto povezani sa kritiˇcnim prelazima, pri kojima mala promena vrednosti parametra izaziva naglu i kvalitativnu promenu dinamiˇckog režima. U blizini takvih prelaza, kompleksni sistemi su vrlo osetljivi na eksterne peturbacije, što može izazvati dinamiku alterniranja (switching) izmedu razliˇcitih (meta)stabilnih stanja. ¯ Takvo ponašanje je tipiˇcno za mnoštvo bioloških sistema saˇcinjenih od spregnutih ekscitabilnih jedinica, medu kojima su i neuronski sistemi, kod kojih na samoorganizaciju utiˇcu koe- ¯ fekti šuma iz raznolikih izvora i višestrukosti vremenskih skala koja potiˇce od lokalne dinamike i dinamike interakcija. Ova disertacija je posve´cena prouˇcavanju nekoliko vrsta samoorganizuju´ce dinamike u spregnutim stohastiˇckim ekscitabilnim sistemima sa dinamikom koja se odvija na višestrukim vremenskim skalama (multiscale dinamika). Ekscitabilno ponašanje pojedinaˇcnih jedinica je detaljno istraženo, kako u pogledu nelinearnog pragovskog (threshold-like) odgovora na eksterne perturbacije, tako i u pogledu karakteristiˇcnog nemonotonog odgovora na šum, manifestovanog kroz razne rezonantne fenomene. Medutim, pri razmatranju ¯ ekscitabilnog ponašanja spregnutih sistema kao nove paradigme emergentne dinamike, na fundamentalnom nivou postoje brojna otvorena pitanja, ukljuˇcuju´ci kako interakcije modifikuju lokalnu dinamiku rezultuju´ci ekscitabilnoš´cu na nivou spregnutog sistema, kao i kako sadejstvo multiscale dinamike i šuma dovodi do switching-a i rezonantnih fenomena. U ovoj disertaciji, saˇcinjenoj od tri komplementarne linije istraživanja, sistematiˇcno pristupamo traženju odgovora na navedena pitanja. U sklopu prve linije istraživanja, proširili smo koncept ekscitabilnosti na spregnute sisteme, razmatraju´ci primere malog motiva saˇcinjenog od lokalno ekscitabilnih jedinica i populacije stohastiˇckih neuronskih mapa. U sluˇcaju motiva, klasifikovali smo razliˇcite vrste ekscitabilnih odgovora i pokazali šta odreduje pragovsko ponašanje, primenivši elemente ¯ teorije singularnih perturbacija. U sluˇcaju populacije, uveli smo koncept makroskopske ixekscitabilnosti pri kojoj se cela populacija ekscitabilnih jedinica ponaša kao ekscitabilni element. Kako bismo ispitali stabilnost i bifurkacije stanja makroskopske ekscitabilnosti, kao i odgovor sistema na perturbaciju, izveli smo prvi efektivni model srednjeg polja (mean-field) za kolektivnu dinamiku spregnutih stohastičkih mapa

Parameter-sweeping techniques for temporal dynamics of neuronal systems: case study of Hindmarsh-Rose model

Background: Development of effective and plausible numerical tools is an imperative task for thorough studies of nonlinear dynamics in life science applications. Results: We have developed a complementary suite of computational tools for twoparameter screening of dynamics in neuronal models. We test a ‘brute-force’ effectiveness of neuroscience plausible techniques specifically tailored for the examination of temporal characteristics, such duty cycle of bursting, interspike interval, spike number deviation in the phenomenological Hindmarsh-Rose model of a bursting neuron and compare the results obtained by calculus-based tools for evaluations of an entire spectrum of Lyapunov exponents broadly employed in studies of nonlinear systems. Conclusions: We have found that the results obtained either way agree exceptionally well, and can identify and differentiate between various fine structures of complex dynamics and underlying global bifurcations in this exemplary model. Our future planes are to enhance the applicability of this computational suite for understanding of polyrhythmic bursting patterns and their functional transformations in small networks

Interpreting multi-stable behaviour in input-driven recurrent neural networks

Recurrent neural networks (RNNs) are computational models inspired by the brain. Although RNNs stand out as state-of-the-art machine learning models to solve challenging tasks as speech recognition, handwriting recognition, language translation, and others, they are plagued by the so-called vanishing/exploding gradient issue. This prevents us from training RNNs with the aim of learning long term dependencies in sequential data. Moreover, a problem of interpretability affects these models, known as the ``black-box issue'' of RNNs. We attempt to open the black box by developing a mechanistic interpretation of errors occurring during the computation. We do this from a dynamical system theory perspective, specifically building on the notion of Excitable Network Attractors. Our methodology is effective at least for those tasks where a number of attractors and a switching pattern between them must be learned. RNNs can be seen as massively large nonlinear dynamical systems driven by external inputs. When it comes to analytically investigate RNNs, often in the literature the input-driven property is neglected or dropped in favour of tight constraints on the input driving the dynamics, which do not match the reality of RNN applications. Trying to bridge this gap, we framed RNNs dynamics driven by generic input sequences in the context of nonautonomous dynamical system theory. This brought us to enquire deeply into a fundamental principle established for RNNs known as the echo state property (ESP). In particular, we argue that input-driven RNNs can be reliable computational models even without satisfying the classical ESP formulation. We prove a sort of input-driven fixed point theorem and exploit it to (i) demonstrate the existence and uniqueness of a global attracting solution for strongly (in amplitude) input-driven RNNs, (ii) deduce the existence of multiple responses for certain input signals which can be reliably exploited for computational purposes, and (iii) study the stability of attracting solutions w.r.t. input sequences. Finally, we highlight the active role of the input in determining qualitative changes in the RNN dynamics, e.g. the number of stable responses, in contrast to commonly known qualitative changes due to variations of model parameters

Rate-induced critical transitions

This thesis focuses on rate-induced critical transitions or tipping points (R-tipping points), where the system undergoes a critical transition if the time-varying external conditions vary faster than some critical rate. Such a critical transition is usually a sudden and unexpected change of the system state. The change can be either irreversible: a permanent tipping point with no return to the original state, or reversible: a temporary tipping point with self-recovery back to the original state, both of which may cause significant consequences in applications. Indeed, R-tipping is an ubiquitous nonlinear phenomenon in nature that remains largely unexplored by the scientists. From a mathematical viewpoint, it is a genuine nonautonomous instability that cannot be explained by the classical (autonomous) bifurcation theory and requires an alternative approach. The first part of the thesis focuses on a mathematical framework for R-tipping in systems of nonautonomous differential equations, where the nonautonomous terms representing time-varying external conditions decay asymptotically. In particular, special compactification techniques for asymptotically autonomous systems are developed to simplify analysis of R-tipping. In the second part of the thesis, the main concepts of edge states and thresholds are introduced to define the R-tipping phenomenon. Then, simple testable criteria for the occurrence of reversible and irreversible R-tipping in arbitrary dimension are given. This part extends the previous results on irreversible R-tipping in one dimension. The third part of the thesis identifies canonical examples of R-tipping based on the system dimension, timescales and the threshold type. These examples are relatively simple low-dimensional nonlinear systems that capture different R-tipping mechanisms. R-tipping analysis of canonical examples, which is underpinned by the compactification framework developed in the second part, reveals intricate R-tipping diagrams with multiple critical rates and transitions between different types of R-tipping

Complex Dynamics in Dedicated / Multifunctional Neural Networks and Chaotic Nonlinear Systems

We study complex behaviors arising in neuroscience and other nonlinear systems by combining dynamical systems analysis with modern computational approaches including GPU parallelization and unsupervised machine learning. To gain insights into the behaviors of brain networks and complex central pattern generators (CPGs), it is important to understand the dynamical principles regulating individual neurons as well as the basic structural and functional building blocks of neural networks. In the first section, we discuss how symbolic methods can help us analyze neural dynamics such as bursting, tonic spiking and chaotic mixed-mode oscillations in various models of individual neurons, the bifurcations that underlie transitions between activity types, as well as emergent network phenomena through synergistic interactions seen in realistic neural circuits, such as network bursting from non-intrinsic bursters. The second section is focused on the origin and coexistence of multistable rhythms in oscillatory neural networks of inhibitory coupled cells. We discuss how network connectivity and intrinsic properties of the cells affect the dynamics, and how even simple circuits can exhibit a variety of mono/multi-stable rhythms including pacemakers, half-center oscillators, multiple traveling-waves, fully synchronous states, as well as various chimeras. Our analyses can help generate verifiable hypotheses for neurophysiological experiments on central pattern generators. In the last section, we demonstrate the inter-disciplinary nature of this research through the applications of these techniques to identify the universal principles governing both simple and complex dynamics, and chaotic structure in diverse nonlinear systems. Using a classical example from nonlinear laser optics, we elaborate on the multiplicity and self-similarity of key organizing structures in 2D parameter space such as homoclinic and heteroclinic bifurcation curves, Bykov T-point spirals, and inclination flips. This is followed by detailed computational reconstructions of the spatial organization and 3D embedding of bifurcation surfaces, parametric saddles, and isolated closed curves (isolas). The generality of our modeling approaches could lead to novel methodologies and nonlinear science applications in biological, medical and engineering systems

Response theory and critical phenomena for noisy interacting systems

In this thesis we investigate critical phenomena for ensembles of identical interacting agents, namely weakly interacting diffusions. These interacting systems undergo two qualitatively different scenarios of criticality, critical transitions and phase transitions. The former situation conforms to the classical tipping point phenomenology that is observed in finite dimensional systems and originates from a setting where negative feedbacks that stabilise the system progressively loose their efficiency, resulting in amplified fluctuations and correlation properties of the system. On the other hand, \textit{phase transitions} stem from the complex interplay between the agents' own dynamics, the coupling among them and the noise, leading to macroscopic emergent behaviour of the system, and are only observed in the thermodynamic limit. Classically, \textit{phase transitions} are investigated with the use of suitable macroscopic variables, called order parameters, acting as effective reaction coordinates that capture the relevant features of the macroscopic dynamics. However, identifying an order parameter is not always possible. In this thesis we adopt a complementary point of view, based on Linear Response theory, to investigate such critical phenomena. We are able to identify the conditions leading either to a critical transition or a phase transition in terms of spectral properties of suitable response operators. We associate critical phenomena to settings where the response of the system breaks down. In particular, we are able to characterise these critical scenarios as settings where the complex valued susceptibility of the system develops a non analytical behaviour for real values of frequencies, resulting in a macroscopic resonance of the system. We provide multiple paradigmatic examples of equilibrium and nonequilibrium phase transitions where we are able to prove mathematically and numerically the clear signature of a singular behaviour of the susceptibility at the phase transition as the thermodynamic limit is reached. Being associated to spectral properties of suitable operators describing either correlation or response properties, these resonant phenomena do not depend on the specific details of the applied forcing nor on the observable under investigation, allowing one to bypass the problem of the identification of the order parameter for the system.Open Acces