Unstable Attractors: Existence and Robustness in Networks of Oscillators With Delayed Pulse Coupling

We consider unstable attractors; Milnor attractors $A$ such that, for some neighbourhood $U$ of $A$, almost all initial conditions leave $U$. Previous research strongly suggests that unstable attractors exist and even occur robustly (i.e. for open sets of parameter values) in a system modelling biological phenomena, namely in globally coupled oscillators with delayed pulse interactions. In the first part of this paper we give a rigorous definition of unstable attractors for general dynamical systems. We classify unstable attractors into two types, depending on whether or not there is a neighbourhood of the attractor that intersects the basin in a set of positive measure. We give examples of both types of unstable attractor; these examples have non-invertible dynamics that collapse certain open sets onto stable manifolds of saddle orbits. In the second part we give the first rigorous demonstration of existence and robust occurrence of unstable attractors in a network of oscillators with delayed pulse coupling. Although such systems are technically hybrid systems of delay differential equations with discontinuous `firing' events, we show that their dynamics reduces to a finite dimensional hybrid system system after a finite time and hence we can discuss Milnor attractors for this reduced finite dimensional system. We prove that for an open set of phase resetting functions there are saddle periodic orbits that are unstable attractors.Comment: 29 pages, 8 figures,submitted to Nonlinearit

Beyond Basins of Attraction: Quantifying Robustness of Natural Dynamics

Properly designing a system to exhibit favorable natural dynamics can greatly simplify designing or learning the control policy. However, it is still unclear what constitutes favorable natural dynamics and how to quantify its effect. Most studies of simple walking and running models have focused on the basins of attraction of passive limit-cycles and the notion of self-stability. We instead emphasize the importance of stepping beyond basins of attraction. We show an approach based on viability theory to quantify robust sets in state-action space. These sets are valid for the family of all robust control policies, which allows us to quantify the robustness inherent to the natural dynamics before designing the control policy or specifying a control objective. We illustrate our formulation using spring-mass models, simple low dimensional models of running systems. We then show an example application by optimizing robustness of a simulated planar monoped, using a gradient-free optimization scheme. Both case studies result in a nonlinear effective stiffness providing more robustness.Comment: 15 pages. This work has been accepted to IEEE Transactions on Robotics (2019

Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach

Modern neural networks are highly overparameterized, with capacity to substantially overfit to training data. Nevertheless, these networks often generalize well in practice. It has also been observed that trained networks can often be "compressed" to much smaller representations. The purpose of this paper is to connect these two empirical observations. Our main technical result is a generalization bound for compressed networks based on the compressed size. Combined with off-the-shelf compression algorithms, the bound leads to state of the art generalization guarantees; in particular, we provide the first non-vacuous generalization guarantees for realistic architectures applied to the ImageNet classification problem. As additional evidence connecting compression and generalization, we show that compressibility of models that tend to overfit is limited: We establish an absolute limit on expected compressibility as a function of expected generalization error, where the expectations are over the random choice of training examples. The bounds are complemented by empirical results that show an increase in overfitting implies an increase in the number of bits required to describe a trained network.Comment: 16 pages, 1 figure. Accepted at ICLR 201

Robustness of large-scale stochastic matrices to localized perturbations

Upper bounds are derived on the total variation distance between the invariant distributions of two stochastic matrices differing on a subset W of rows. Such bounds depend on three parameters: the mixing time and the minimal expected hitting time on W for the Markov chain associated to one of the matrices; and the escape time from W for the Markov chain associated to the other matrix. These results, obtained through coupling techniques, prove particularly useful in scenarios where W is a small subset of the state space, even if the difference between the two matrices is not small in any norm. Several applications to large-scale network problems are discussed, including robustness of Google's PageRank algorithm, distributed averaging and consensus algorithms, and interacting particle systems.Comment: 12 pages, 4 figure

Sensitivity analysis of oscillator models in the space of phase-response curves: Oscillators as open systems

Oscillator models are central to the study of system properties such as entrainment or synchronization. Due to their nonlinear nature, few system-theoretic tools exist to analyze those models. The paper develops a sensitivity analysis for phase-response curves, a fundamental one-dimensional phase reduction of oscillator models. The proposed theoretical and numerical analysis tools are illustrated on several system-theoretic questions and models arising in the biology of cellular rhythms