7 research outputs found
A Behavioral Approach to Robust Machine Learning
Machine learning is revolutionizing almost all fields of science and technology and has been proposed as a pathway to solving many previously intractable problems such as autonomous driving and other complex robotics tasks. While the field has demonstrated impressive results on certain problems, many of these results have not translated to applications in physical systems, partly due to the cost of system fail-
ure and partly due to the difficulty of ensuring reliable and robust model behavior. Deep neural networks, for instance, have simultaneously demonstrated both incredible performance in game playing and image processing, and remarkable fragility. This combination of high average performance and a catastrophically bad worst case performance presents a serious danger as deep neural networks are currently being
used in safety critical tasks such as assisted driving.
In this thesis, we propose a new approach to training models that have built in robustness guarantees. Our approach to ensuring stability and robustness of the models trained is distinct from prior methods; where prior methods learn a model and then attempt to verify robustness/stability, we directly optimize over sets of
models where the necessary properties are known to hold.
Specifically, we apply methods from robust and nonlinear control to the analysis and synthesis of recurrent neural networks, equilibrium neural networks, and recurrent equilibrium neural networks. The techniques developed allow us to enforce properties such as incremental stability, incremental passivity, and incremental l2 gain bounds / Lipschitz bounds. A central consideration in the development of our model sets is the difficulty of fitting models. All models can be placed in the image of a convex set, or even R^N , allowing useful properties to be easily imposed during the training procedure via simple interior point methods, penalty methods, or unconstrained optimization.
In the final chapter, we study the problem of learning networks of interacting models with guarantees that the resulting networked system is stable and/or monotone, i.e., the order relations between states are preserved. While our approach to learning in this chapter is similar to the previous chapters, the model set that we propose has a separable structure that allows for the scalable and distributed identification of large-scale systems via the alternating directions method of multipliers (ADMM)
Recommended from our members
Networked Dynamical Systems: Privacy, Control, and Cognition
Many natural and man-made systems, ranging from thenervous system to power and transportation grids to societies, exhibitdynamic behaviors that evolve over a sparse and complex network. This networked aspect raises significant challenges and opportunities for the identification, analysis, and control of such dynamic behaviors. While some of these challenges emanate from the networked aspect \emph{per se} (such as the sparsity of connections between system components and the interplay between nodal \emph{communication} and network dynamics), various challenges arise from the specific application areas (such as privacy concerns in cyber-physical systems or the need for \emph{scalable} algorithm designs due to the large size of various biological and engineered networks). On the other hand, networked systems provide significant opportunities and allow for performance and robustness levels that are far beyond reach for centralized systems, with examples ranging from the Internet (of Things) to the smart grid and the brain. This dissertation aims to address several of these challenges and harness these opportunities. The dissertation is divided into three parts. In the first part, we study privacy concerns whose resolution is vital for the utility of networked cyber-physical systems. We study the problems of average consensus and convex optimization as two principal distributed computations occurring over networks and design algorithm with rigorous privacy guarantees that provide a \emph{best achievable} tradeoff between network utility and privacy. In the second part, we analyze networks with resource constraints. More specifically, we study three problems of stabilization under communication (bandwidth and latency) limitations in sensing and actuation, optimal time-varying control scheduling problem under limited number of actuators and control energy, and the structure identification problem of under-sensed networks (i.e., networks with latent nodes). Finally in the last part, we focus on the intersection of networked dynamical systems and neuroscience and draw connections between brain network dynamics and two extensively studied but yet not fully understood neuro-cognitive phenomena: goal-driven selective attention and neural oscillations. Using a novel axiomatic approach, we establish these connections in the form of necessary and/or sufficient conditions on the network structure that match the network output trajectories with experimentally observed brain activity
Model identification of a network as compressing sensing
In many applications, it is of interest to derive information about the topology and the internal connections of multiple dynamical systems interacting together. Examples can be found in fields as diverse as Economics, Neuroscience and Biochemistry. The paper deals with the problem of deriving a descriptive model of a network with no a-priori knowledge on its topology. It is assumed that the network nodes are passively observed and data are collected in the form of time series. The underlying structure is then determined by the non-zero entries of a “sparse Wiener filter”. We cast the problem as the optimization of a quadratic cost function, where a set of parameters are used to operate a trade-off between accuracy and complexity in the final model