948 research outputs found
LSTM Neural Networks: Input to State Stability and Probabilistic Safety Verification
The goal of this paper is to analyze Long Short Term Memory (LSTM) neural
networks from a dynamical system perspective. The classical recursive equations
describing the evolution of LSTM can be recast in state space form, resulting
in a time-invariant nonlinear dynamical system. A sufficient condition
guaranteeing the Input-to-State (ISS) stability property of this class of
systems is provided. The ISS property entails the boundedness of the output
reachable set of the LSTM. In light of this result, a novel approach for the
safety verification of the network, based on the Scenario Approach, is devised.
The proposed method is eventually tested on a pH neutralization process.Comment: Accepted for Learning for dynamics & control (L4DC) 202
Differential Dynamic Programming for time-delayed systems
Trajectory optimization considers the problem of deciding how to control a
dynamical system to move along a trajectory which minimizes some cost function.
Differential Dynamic Programming (DDP) is an optimal control method which
utilizes a second-order approximation of the problem to find the control. It is
fast enough to allow real-time control and has been shown to work well for
trajectory optimization in robotic systems. Here we extend classic DDP to
systems with multiple time-delays in the state. Being able to find optimal
trajectories for time-delayed systems with DDP opens up the possibility to use
richer models for system identification and control, including recurrent neural
networks with multiple timesteps in the state. We demonstrate the algorithm on
a two-tank continuous stirred tank reactor. We also demonstrate the algorithm
on a recurrent neural network trained to model an inverted pendulum with
position information only.Comment: 7 pages, 6 figures, conference, Decision and Control (CDC), 2016 IEEE
55th Conference o
Analog approach for the eigen-decomposition of positive definite matrices
AbstractThis paper proposes an analog approach for performing the eigen-decomposition of positive definite matrices. We show analytically and by simulations that the proposed circuit is guaranteed to converge to the desired eigenvectors and eigenvalues of positive definite matrices
Neural network training under semidefinite constraints
This paper is concerned with the training of neural networks (NNs) under
semidefinite constraints, which allows for NN training with robustness and
stability guarantees. In particular, we focus on Lipschitz bounds for NNs.
Exploiting the banded structure of the underlying matrix constraint, we set up
an efficient and scalable training scheme for NN training problems of this kind
based on interior point methods. Our implementation allows to enforce Lipschitz
constraints in the training of large-scale deep NNs such as Wasserstein
generative adversarial networks (WGANs) via semidefinite constraints. In
numerical examples, we show the superiority of our method and its applicability
to WGAN training.Comment: to be published in 61st IEEE Conference on Decision and Contro
Fast semidefinite programming with feedforward neural networks
Semidefinite programming is an important optimization task, often used in
time-sensitive applications. Though they are solvable in polynomial time, in
practice they can be too slow to be used in online, i.e. real-time
applications. Here we propose to solve feasibility semidefinite programs using
artificial neural networks. Given the optimization constraints as an input, a
neural network outputs values for the optimization parameters such that the
constraints are satisfied, both for the primal and the dual formulations of the
task. We train the network without having to exactly solve the semidefinite
program even once, thus avoiding the possibly time-consuming task of having to
generate many training samples with conventional solvers. The neural network
method is only inconclusive if both the primal and dual models fail to provide
feasible solutions. Otherwise we always obtain a certificate, which guarantees
false positives to be excluded. We examine the performance of the method on a
hierarchy of quantum information tasks, the Navascu\'es-Pironio-Ac\'in
hierarchy applied to the Bell scenario. We demonstrate that the trained neural
network gives decent accuracy, while showing orders of magnitude increase in
speed compared to a traditional solver
- …