A characterization of the Edge of Criticality in Binary Echo State Networks

Echo State Networks (ESNs) are simplified recurrent neural network models composed of a reservoir and a linear, trainable readout layer. The reservoir is tunable by some hyper-parameters that control the network behaviour. ESNs are known to be effective in solving tasks when configured on a region in (hyper-)parameter space called \emph{Edge of Criticality} (EoC), where the system is maximally sensitive to perturbations hence affecting its behaviour. In this paper, we propose binary ESNs, which are architecturally equivalent to standard ESNs but consider binary activation functions and binary recurrent weights. For these networks, we derive a closed-form expression for the EoC in the autonomous case and perform simulations in order to assess their behavior in the case of noisy neurons and in the presence of a signal. We propose a theoretical explanation for the fact that the variance of the input plays a major role in characterizing the EoC

Robust filtering for stochastic genetic regulatory networks with time-varying delay

This is the post print version of the article. The official published version can be obtained from the link - Copyright 2009 Elsevier LtdThis paper addresses the robust filtering problem for a class of linear genetic regulatory networks (GRNs) with stochastic disturbances, parameter uncertainties and time delays. The parameter uncertainties are assumed to reside in a polytopic region, the stochastic disturbance is state-dependent described by a scalar Brownian motion, and the time-varying delays enter into both the translation process and the feedback regulation process. We aim to estimate the true concentrations of mRNA and protein by designing a linear filter such that, for all admissible time delays, stochastic disturbances as well as polytopic uncertainties, the augmented state estimation dynamics is exponentially mean square stable with an expected decay rate. A delay-dependent linear matrix inequality (LMI) approach is first developed to derive sufficient conditions that guarantee the exponential stability of the augmented dynamics, and then the filter gains are parameterized in terms of the solution to a set of LMIs. Note that LMIs can be easily solved by using standard software packages. A simulation example is exploited in order to illustrate the effectiveness of the proposed design procedures.This work was supported in part by the Biotechnology and Biological Sciences Research Council (BBSRC) of the U.K. under Grants BB/C506264/1 and 100/EGM17735, an International Joint Project sponsored by the Royal Society of the U.K., the Research Grants Council of Hong Kong under Grant HKU 7031/06P, the National Natural Science Foundation of China under Grant 60804028, and the Alexander von Humboldt Foundation of Germany

Stability of networks of nonlinear elements with logical properties

Imperial Users onl

Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review

The paper characterizes classes of functions for which deep learning can be exponentially better than shallow learning. Deep convolutional networks are a special case of these conditions, though weight sharing is not the main reason for their exponential advantage

Measurements, Models, Systems and Design

531 s.

COMPUTER SIMULATION AND COMPUTABILITY OF BIOLOGICAL SYSTEMS

The ability to simulate a biological organism by employing a computer is related to the ability of the computer to calculate the behavior of such a dynamical system, or the "computability" of the system.* However, the two questions of computability and simulation are not equivalent. Since the question of computability can be given a precise answer in terms of recursive functions, automata theory and dynamical systems, it will be appropriate to consider it first. The more elusive question of adequate simulation of biological systems by a computer will be then addressed and a possible connection between the two answers given will be considered. A conjecture is formulated that suggests the possibility of employing an algebraic-topological, "quantum" computer (Baianu, 1971b) for analogous and symbolic simulations of biological systems that may include chaotic processes that are not, in genral, either recursively or digitally computable. Depending on the biological network being modelled, such as the Human Genome/Cell Interactome or a trillion-cell Cognitive Neural Network system, the appropriate logical structure for such simulations might be either the Quantum MV-Logic (QMV) discussed in recent publications (Chiara, 2004, and references cited therein)or Lukasiewicz Logic Algebras that were shown to be isomorphic to MV-logic algebras (Georgescu et al, 2001)