A recurrent neural network with ever changing synapses

A recurrent neural network with noisy input is studied analytically, on the basis of a Discrete Time Master Equation. The latter is derived from a biologically realizable learning rule for the weights of the connections. In a numerical study it is found that the fixed points of the dynamics of the net are time dependent, implying that the representation in the brain of a fixed piece of information (e.g., a word to be recognized) is not fixed in time.Comment: 17 pages, LaTeX, 4 figure

Re-examination of steel frame office buildings in preventing collapse when subject to intense fires

The purpose of the paper is to investigate the extent to which present-day design of steel framed buildings is susceptible to total collapse when subjected to extreme fire events. We select a 50 storey structure in which 2 and 4 adjacent storeys located at different above-ground heights are, in separate scenarios engulfed in raging fires. A total of 8 scenarios are analyzed, employing Newtonian mechanics and realistic energy dissipating properties of H-shaped columns and normal concrete floor slabs possessing secondary (shrinkage and temperature) reinforcement alone. While the present Canadian building code is the basis for our column designs, other standards provide very similar specifications. Although fire proofing is required in virtually all high rise building construction, we are excluding such materials in order to simplify the analyses, but clearly do not advocate its omission – quite the opposite in fact. As well, attributes such as floor beams, partitions and furnishings of every description, all of which would in practice participate in absorbing the kinetic energy of a crush-down upper block are excluded. Despite such a vast array of conservative assumptions, it is shown that partial collapse may occur during crush-down, however, in no case will total collapse be the consequence. These results should provide some comfort to code writers that their requirements should indeed prevent the most catastrophic of failures due to fires

Through-membrane electron-beam lithography for ultrathin membrane applications

We present a technique to fabricate ultrathin (down to 20 nm) uniform electron transparent windows at dedicated locations in a SiN membrane for in situ transmission electron microscopy experiments. An electron-beam (e-beam) resist is spray-coated on the backside of the membrane in a KOH- etched cavity in silicon which is patterned using through-membrane electron-beam lithography. This is a controlled way to make transparent windows in membranes, whilst the topside of the membrane remains undamaged and retains its flatness. Our approach was optimized for MEMS-based heating chips but can be applied to any chip design. We show two different applications of this technique for (1) fabrication of a nanogap electrode by means of electromigration in thin free-standing metal films and (2) making low-noise graphene nanopore devices

Controlling chaos in diluted networks with continuous neurons

Diluted neural networks with continuous neurons and nonmonotonic transfer function are studied, with both fixed and dynamic synapses. A noisy stimulus with periodic variance results in a mechanism for controlling chaos in neural systems with fixed synapses: a proper amount of external perturbation forces the system to behave periodically with the same period as the stimulus.Comment: 11 pages, 8 figure

Damage spreading in the mode-coupling equations for glasses

We examine the problem of damage spreading in the off-equilibrium mode coupling equations. The study is done for the spherical $p$-spin model introduced by Crisanti, Horner and Sommers. For $p>2$ we show the existence of a temperature transition $T_0$ well above any relevant thermodynamic transition temperature. Above $T_0$ the asymptotic damage decays to zero while below $T_0$ it decays to a finite value independent of the initial damage. This transition is stable in the presence of asymmetry in the interactions. We discuss the physical origin of this peculiar phase transition which occurs as a consequence of the non-linear coupling between the damage and the two-time correlation functions.Comment: 5 pages, 2 figures, Revtex fil

Dynamical TAP approach to mean field glassy systems

The Thouless, Anderson, Palmer (TAP) approach to thermodynamics of mean field spin-glasses is generalised to dynamics. A method to compute the dynamical TAP equations is developed and applied to the p-spin spherical model. In this context we show to what extent the dynamics can be represented as an evolution in the free energy landscape. In particular the relationship between the long-time dynamics and the local properties of the free energy landscape shows up explicitly within this approach. Conversely, by an instantaneous normal modes analysis we show that the local properties of the energy landscape seen by the system during its dynamical evolution do not change qualitatively at the dynamical transition.Comment: final version, 21 pages, 1 eps figur

Diluted neural networks with adapting and correlated synapses

We consider the dynamics of diluted neural networks with clipped and adapting synapses. Unlike previous studies, the learning rate is kept constant as the connectivity tends to infinity: the synapses evolve on a time scale intermediate between the quenched and annealing limits and all orders of synaptic correlations must be taken into account. The dynamics is solved by mean-field theory, the order parameter for synapses being a function. We describe the effects, in the double dynamics, due to synaptic correlations.Comment: 6 pages, 3 figures. Accepted for publication in PR

Damage spreading transition in glasses: a probe for the ruggedness of the configurational landscape

We consider damage spreading transitions in the framework of mode-coupling theory. This theory describes relaxation processes in glasses in the mean-field approximation which are known to be characterized by the presence of an exponentially large number of meta-stable states. For systems evolving under identical but arbitrarily correlated noises we demonstrate that there exists a critical temperature $T_0$ which separates two different dynamical regimes depending on whether damage spreads or not in the asymptotic long-time limit. This transition exists for generic noise correlations such that the zero damage solution is stable at high-temperatures being minimal for maximal noise correlations. Although this dynamical transition depends on the type of noise correlations we show that the asymptotic damage has the good properties of an dynamical order parameter such as: 1) Independence on the initial damage; 2) Independence on the class of initial condition and 3) Stability of the transition in the presence of asymmetric interactions which violate detailed balance. For maximally correlated noises we suggest that damage spreading occurs due to the presence of a divergent number of saddle points (as well as meta-stable states) in the thermodynamic limit consequence of the ruggedness of the free energy landscape which characterizes the glassy state. These results are then compared to extensive numerical simulations of a mean-field glass model (the Bernasconi model) with Monte Carlo heat-bath dynamics. The freedom of choosing arbitrary noise correlations for Langevin dynamics makes damage spreading a interesting tool to probe the ruggedness of the configurational landscape.Comment: 25 pages, 13 postscript figures. Paper extended to include cross-correlation

Hierarchical Self-Programming in Recurrent Neural Networks

We study self-programming in recurrent neural networks where both neurons (the `processors') and synaptic interactions (`the programme') evolve in time simultaneously, according to specific coupled stochastic equations. The interactions are divided into a hierarchy of $L$ groups with adiabatically separated and monotonically increasing time-scales, representing sub-routines of the system programme of decreasing volatility. We solve this model in equilibrium, assuming ergodicity at every level, and find as our replica-symmetric solution a formalism with a structure similar but not identical to Parisi's $L$-step replica symmetry breaking scheme. Apart from differences in details of the equations (due to the fact that here interactions, rather than spins, are grouped into clusters with different time-scales), in the present model the block sizes $m_i$ of the emerging ultrametric solution are not restricted to the interval $[0,1]$, but are independent control parameters, defined in terms of the noise strengths of the various levels in the hierarchy, which can take any value in [0,\infty\ket. This is shown to lead to extremely rich phase diagrams, with an abundance of first-order transitions especially when the level of stochasticity in the interaction dynamics is chosen to be low.Comment: 53 pages, 19 figures. Submitted to J. Phys.