Thouless-Anderson-Palmer equation for analog neural network with temporally fluctuating white synaptic noise

Effects of synaptic noise on the retrieval process of associative memory neural networks are studied from the viewpoint of neurobiological and biophysical understanding of information processing in the brain. We investigate the statistical mechanical properties of stochastic analog neural networks with temporally fluctuating synaptic noise, which is assumed to be white noise. Such networks, in general, defy the use of the replica method, since they have no energy concept. The self-consistent signal-to-noise analysis (SCSNA), which is an alternative to the replica method for deriving a set of order parameter equations, requires no energy concept and thus becomes available in studying networks without energy functions. Applying the SCSNA to stochastic network requires the knowledge of the Thouless-Anderson-Palmer (TAP) equation which defines the deterministic networks equivalent to the original stochastic ones. The study of the TAP equation which is of particular interest for the case without energy concept is very few, while it is closely related to the SCSNA in the case with energy concept. This paper aims to derive the TAP equation for networks with synaptic noise together with a set of order parameter equations by a hybrid use of the cavity method and the SCSNA.Comment: 13 pages, 3 figure

Oscillator neural network model with distributed native frequencies

We study associative memory of an oscillator neural network with distributed native frequencies. The model is based on the use of the Hebb learning rule with random patterns ($\xi_i^{\mu}=\pm 1$), and the distribution function of native frequencies is assumed to be symmetric with respect to its average. Although the system with an extensive number of stored patterns is not allowed to get entirely synchronized, long time behaviors of the macroscopic order parameters describing partial synchronization phenomena can be obtained by discarding the contribution from the desynchronized part of the system. The oscillator network is shown to work as associative memory accompanied by synchronized oscillations. A phase diagram representing properties of memory retrieval is presented in terms of the parameters characterizing the native frequency distribution. Our analytical calculations based on the self-consistent signal-to-noise analysis are shown to be in excellent agreement with numerical simulations, confirming the validity of our theoretical treatment.Comment: 9 pages, revtex, 6 postscript figures, to be published in J. Phys.

Response Functions Improving Performance in Analog Attractor Neural Networks

In the context of attractor neural networks, we study how the equilibrium analog neural activities, reached by the network dynamics during memory retrieval, may improve storage performance by reducing the interferences between the recalled pattern and the other stored ones. We determine a simple dynamics that stabilizes network states which are highly correlated with the retrieved pattern, for a number of stored memories that does not exceed $\alpha_{\star} N$, where $\alpha_{\star}\in[0,0.41]$ depends on the global activity level in the network and $N$ is the number of neurons.Comment: 13 pages (with figures), LaTex (RevTex), to appear on Phys.Rev.E (RC

Influence of synaptic depression on memory storage capacity

Synaptic efficacy between neurons is known to change within a short time scale dynamically. Neurophysiological experiments show that high-frequency presynaptic inputs decrease synaptic efficacy between neurons. This phenomenon is called synaptic depression, a short term synaptic plasticity. Many researchers have investigated how the synaptic depression affects the memory storage capacity. However, the noise has not been taken into consideration in their analysis. By introducing "temperature", which controls the level of the noise, into an update rule of neurons, we investigate the effects of synaptic depression on the memory storage capacity in the presence of the noise. We analytically compute the storage capacity by using a statistical mechanics technique called Self Consistent Signal to Noise Analysis (SCSNA). We find that the synaptic depression decreases the storage capacity in the case of finite temperature in contrast to the case of the low temperature limit, where the storage capacity does not change

Bi-stability of mixed states in neural network storing hierarchical patterns

We discuss the properties of equilibrium states in an autoassociative memory model storing hierarchically correlated patterns (hereafter, hierarchical patterns). We will show that symmetric mixed states (hereafter, mixed states) are bi-stable on the associative memory model storing the hierarchical patterns in a region of the ferromagnetic phase. This means that the first-order transition occurs in this ferromagnetic phase. We treat these contents with a statistical mechanical method (SCSNA) and by computer simulation. Finally, we discuss a physiological implication of this model. Sugase et al. analyzed the time-course of the information carried by the firing of face-responsive neurons in the inferior temporal cortex. We also discuss the relation between the theoretical results and the physiological experiments of Sugase et al.Comment: 18 pages, 6 figure

An associative network with spatially organized connectivity

We investigate the properties of an autoassociative network of threshold-linear units whose synaptic connectivity is spatially structured and asymmetric. Since the methods of equilibrium statistical mechanics cannot be applied to such a network due to the lack of a Hamiltonian, we approach the problem through a signal-to-noise analysis, that we adapt to spatially organized networks. The conditions are analyzed for the appearance of stable, spatially non-uniform profiles of activity with large overlaps with one of the stored patterns. It is also shown, with simulations and analytic results, that the storage capacity does not decrease much when the connectivity of the network becomes short range. In addition, the method used here enables us to calculate exactly the storage capacity of a randomly connected network with arbitrary degree of dilution.Comment: 27 pages, 6 figures; Accepted for publication in JSTA

Transient dynamics for sequence processing neural networks

An exact solution of the transient dynamics for a sequential associative memory model is discussed through both the path-integral method and the statistical neurodynamics. Although the path-integral method has the ability to give an exact solution of the transient dynamics, only stationary properties have been discussed for the sequential associative memory. We have succeeded in deriving an exact macroscopic description of the transient dynamics by analyzing the correlation of crosstalk noise. Surprisingly, the order parameter equations of this exact solution are completely equivalent to those of the statistical neurodynamics, which is an approximation theory that assumes crosstalk noise to obey the Gaussian distribution. In order to examine our theoretical findings, we numerically obtain cumulants of the crosstalk noise. We verify that the third- and fourth-order cumulants are equal to zero, and that the crosstalk noise is normally distributed even in the non-retrieval case. We show that the results obtained by our theory agree with those obtained by computer simulations. We have also found that the macroscopic unstable state completely coincides with the separatrix.Comment: 21 pages, 4 figure

Associative memory storing an extensive number of patterns based on a network of oscillators with distributed natural frequencies in the presence of external white noise

We study associative memory based on temporal coding in which successful retrieval is realized as an entrainment in a network of simple phase oscillators with distributed natural frequencies under the influence of white noise. The memory patterns are assumed to be given by uniformly distributed random numbers on $[0,2\pi)$ so that the patterns encode the phase differences of the oscillators. To derive the macroscopic order parameter equations for the network with an extensive number of stored patterns, we introduce the effective transfer function by assuming the fixed-point equation of the form of the TAP equation, which describes the time-averaged output as a function of the effective time-averaged local field. Properties of the networks associated with synchronization phenomena for a discrete symmetric natural frequency distribution with three frequency components are studied based on the order parameter equations, and are shown to be in good agreement with the results of numerical simulations. Two types of retrieval states are found to occur with respect to the degree of synchronization, when the size of the width of the natural frequency distribution is changed.Comment: published in Phys. Rev.

Implementation of earlier antibiotic administration in patients with severe sepsis and septic shock in Japan: a descriptive analysis of a prospective observational study.

BACKGROUND: Time to antibiotic administration is a key element in sepsis care; however, it is difficult to implement sepsis care bundles. Additionally, sepsis is different from other emergent conditions including acute coronary syndrome, stroke, or trauma. We aimed to describe the association between time to antibiotic administration and outcomes in patients with severe sepsis and septic shock in Japan. METHODS: This prospective observational study enrolled 1184 adult patients diagnosed with severe sepsis based on the Sepsis-2 criteria and admitted to 59 intensive care units (ICUs) in Japan between January 1, 2016, and March 31, 2017, as the sepsis cohort of the Focused Outcomes Research in Emergency Care in Acute Respiratory Distress Syndrome, Sepsis and Trauma (FORECAST) study. We compared the characteristics and in-hospital mortality of patients administered with antibiotics at varying durations after sepsis recognition, i.e., 0-60, 61-120, 121-180, 181-240, 241-360, and 361-1440 min, and estimated the impact of antibiotic timing on risk-adjusted in-hospital mortality using the generalized estimating equation model (GEE) with an exchangeable, within-group correlation matrix, with "hospital" as the grouping variable. RESULTS: Data from 1124 patients in 54 hospitals were used for analyses. Of these, 30.5% and 73.9% received antibiotics within 1 h and 3 h, respectively. Overall, the median time to antibiotic administration was 102 min [interquartile range (IQR), 55-189]. Compared with patients diagnosed in the emergency department [90 min (IQR, 48-164 min)], time to antibiotic administration was shortest in patients diagnosed in ICUs [60 min (39-180 min)] and longest in patients transferred from wards [120 min (62-226)]. Overall crude mortality was 23.4%, where patients in the 0-60 min group had the highest mortality (28.0%) and a risk-adjusted mortality rate [28.7% (95% CI 23.3-34.1%)], whereas those in the 61-120 min group had the lowest mortality (20.2%) and risk-adjusted mortality rates [21.6% (95% CI 16.5-26.6%)]. Differences in mortality were noted only between the 0-60 min and 61-120 min groups. CONCLUSIONS: We could not find any association between earlier antibiotic administration and reduction in in-hospital mortality in patients with severe sepsis