Information Storage in the Stochastic Ising Model

Most information systems store data by modifying the local state of matter, in the hope that atomic (or sub-atomic) local interactions would stabilize the state for a sufficiently long time, thereby allowing later recovery. In this work we initiate the study of information retention in locally-interacting systems. The evolution in time of the interacting particles is modeled via the stochastic Ising model (SIM). The initial spin configuration $X_0$ serves as the user-controlled input. The output configuration $X_t$ is produced by running $t$ steps of the Glauber chain. Our main goal is to evaluate the information capacity $I_n(t)\triangleq\max_{p_{X_0}}I(X_0;X_t)$ when the time $t$ scales with the size of the system $n$. For the zero-temperature SIM on the two-dimensional $\sqrt{n}\times\sqrt{n}$ grid and free boundary conditions, it is easy to show that $I_n(t) = \Theta(n)$ for $t=O(n)$. In addition, we show that on the order of $\sqrt{n}$ bits can be stored for infinite time in striped configurations. The $\sqrt{n}$ achievability is optimal when $t\to\infty$ and $n$ is fixed. One of the main results of this work is an achievability scheme that stores more than $\sqrt{n}$ bits (in orders of magnitude) for superlinear (in $n$) times. The analysis of the scheme decomposes the system into $\Omega(\sqrt{n})$ independent Z-channels whose crossover probability is found via the (recently rigorously established) Lifshitz law of phase boundary movement. We also provide results for the positive but small temperature regime. We show that an initial configuration drawn according to the Gibbs measure cannot retain more than a single bit for $t\geq e^{cn^{\frac{1}{4}+\epsilon}}$. On the other hand, when scaling time with $\beta$, the stripe-based coding scheme (that stores for infinite time at zero temperature) is shown to retain its bits for time that is exponential in $\beta$

Retrieval behavior and thermodynamic properties of symmetrically diluted Q-Ising neural networks

The retrieval behavior and thermodynamic properties of symmetrically diluted Q-Ising neural networks are derived and studied in replica-symmetric mean-field theory generalizing earlier works on either the fully connected or the symmetrical extremely diluted network. Capacity-gain parameter phase diagrams are obtained for the Q=3, Q=4 and $Q=\infty$ state networks with uniformly distributed patterns of low activity in order to search for the effects of a gradual dilution of the synapses. It is shown that enlarged regions of continuous changeover into a region of optimal performance are obtained for finite stochastic noise and small but finite connectivity. The de Almeida-Thouless lines of stability are obtained for arbitrary connectivity, and the resulting phase diagrams are used to draw conclusions on the behavior of symmetrically diluted networks with other pattern distributions of either high or low activity.Comment: 21 pages, revte

Near-optimal protocols in complex nonequilibrium transformations

The development of sophisticated experimental means to control nanoscale systems has motivated efforts to design driving protocols which minimize the energy dissipated to the environment. Computational models are a crucial tool in this practical challenge. We describe a general method for sampling an ensemble of finite-time, nonequilibrium protocols biased towards a low average dissipation. We show that this scheme can be carried out very efficiently in several limiting cases. As an application, we sample the ensemble of low-dissipation protocols that invert the magnetization of a 2D Ising model and explore how the diversity of the protocols varies in response to constraints on the average dissipation. In this example, we find that there is a large set of protocols with average dissipation close to the optimal value, which we argue is a general phenomenon.Comment: 6 pages and 3 figures plus 4 pages and 5 figures of supplemental materia

Neural Networks retrieving Boolean patterns in a sea of Gaussian ones

Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for associative memory. In this equivalence, weights in the former play as patterns in the latter. As Boltzmann machines usually require real weights to be trained with gradient descent like methods, while Hopfield networks typically store binary patterns to be able to retrieve, the investigation of a mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete (e.g., Boolean) patterns naturally arises. We prove that, in the challenging regime of a high storage of real patterns, where retrieval is forbidden, an extra load of Boolean patterns can still be retrieved, as long as the ratio among the overall load and the network size does not exceed a critical threshold, that turns out to be the same of the standard Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case of a low load of Boolean patterns combining the stochastic stability and Hamilton-Jacobi interpolating techniques. The result can be extended to the high load by a non rigorous but standard replica computation argument.Comment: 16 pages, 1 figur

Synchronous versus sequential updating in the three-state Ising neural network with variable dilution

The three-state Ising neural network with synchronous updating and variable dilution is discussed starting from the appropriate Hamiltonians. The thermodynamic and retrieval properties are examined using replica mean-field theory. Capacity-temperature phase diagrams are derived for several values of the pattern activity and different gradations of dilution, and the information content is calculated. The results are compared with those for sequential updating. The effect of self-coupling is established. Also the dynamics is studied using the generating function technique for both synchronous and sequential updating. Typical flow diagrams for the overlap order parameter are presented. The differences with the signal-to-noise approach are outlined.Comment: 21 pages Latex, 12 eps figures and 1 ps figur