Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit

Restricted Boltzmann machines (RBMs) constitute one of the main models for machine statistical inference and they are widely employed in Artificial Intelligence as powerful tools for (deep) learning. However, in contrast with countless remarkable practical successes, their mathematical formalization has been largely elusive: from a statistical-mechanics perspective these systems display the same (random) Gibbs measure of bi-partite spin-glasses, whose rigorous treatment is notoriously difficult. In this work, beyond providing a brief review on RBMs from both the learning and the retrieval perspectives, we aim to contribute to their analytical investigation, by considering two distinct realizations of their weights (i.e., Boolean and Gaussian) and studying the properties of their related free energies. More precisely, focusing on a RBM characterized by digital couplings, we first extend the Pastur-Shcherbina-Tirozzi method (originally developed for the Hopfield model) to prove the self-averaging property for the free energy, over its quenched expectation, in the infinite volume limit, then we explicitly calculate its simplest approximation, namely its annealed bound. Next, focusing on a RBM characterized by analogical weights, we extend Guerra's interpolating scheme to obtain a control of the quenched free-energy under the assumption of replica symmetry: we get self-consistencies for the order parameters (in full agreement with the existing Literature) as well as the critical line for ergodicity breaking that turns out to be the same obtained in AGS theory. As we discuss, this analogy stems from the slow-noise universality. Finally, glancing beyond replica symmetry, we analyze the fluctuations of the overlaps for an estimate of the (slow) noise affecting the retrieval of the signal, and by a stability analysis we recover the Aizenman-Contucci identities typical of glassy systems.Comment: 21 pages, 1 figur

Central limit theorem for fluctuations of linear eigenvalue statistics of large random graphs. Diluted regime

We study the linear eigenvalue statistics of large random graphs in the regimes when the mean number of edges for each vertex tends to infinity. We prove that for a rather wide class of test functions the fluctuations of linear eigenvalue statistics converges in distribution to a Gaussian random variable with zero mean and variance which coincides with "non gaussian" part of the Wigner ensemble variance.Comment: 19 page

Phoneme-retrieval; voice recognition; vowels recognition

A phoneme-retrieval technique is proposed, which is due to the particular way of the construction of the network. An initial set of neurons is given. The number of these neurons is approximately equal to the number of typical structures of the data. For example if the network is built for voice retrieval then the number of neurons must be equal to the number of characteristic phonemes of the alphabet of the language spoken by the social group to which the particular person belongs. Usually this task is very complicated and the network can depend critically on the samples used for the learning. If the network is built for image retrieval then it works only if the data to be retrieved belong to a particular set of images. If the network is built for voice recognition it works only for some particular set of words. A typical example is the words used for the flight of airplanes. For example a command like the "airplane should make a turn of 120 degrees towards the east" can be easily recognized by the network if a suitable learning procedure is used.Comment: 10 page

Central limit theorem for fluctuations of linear eigenvalue statistics of large random graphs

We consider the adjacency matrix $A$ of a large random graph and study fluctuations of the function $f_n(z,u)=\frac{1}{n}\sum_{k=1}^n\exp\{-uG_{kk}(z)\}$ with $G(z)=(z-iA)^{-1}$. We prove that the moments of fluctuations normalized by $n^{-1/2}$ in the limit $n\to\infty$ satisfy the Wick relations for the Gaussian random variables. This allows us to prove central limit theorem for $\hbox{Tr}G(z)$ and then extend the result on the linear eigenvalue statistics $\hbox{Tr}\phi(A)$ of any function $\phi:\mathbb{R}\to\mathbb{R}$ which increases, together with its first two derivatives, at infinity not faster than an exponential.Comment: 22 page

Scattering of Lower Hybrid Waves in a Magnetized Plasma

In this paper, the Maxwell equations for the electric field in a cold magnetized plasma in the half-space of x≥0 cm are solved. The boundary conditions for the electric field include a pointwise source at the plane x=0 cm, the derivatives of the electric field that are zero statV/cm2 at x=0 cm, and the field with all its derivatives that are zero at infinity. The solution is explored in terms of the Laplace transform in x and the Fourier transform in y-z directions. The expressions of the field components are obtained by the inverse Laplace transform and the inverse Fourier transform. The saddle-point technique and power expansion have been used for evaluating the inverse Fourier transform. The model represents the propagation of a lower hybrid wave generated by a pointwise antenna located at the boundary of the plasma. Here, the antenna is the boundary condition. The validation of the model is performed assuming that the electric field component Ey=0 statV/cm and by comparing it with the model of electromagnetic waves generated by a local small antenna located near the boundary of a tokamak, and an experiment is suggested

Magnetic Force-Free Theory: Nonlinear Case

In this paper, a theory of force-free magnetic field useful for explaining the formation of convex closed sets, bounded by a magnetic separatrix in the plasma, is developed. This question is not new and has been addressed by many authors. Force-free magnetic fields appear in many laboratory and astrophysical plasmas. These fields are defined by the solution of the problem ∇×B=ΛB with some field conditions B∂Ω on the boundary ∂Ω of the plasma region. In many physical situations, it has been noticed that Λ is not constant but may vary in the domain Ω giving rise to many different interesting physical situations. We set Λ=Λ(ψ) with ψ being the poloidal magnetic flux function. Then, an analytic method, based on a first-order expansion of ψ with respect to a small parameter α, is developed. The Grad–Shafranov equation for ψ is solved by expanding the solution in the eigenfunctions of the zero-order operator. An analytic expression for the solution is obtained deriving results on the transition through resonances, the amplification with respect to the gun inflow. Thus, the formation of spheromaks or protosphera structure of the plasma is determined in the case of nonconstant Λ