Asymptotic Normality of the Maximum Pseudolikelihood Estimator for Fully Visible Boltzmann Machines

Boltzmann machines (BMs) are a class of binary neural networks for which there have been numerous proposed methods of estimation. Recently, it has been shown that in the fully visible case of the BM, the method of maximum pseudolikelihood estimation (MPLE) results in parameter estimates which are consistent in the probabilistic sense. In this article, we investigate the properties of MPLE for the fully visible BMs further, and prove that MPLE also yields an asymptotically normal parameter estimator. These results can be used to construct confidence intervals and to test statistical hypotheses. We support our theoretical results by showing that the estimator behaves as expected in a simulation study

Financial interaction networks inferred from traded volumes

In order to use the advanced inference techniques available for Ising models, we transform complex data (real vectors) into binary strings, by local averaging and thresholding. This transformation introduces parameters, which must be varied to characterize the behaviour of the system. The approach is illustrated on financial data, using three inference methods -- equilibrium, synchronous and asynchronous inference -- to construct functional connections between stocks. We show that the traded volume information is enough to obtain well known results about financial markets, which use however the presumably richer price information: collective behaviour ("market mode") and strong interactions within industry sectors. Synchronous and asynchronous Ising inference methods give results which are coherent with equilibrium ones, and more detailed since the obtained interaction networks are directed.Comment: 14 pages, 6 figure

Exact mean field inference in asymmetric kinetic Ising systems

We develop an elementary mean field approach for fully asymmetric kinetic Ising models, which can be applied to a single instance of the problem. In the case of the asymmetric SK model this method gives the exact values of the local magnetizations and the exact relation between equal-time and time-delayed correlations. It can also be used to solve efficiently the inverse problem, i.e. determine the couplings and local fields from a set of patterns, also in cases where the fields and couplings are time-dependent. This approach generalizes some recent attempts to solve this dynamical inference problem, which were valid in the limit of weak coupling. It provides the exact solution to the problem also in strongly coupled problems. This mean field inference can also be used as an efficient approximate method to infer the couplings and fields in problems which are not infinite range, for instance in diluted asymmetric spin glasses.Comment: 10 pages, 7 figure

Intrinsic limitations of inverse inference in the pairwise Ising spin glass

We analyze the limits inherent to the inverse reconstruction of a pairwise Ising spin glass based on susceptibility propagation. We establish the conditions under which the susceptibility propagation algorithm is able to reconstruct the characteristics of the network given first- and second-order local observables, evaluate eventual errors due to various types of noise in the originally observed data, and discuss the scaling of the problem with the number of degrees of freedom

Beyond inverse Ising model: structure of the analytical solution for a class of inverse problems

I consider the problem of deriving couplings of a statistical model from measured correlations, a task which generalizes the well-known inverse Ising problem. After reminding that such problem can be mapped on the one of expressing the entropy of a system as a function of its corresponding observables, I show the conditions under which this can be done without resorting to iterative algorithms. I find that inverse problems are local (the inverse Fisher information is sparse) whenever the corresponding models have a factorized form, and the entropy can be split in a sum of small cluster contributions. I illustrate these ideas through two examples (the Ising model on a tree and the one-dimensional periodic chain with arbitrary order interaction) and support the results with numerical simulations. The extension of these methods to more general scenarios is finally discussed.Comment: 15 pages, 6 figure

Dynamics and Performance of Susceptibility Propagation on Synthetic Data

We study the performance and convergence properties of the Susceptibility Propagation (SusP) algorithm for solving the Inverse Ising problem. We first study how the temperature parameter (T) in a Sherrington-Kirkpatrick model generating the data influences the performance and convergence of the algorithm. We find that at the high temperature regime (T>4), the algorithm performs well and its quality is only limited by the quality of the supplied data. In the low temperature regime (T<4), we find that the algorithm typically does not converge, yielding diverging values for the couplings. However, we show that by stopping the algorithm at the right time before divergence becomes serious, good reconstruction can be achieved down to T~2. We then show that dense connectivity, loopiness of the connectivity, and high absolute magnetization all have deteriorating effects on the performance of the algorithm. When absolute magnetization is high, we show that other methods can be work better than SusP. Finally, we show that for neural data with high absolute magnetization, SusP performs less well than TAP inversion.Comment: 9 pages, 7 figure

Dynamic message-passing approach for kinetic spin models with reversible dynamics

A method to approximately close the dynamic cavity equations for synchronous reversible dynamics on a locally tree-like topology is presented. The method builds on $(a)$ a graph expansion to eliminate loops from the normalizations of each step in the dynamics, and $(b)$ an assumption that a set of auxilary probability distributions on histories of pairs of spins mainly have dependencies that are local in time. The closure is then effectuated by projecting these probability distributions on $n$-step Markov processes. The method is shown in detail on the level of ordinary Markov processes ($n=1$), and outlined for higher-order approximations ($n>1$). Numerical validations of the technique are provided for the reconstruction of the transient and equilibrium dynamics of the kinetic Ising model on a random graph with arbitrary connectivity symmetry.Comment: 6 pages, 4 figure