29 research outputs found
Statistical mechanics of sparse generalization and model selection
One of the crucial tasks in many inference problems is the extraction of
sparse information out of a given number of high-dimensional measurements. In
machine learning, this is frequently achieved using, as a penality term, the
norm of the model parameters, with for efficient dilution. Here
we propose a statistical-mechanics analysis of the problem in the setting of
perceptron memorization and generalization. Using a replica approach, we are
able to evaluate the relative performance of naive dilution (obtained by
learning without dilution, following by applying a threshold to the model
parameters), dilution (which is frequently used in convex optimization)
and dilution (which is optimal but computationally hard to implement).
Whereas both diluted approaches clearly outperform the naive approach, we
find a small region where works almost perfectly and strongly outperforms
the simpler to implement dilution.Comment: 18 pages, 9 eps figure
Contamination source inference in water distribution networks
We study the inference of the origin and the pattern of contamination in
water distribution networks. We assume a simplified model for the dyanmics of
the contamination spread inside a water distribution network, and assume that
at some random location a sensor detects the presence of contaminants. We
transform the source location problem into an optimization problem by
considering discrete times and a binary contaminated/not contaminated state for
the nodes of the network. The resulting problem is solved by Mixed Integer
Linear Programming. We test our results on random networks as well as in the
Modena city network
Stability of the replica symmetric solution in diluted perceptron learning
We study the role played by the dilution in the average behavior of a
perceptron model with continuous coupling with the replica method. We analyze
the stability of the replica symmetric solution as a function of the dilution
field for the generalization and memorization problems. Thanks to a Gardner
like stability analysis we show that at any fixed ratio between the
number of patterns M and the dimension N of the perceptron (),
there exists a critical dilution field above which the replica symmetric
ansatz becomes unstable.Comment: Stability of the solution in arXiv:0907.3241, 13 pages, (some typos
corrected
Gauge-free cluster variational method by maximal messages and moment matching
We present an implementation of the cluster variational method (CVM) as a message passing algorithm. The kind of message passing algorithm used for CVM, usually named generalized belief propagation (GBP), is a generalization of the belief propagation algorithm in the same way that CVM is a generalization of the Bethe approximation for estimating the partition function. However, the connection between fixed points of GBP and the extremal points of the CVM free energy is usually not a one-to-one correspondence because of the existence of a gauge transformation involving the GBP messages. Our contribution is twofold. First, we propose a way of defining messages (fields) in a generic CVM approximation, such that messages arrive on a given region from all its ancestors, and not only from its direct parents, as in the standard parent-to-child GBP. We call this approach maximal messages. Second, we focus on the case of binary variables, reinterpreting the messages as fields enforcing the consistency between the moments of the local (marginal) probability distributions. We provide a precise rule to enforce all consistencies, avoiding any redundancy, that would otherwise lead to a gauge transformation on the messages. This moment matching method is gauge free, i.e., it guarantees that the resulting GBP is not gauge invariant. We apply our maximal messages and moment matching GBP to obtain an analytical expression for the critical temperature of the Ising model in general dimensions at the level of plaquette CVM. The values obtained outperform Bethe estimates, and are comparable with loop corrected belief propagation equations. The method allows for a straightforward generalization to disordered systems
Opinion formation by belief propagation: A heuristic to identify low-credible sources of information
With social media, the flow of uncertified information is constantly
increasing, with the risk that more people will trust low-credible information
sources. To design effective strategies against this phenomenon, it is of
paramount importance to understand how people end up believing one source
rather than another. To this end, we propose a realistic and cognitively
affordable heuristic mechanism for opinion formation inspired by the well-known
belief propagation algorithm. In our model, an individual observing a network
of information sources must infer which of them are reliable and which are not.
We study how the individual's ability to identify credible sources, and hence
to form correct opinions, is affected by the noise in the system, intended as
the amount of disorder in the relationships between the information sources in
the network. We find numerically and analytically that there is a critical
noise level above which it is impossible for the individual to detect the
nature of the sources. Moreover, by comparing our opinion formation model with
existing ones in the literature, we show under what conditions people's
opinions can be reliable. Overall, our findings imply that the increasing
complexity of the information environment is a catalyst for misinformation
channels