Thermal Equilibrium with the Wiener Potential: Testing the Replica Variational Approximation

We consider the statistical mechanics of a classical particle in a one-dimensional box subjected to a random potential which constitutes a Wiener process on the coordinate axis. The distribution of the free energy and all correlation functions of the Gibbs states may be calculated exactly as a function of the box length and temperature. This allows for a detailed test of results obtained by the replica variational approximation scheme. We show that this scheme provides a reasonable estimate of the averaged free energy. Furthermore our results shed more light on the validity of the concept of approximate ultrametricity which is a central assumption of the replica variational method.Comment: 6 pages, 1 file LaTeX2e generating 2 eps-files for 2 figures automaticall

On-Line AdaTron Learning of Unlearnable Rules

We study the on-line AdaTron learning of linearly non-separable rules by a simple perceptron. Training examples are provided by a perceptron with a non-monotonic transfer function which reduces to the usual monotonic relation in a certain limit. We find that, although the on-line AdaTron learning is a powerful algorithm for the learnable rule, it does not give the best possible generalization error for unlearnable problems. Optimization of the learning rate is shown to greatly improve the performance of the AdaTron algorithm, leading to the best possible generalization error for a wide range of the parameter which controls the shape of the transfer function.)Comment: RevTeX 17 pages, 8 figures, to appear in Phys.Rev.

Inference and Optimization of Real Edges on Sparse Graphs - A Statistical Physics Perspective

Inference and optimization of real-value edge variables in sparse graphs are studied using the Bethe approximation and replica method of statistical physics. Equilibrium states of general energy functions involving a large set of real edge-variables that interact at the network nodes are obtained in various cases. When applied to the representative problem of network resource allocation, efficient distributed algorithms are also devised. Scaling properties with respect to the network connectivity and the resource availability are found, and links to probabilistic Bayesian approximation methods are established. Different cost measures are considered and algorithmic solutions in the various cases are devised and examined numerically. Simulation results are in full agreement with the theory.Comment: 21 pages, 10 figures, major changes: Sections IV to VII updated, Figs. 1 to 3 replace

Equilibration through local information exchange in networks

We study the equilibrium states of energy functions involving a large set of real variables, defined on the links of sparsely connected networks, and interacting at the network nodes, using the cavity and replica methods. When applied to the representative problem of network resource allocation, an efficient distributed algorithm is devised, with simulations showing full agreement with theory. Scaling properties with the network connectivity and the resource availability are found.Comment: v1: 7 pages, 1 figure, v2: 4 pages, 2 figures, simplified analysis and more organized results, v3: minor change

Statistical Mechanics of Learning: A Variational Approach for Real Data

Using a variational technique, we generalize the statistical physics approach of learning from random examples to make it applicable to real data. We demonstrate the validity and relevance of our method by computing approximate estimators for generalization errors that are based on training data alone.Comment: 4 pages, 2 figure

Perceptron capacity revisited: classification ability for correlated patterns

In this paper, we address the problem of how many randomly labeled patterns can be correctly classified by a single-layer perceptron when the patterns are correlated with each other. In order to solve this problem, two analytical schemes are developed based on the replica method and Thouless-Anderson-Palmer (TAP) approach by utilizing an integral formula concerning random rectangular matrices. The validity and relevance of the developed methodologies are shown for one known result and two example problems. A message-passing algorithm to perform the TAP scheme is also presented

Charge Symmetry Breaking and QCD

Charge symmetry breaking (CSB) in the strong interaction occurs because of the difference between the masses of the up and down quarks. The use of effective field theories allows us to follow this influence of confined quarks in hadronic and nuclear systems. The progress in observing and understanding CSB is reviewed with particular attention to the recent successful observations of CSB in measurements involving the production of a single neutral pion and to the related theoretical progress.Comment: 41 pages, 10 figures, for Nov. 2006 edition Annual Review of Nuclear and Particle Physic

Generalizing with perceptrons in case of structured phase- and pattern-spaces

We investigate the influence of different kinds of structure on the learning behaviour of a perceptron performing a classification task defined by a teacher rule. The underlying pattern distribution is permitted to have spatial correlations. The prior distribution for the teacher coupling vectors itself is assumed to be nonuniform. Thus classification tasks of quite different difficulty are included. As learning algorithms we discuss Hebbian learning, Gibbs learning, and Bayesian learning with different priors, using methods from statistics and the replica formalism. We find that the Hebb rule is quite sensitive to the structure of the actual learning problem, failing asymptotically in most cases. Contrarily, the behaviour of the more sophisticated methods of Gibbs and Bayes learning is influenced by the spatial correlations only in an intermediate regime of $\alpha$, where $\alpha$ specifies the size of the training set. Concerning the Bayesian case we show, how enhanced prior knowledge improves the performance.Comment: LaTeX, 32 pages with eps-figs, accepted by J Phys

Implicitly Constrained Semi-Supervised Least Squares Classification

We introduce a novel semi-supervised version of the least squares classifier. This implicitly constrained least squares (ICLS) classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-supervised methods, our approach does not introduce explicit additional assumptions into the objective function, but leverages implicit assumptions already present in the choice of the supervised least squares classifier. We show this approach can be formulated as a quadratic programming problem and its solution can be found using a simple gradient descent procedure. We prove that, in a certain way, our method never leads to performance worse than the supervised classifier. Experimental results corroborate this theoretical result in the multidimensional case on benchmark datasets, also in terms of the error rate.Comment: 12 pages, 2 figures, 1 table. The Fourteenth International Symposium on Intelligent Data Analysis (2015), Saint-Etienne, Franc

Optimal Resource Allocation in Random Networks with Transportation Bandwidths

We apply statistical physics to study the task of resource allocation in random sparse networks with limited bandwidths for the transportation of resources along the links. Useful algorithms are obtained from recursive relations. Bottlenecks emerge when the bandwidths are small, causing an increase in the fraction of idle links. For a given total bandwidth per node, the efficiency of allocation increases with the network connectivity. In the high connectivity limit, we find a phase transition at a critical bandwidth, above which clusters of balanced nodes appear, characterised by a profile of homogenized resource allocation similar to the Maxwell's construction.Comment: 28 pages, 11 figure