Laplace's rule of succession in information geometry

Laplace's "add-one" rule of succession modifies the observed frequencies in a sequence of heads and tails by adding one to the observed counts. This improves prediction by avoiding zero probabilities and corresponds to a uniform Bayesian prior on the parameter. The canonical Jeffreys prior corresponds to the "add-one-half" rule. We prove that, for exponential families of distributions, such Bayesian predictors can be approximated by taking the average of the maximum likelihood predictor and the \emph{sequential normalized maximum likelihood} predictor from information theory. Thus in this case it is possible to approximate Bayesian predictors without the cost of integrating or sampling in parameter space

Parametric Fokker-Planck equation

We derive the Fokker-Planck equation on the parametric space. It is the Wasserstein gradient flow of relative entropy on the statistical manifold. We pull back the PDE to a finite dimensional ODE on parameter space. Some analytical example and numerical examples are presented

Hamilton-Jacobi Theory and Information Geometry

Recently, a method to dynamically define a divergence function $D$ for a given statistical manifold $(\mathcal{M}\,,g\,,T)$ by means of the Hamilton-Jacobi theory associated with a suitable Lagrangian function $\mathfrak{L}$ on $T\mathcal{M}$ has been proposed. Here we will review this construction and lay the basis for an inverse problem where we assume the divergence function $D$ to be known and we look for a Lagrangian function $\mathfrak{L}$ for which $D$ is a complete solution of the associated Hamilton-Jacobi theory. To apply these ideas to quantum systems, we have to replace probability distributions with probability amplitudes.Comment: 8 page

Magnetohydrostatic solar prominences in near-potential coronal magnetic fields

We present numerical magnetohydrostatic solutions describing the gravitationally stratified, bulk equilibrium of cool, dense prominence plasma embedded in a near-potential coronal field. These solutions are calculated using the FINESSE magnetohydrodynamics equilibrium solver and describe the morphologies of magnetic field distributions in and around prominences and the cool prominence plasma that these fields support. The equilibrium condition for this class of problem is usually different in distinct subdomains, separated by free boundaries, across which solutions are matched by suitable continuity or jump conditions describing force balance. We employ our precise finite element elliptic solver to calculate solutions not accessible by previous analytical techniques with temperature or entropy prescribed as free functions of the magnetic flux function, including a range of values of the polytropic index, temperature variations mainly across magnetic field lines and photospheric field profiles sheared close to the polarity inversion line. Out of the many examples computed here, perhaps the most noteworthy is one which reproduces precisely the three-part structure often encountered in observations: a cool dense prominence within a cavity/flux rope embedded in a hot corona. The stability properties of these new equilibria, which may be relevant to solar eruptions, can be determined in the form of a full resistive MHD spectrum using a companion hyperbolic stability solver.Comment: To appear in ApJ August 200

Statistical physics of independent component analysis

Statistical physics is used to investigate independent component analysis with polynomial contrast functions. While the replica method fails, an adapted cavity approach yields valid results. The learning curves, obtained in a suitable thermodynamic limit, display a first order phase transition from poor to perfect generalization.Comment: 7 pages, 1 figure, to appear in Europhys. Lett

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

Incremental learning (IL) has received a lot of attention recently, however, the literature lacks a precise problem definition, proper evaluation settings, and metrics tailored specifically for the IL problem. One of the main objectives of this work is to fill these gaps so as to provide a common ground for better understanding of IL. The main challenge for an IL algorithm is to update the classifier whilst preserving existing knowledge. We observe that, in addition to forgetting, a known issue while preserving knowledge, IL also suffers from a problem we call intransigence, inability of a model to update its knowledge. We introduce two metrics to quantify forgetting and intransigence that allow us to understand, analyse, and gain better insights into the behaviour of IL algorithms. We present RWalk, a generalization of EWC++ (our efficient version of EWC [Kirkpatrick2016EWC]) and Path Integral [Zenke2017Continual] with a theoretically grounded KL-divergence based perspective. We provide a thorough analysis of various IL algorithms on MNIST and CIFAR-100 datasets. In these experiments, RWalk obtains superior results in terms of accuracy, and also provides a better trade-off between forgetting and intransigence

Quantum state decorrelation

We address the general problem of removing correlations from quantum states while preserving local quantum information as much as possible. We provide a complete solution in the case of two qubits, by evaluating the minimum amount of noise that is necessary to decorrelate covariant sets of bipartite states. We show that two harmonic oscillators in arbitrary Gaussian state can be decorrelated by a Gaussian covariant map. Finally, for finite-dimensional Hilbert spaces, we prove that states obtained from most cloning channels (e.g., universal and phase-covariant cloning) can be decorrelated only at the expense of a complete erasure of information about the copied state. More generally, in finite dimension, cloning without correlations is impossible for continuous sets of states. On the contrary, for continuos variables cloning, a slight modification of the customary set-up for cloning coherent states allows one to obtain clones without correlations.Comment: 11 pages, 2 figures, RevTex

SIMS chemical analysis of extended impacts on the leading and trailing edges of LDEF experiment AO187-2

Numerous 'extended impacts' found in both leading and trailing edge capture cells were successfully analyzed for the chemical composition of projectile residues by secondary ion mass spectrometry (SIMS). Most data were obtained from the trailing edge cells where 45 of 58 impacts were classified as 'probably natural' and the remainder as 'possibly man-made debris.' This is in striking contrast to leading edge cells where 9 of 11 impacts so far measured are definitely classified as orbital debris. Although all the leading edge cells had lost their plastic entrance foils during flight, the rate of foil failure was similar to that of the trailing edge cells, 10 percent of which were recovered intact. Ultraviolet embrittlement is suspected as the major cause of failure on both leading and trailing edges. The major impediment to the accurate determination of projectile chemistry is the fractionation of volatile and refractory elements in the hypervelocity impact and redeposition processes. This effect had been noted in a simulation experiment but is more pronounced in the LDEF capture cells, probably due to the higher average velocities of the space impacts. Surface contamination of the pure Ge surfaces with a substance rich in Si, but also containing Mg and Al, provides an additional problem for the accurate determination of impactor chemistry. The effect is variable, being much larger on surfaces that were exposed to space than in those cells that remained intact. Future work will concentrate on the analyses of more leading edge impacts and the development of new SIMS techniques for the measurement of elemental abundances in extended impacts

Functional Optimisation of Online Algorithms in Multilayer Neural Networks

We study the online dynamics of learning in fully connected soft committee machines in the student-teacher scenario. The locally optimal modulation function, which determines the learning algorithm, is obtained from a variational argument in such a manner as to maximise the average generalisation error decay per example. Simulations results for the resulting algorithm are presented for a few cases. The symmetric phase plateaux are found to be vastly reduced in comparison to those found when online backpropagation algorithms are used. A discussion of the implementation of these ideas as practical algorithms is given