Laplace's rule of succession in information geometry

Laplace's "add-one" rule of succession modifies the observed frequencies in a sequence of heads and tails by adding one to the observed counts. This improves prediction by avoiding zero probabilities and corresponds to a uniform Bayesian prior on the parameter. The canonical Jeffreys prior corresponds to the "add-one-half" rule. We prove that, for exponential families of distributions, such Bayesian predictors can be approximated by taking the average of the maximum likelihood predictor and the \emph{sequential normalized maximum likelihood} predictor from information theory. Thus in this case it is possible to approximate Bayesian predictors without the cost of integrating or sampling in parameter space

Parametric Fokker-Planck equation

We derive the Fokker-Planck equation on the parametric space. It is the Wasserstein gradient flow of relative entropy on the statistical manifold. We pull back the PDE to a finite dimensional ODE on parameter space. Some analytical example and numerical examples are presented

Field Theoretical Analysis of On-line Learning of Probability Distributions

On-line learning of probability distributions is analyzed from the field theoretical point of view. We can obtain an optimal on-line learning algorithm, since renormalization group enables us to control the number of degrees of freedom of a system according to the number of examples. We do not learn parameters of a model, but probability distributions themselves. Therefore, the algorithm requires no a priori knowledge of a model.Comment: 4 pages, 1 figure, RevTe

Controlling Model Complexity in Probabilistic Model-Based Dynamic Optimization of Neural Network Structures

A method of simultaneously optimizing both the structure of neural networks and the connection weights in a single training loop can reduce the enormous computational cost of neural architecture search. We focus on the probabilistic model-based dynamic neural network structure optimization that considers the probability distribution of structure parameters and simultaneously optimizes both the distribution parameters and connection weights based on gradient methods. Since the existing algorithm searches for the structures that only minimize the training loss, this method might find overly complicated structures. In this paper, we propose the introduction of a penalty term to control the model complexity of obtained structures. We formulate a penalty term using the number of weights or units and derive its analytical natural gradient. The proposed method minimizes the objective function injected the penalty term based on the stochastic gradient descent. We apply the proposed method in the unit selection of a fully-connected neural network and the connection selection of a convolutional neural network. The experimental results show that the proposed method can control model complexity while maintaining performance.Comment: Accepted as a conference paper at the 28th International Conference on Artificial Neural Networks (ICANN 2019). The final authenticated publication will be available in the Springer Lecture Notes in Computer Science (LNCS). 13 page

Congruent families and invariant tensors

Classical results of Chentsov and Campbell state that -- up to constant multiples -- the only $2$-tensor field of a statistical model which is invariant under congruent Markov morphisms is the Fisher metric and the only invariant $3$-tensor field is the Amari-Chentsov tensor. We generalize this result for arbitrary degree $n$, showing that any family of $n$-tensors which is invariant under congruent Markov morphisms is algebraically generated by the canonical tensor fields defined in an earlier paper

Storage and recall of weak coherent optical pulses with an efficiency of 25%

We demonstrate experimentally a quantum memory scheme for the storage of weak coherent light pulses in an inhomogeneously broadened optical transition in a Pr^{3+}: YSO crystal at 2.1 K. Precise optical pumping using a frequency stable (about 1kHz linewidth) laser is employed to create a highly controllable Atomic Frequency Comb (AFC) structure. We report single photon storage and retrieval efficiencies of 25%, based on coherent photon echo type re-emission in the forward direction. The coherence property of the quantum memory is proved through interference between a super Gaussian pulse and the emitted echo. Backward retrieval of the photon echo emission has potential for increasing storage and recall efficiency.Comment: 5,

Observational Evidence for Coronal Twisted Flux Rope

Multi-instrument data sets of NOAA AR10938 on Jan. 16, 2007, (e.g., {\emph{Hinode}}, {\it{STEREO}}, {\it{GOES}}, {\it{MLSO}} and {\it{ISOON}} H$\alpha$) are utilized to study the fine structure and evolution of a magnetic loop system exhibiting multiple crossing threads, whose arrangement and individual shapes are very suggestive of individual field lines in a flux rope. The footpoints of the magnetic threads are closely rooted into pores and plage areas. A C-class flare recorded by {\it{GOES}} at approximately 2:35 UT near one of the footpoints of the multi-thread system (along with a wisp of loop material shown by EUV data) led to the brightening of the magnetic structure revealing its fine structure with several threads that indicate a high degree of linking (suggesting a left-handed helical pattern as shown by the filament structure formed later-on). EUV observations by {\emph{Hinode}}/EIS of hot spectral lines at 2:46 UT show a complex structure of coronal loops. The same features were observed about 20 minutes later in X-ray images from {\emph{Hinode}}/XRT and about 30 minutes further in EUV images of {\it{STEREO}}/SECCHI/EUVI with much better resolution. H$\alpha$ and 304 {\AA} images revealed the presence of several filament fibrils in the same area. They evolved a few hours later into a denser structure seemingly showing helical structure, which persistently lasted for several days forming a segment of a larger scale filament. The present observations provide an important indication for a flux robe as a precursor of a solar filament.Comment: 13 pages, 4 figure

Transient dynamics for sequence processing neural networks

An exact solution of the transient dynamics for a sequential associative memory model is discussed through both the path-integral method and the statistical neurodynamics. Although the path-integral method has the ability to give an exact solution of the transient dynamics, only stationary properties have been discussed for the sequential associative memory. We have succeeded in deriving an exact macroscopic description of the transient dynamics by analyzing the correlation of crosstalk noise. Surprisingly, the order parameter equations of this exact solution are completely equivalent to those of the statistical neurodynamics, which is an approximation theory that assumes crosstalk noise to obey the Gaussian distribution. In order to examine our theoretical findings, we numerically obtain cumulants of the crosstalk noise. We verify that the third- and fourth-order cumulants are equal to zero, and that the crosstalk noise is normally distributed even in the non-retrieval case. We show that the results obtained by our theory agree with those obtained by computer simulations. We have also found that the macroscopic unstable state completely coincides with the separatrix.Comment: 21 pages, 4 figure