Statistical Mechanics of Dictionary Learning

Finding a basis matrix (dictionary) by which objective signals are represented sparsely is of major relevance in various scientific and technological fields. We consider a problem to learn a dictionary from a set of training signals. We employ techniques of statistical mechanics of disordered systems to evaluate the size of the training set necessary to typically succeed in the dictionary learning. The results indicate that the necessary size is much smaller than previously estimated, which theoretically supports and/or encourages the use of dictionary learning in practical situations.Comment: 6 pages, 4 figure

A remarkable recurrent nova in M 31: The 2010 eruption recovered and evidence of a six-month period

The Andromeda Galaxy recurrent nova M31N 2008-12a has been caught in eruption nine times. Six observed eruptions in the seven years from 2008 to 2014 suggested a duty cycle of ~1 year, which makes this the most rapidly recurring system known and the leading single-degenerate Type Ia Supernova progenitor candidate; but no 2010 eruption has been found so far. Here we present evidence supporting the recovery of the 2010 eruption, based on archival images taken at and around the time. We detect the 2010 eruption in a pair of images at 2010 Nov 20.52 UT, with a magnitude of m_R = 17.84 +/- 0.19. The sequence of seven eruptions shows significant indications of a duty cycle slightly shorter than one year, which makes successive eruptions occur progressively earlier in the year. We compared three archival X-ray detections with the well observed multi-wavelength light curve of the 2014 eruption to accurately constrain the time of their optical peaks. The results imply that M31N 2008-12a might have in fact a recurrence period of ~6 months (175 +/- 11 days), making it even more exceptional. If this is the case, then we predict that soon two eruptions per year will be observable. Furthermore, we predict the next eruption will occur around late Sep 2015. We encourage additional observations.Comment: 4 pages, 3 figures, 2 tables; submitted to A&A Letter

Analysis of CDMA systems that are characterized by eigenvalue spectrum

An approach by which to analyze the performance of the code division multiple access (CDMA) scheme, which is a core technology used in modern wireless communication systems, is provided. The approach characterizes the objective system by the eigenvalue spectrum of a cross-correlation matrix composed of signature sequences used in CDMA communication, which enables us to handle a wider class of CDMA systems beyond the basic model reported by Tanaka. The utility of the novel scheme is shown by analyzing a system in which the generation of signature sequences is designed for enhancing the orthogonality.Comment: 7 pages, 2 figure

On-line learning of non-monotonic rules by simple perceptron

We study the generalization ability of a simple perceptron which learns unlearnable rules. The rules are presented by a teacher perceptron with a non-monotonic transfer function. The student is trained in the on-line mode. The asymptotic behaviour of the generalization error is estimated under various conditions. Several learning strategies are proposed and improved to obtain the theoretical lower bound of the generalization error.Comment: LaTeX 20 pages using IOP LaTeX preprint style file, 14 figure

Parallel dynamics of continuous Hopfield model revisited

We have applied the generating functional analysis (GFA) to the continuous Hopfield model. We have also confirmed that the GFA predictions in some typical cases exhibit good consistency with computer simulation results. When a retarded self-interaction term is omitted, the GFA result becomes identical to that obtained using the statistical neurodynamics as well as the case of the sequential binary Hopfield model.Comment: 4 pages, 2 figure

Statistical Physics of Irregular Low-Density Parity-Check Codes

Low-density parity-check codes with irregular constructions have been recently shown to outperform the most advanced error-correcting codes to date. In this paper we apply methods of statistical physics to study the typical properties of simple irregular codes. We use the replica method to find a phase transition which coincides with Shannon's coding bound when appropriate parameters are chosen. The decoding by belief propagation is also studied using statistical physics arguments; the theoretical solutions obtained are in good agreement with simulations. We compare the performance of irregular with that of regular codes and discuss the factors that contribute to the improvement in performance.Comment: 20 pages, 9 figures, revised version submitted to JP

One step RSB scheme for the rate distortion function

We apply statistical mechanics to an inverse problem of linear mapping to investigate the physics of the irreversible compression. We use the replica symmetry breaking (RSB) technique with a toy model to demonstrate the Shannon's result. The rate distortion function, which is widely known as the theoretical limit of the compression with a fidelity criterion, is derived using the Parisi one step RSB scheme. The bound can not be achieved in the sparsely-connected systems, where suboptimal solutions dominate the capacity.Comment: 8 pages, 2 figure

A remarkable recurrent nova in M 31: The 2010 eruption recovered and evidence of a six-month period

The Andromeda Galaxy recurrent nova M31N 2008-12a has been caught in eruption nine times. Six observed eruptions in the seven years from 2008 to 2014 suggested a duty cycle of ~1 year, which makes this the most rapidly recurring system known and the leading single-degenerate Type Ia Supernova progenitor candidate; but no 2010 eruption has been found so far. Here we present evidence supporting the recovery of the 2010 eruption, based on archival images taken at and around the time. We detect the 2010 eruption in a pair of images at 2010 Nov 20.52 UT, with a magnitude of m_R = 17.84 +/- 0.19. The sequence of seven eruptions shows significant indications of a duty cycle slightly shorter than one year, which makes successive eruptions occur progressively earlier in the year. We compared three archival X-ray detections with the well observed multi-wavelength light curve of the 2014 eruption to accurately constrain the time of their optical peaks. The results imply that M31N 2008-12a might have in fact a recurrence period of ~6 months (175 +/- 11 days), making it even more exceptional. If this is the case, then we predict that soon two eruptions per year will be observable. Furthermore, we predict the next eruption will occur around late Sep 2015. We encourage additional observations

An efficient CDMA decoder for correlated information sources

We consider the detection of correlated information sources in the ubiquitous Code-Division Multiple-Access (CDMA) scheme. We propose a message-passing based scheme for detecting correlated sources directly, with no need for source coding. The detection is done simultaneously over a block of transmitted binary symbols (word). Simulation results are provided demonstrating a substantial improvement in bit-error-rate in comparison with the unmodified detector and the alternative of source compression. The robustness of the error-performance improvement is shown under practical model settings, including wrong estimation of the generating Markov transition matrix and finite-length spreading codes.Comment: 11 page

Bi-stability of mixed states in neural network storing hierarchical patterns

We discuss the properties of equilibrium states in an autoassociative memory model storing hierarchically correlated patterns (hereafter, hierarchical patterns). We will show that symmetric mixed states (hereafter, mixed states) are bi-stable on the associative memory model storing the hierarchical patterns in a region of the ferromagnetic phase. This means that the first-order transition occurs in this ferromagnetic phase. We treat these contents with a statistical mechanical method (SCSNA) and by computer simulation. Finally, we discuss a physiological implication of this model. Sugase et al. analyzed the time-course of the information carried by the firing of face-responsive neurons in the inferior temporal cortex. We also discuss the relation between the theoretical results and the physiological experiments of Sugase et al.Comment: 18 pages, 6 figure