Monte Carlo simulations of the three-dimensional XY spin glass focusing on the chiral and the spin order

The ordering of the three-dimensional isotropic {\it XY} spin glass with the nearest-neighbor random Gaussian coupling is studied by extensive Monte Carlo simulations. To investigate the ordering of the spin and the chirality, we compute several independent physical quantities including the glass order parameter, the Binder parameter, the correlation-length ratio, the overlap distribution and the non-self-averageness parameter, {\it etc}, for both the spin-glass (SG) and the chiral-glass (CG) degrees of freedom. Evidence of the spin-chirality decoupling, {\it i.e.}, the CG and the SG order occurring at two separated temperatures, $0<T_{SG}<T_{CG}$, is obtained from the glass order parameter, which is fully corroborated by the Binder parameter. By contrast, the CG correlation-length ratio yields a rather pathological and inconsistent result in the range of sizes we studied, which may originate from the finite-size effect associated with a significant short-length drop-off of the spatial CG correlations. Finite-size-scaling analysis yields the CG exponents $\nu_{CG}=1.36^{+0.15}_{-0.37}$ and $\eta_{CG}=0.26^{+0.29}_{-0.26}$, and the SG exponents $\nu_{SG}=1.22^{+0.26}_{-0.06}$ and $\eta_{SG}=-0.54^{+0.24}_{-0.52}$. The obtained exponents are close to those of the Heisenberg SG, but are largely different from those of the Ising SG. The chiral overlap distribution and the chiral Binder parameter exhibit the feature of a continuous one-step replica-symmetry breaking (1RSB), consistently with the previous reports. Such a 1RSB feature is again in common with that of the Heisenberg SG, but is different from the Ising one, which may be the cause of the difference in the CG critical properties from the Ising SG ones despite of a common $Z_2$ symmetry.Comment: 15 pages, 31 figure

Zeros of the partition function and dynamical singularities in spin-glass systems

We study spin-glass systems characterized by continuous occurrence of singularities. The theory of Lee-Yang zeros is used to find the singularities. By using the replica method in mean-field systems, we show that two-dimensional distributions of zeros of the partition function in a complex parameter plane are characteristic feature of random systems. The results of several models indicate that the concept of chaos in the spin-glass state is different from that of the replica symmetry breaking. We discuss that a chaotic phase at imaginary temperature is different from the spin-glass phase and is accessible by quantum dynamics in a quenching protocol.Comment: 11 pages, 6 figures, proceedings of the ICSG201

Dynamical Singularities of Glassy Systems in a Quantum Quench

We present a prototype of behavior of glassy systems driven by quantum dynamics in a quenching protocol by analyzing the random energy model in a transverse field. We calculate several types of dynamical quantum amplitude and find a freezing transition at some critical time. The behavior is understood by the partition-function zeros in the complex temperature plane. We discuss the properties of the freezing phase as a dynamical chaotic phase, which are contrasted to those of the spin-glass phase in the static system.Comment: 6 pages, 5 figure

Sparse approximation problem: how rapid simulated annealing succeeds and fails

Information processing techniques based on sparseness have been actively studied in several disciplines. Among them, a mathematical framework to approximately express a given dataset by a combination of a small number of basis vectors of an overcomplete basis is termed the {\em sparse approximation}. In this paper, we apply simulated annealing, a metaheuristic algorithm for general optimization problems, to sparse approximation in the situation where the given data have a planted sparse representation and noise is present. The result in the noiseless case shows that our simulated annealing works well in a reasonable parameter region: the planted solution is found fairly rapidly. This is true even in the case where a common relaxation of the sparse approximation problem, the $\ell_1$-relaxation, is ineffective. On the other hand, when the dimensionality of the data is close to the number of non-zero components, another metastable state emerges, and our algorithm fails to find the planted solution. This phenomenon is associated with a first-order phase transition. In the case of very strong noise, it is no longer meaningful to search for the planted solution. In this situation, our algorithm determines a solution with close-to-minimum distortion fairly quickly.Comment: 12 pages, 7 figures, a proceedings of HD^3-201

Partition-function zeros of spherical spin glasses and their relevance to chaos

We investigate partition-function zeros of the many-body interacting spherical spin glass, the so-called $p$-spin spherical model, with respect to the complex temperature in the thermodynamic limit. We use the replica method and extend the procedure of the replica symmetry breaking ansatz to be applicable in the complex-parameter case. We derive the phase diagrams in the complex-temperature plane and calculate the density of zeros in each phase. Near the imaginary axis away from the origin, there is a replica symmetric phase having a large density. On the other hand, we observe no density in the spin-glass phases, irrespective of the replica symmetry breaking. We speculate that this suggests the absence of the temperature chaos. To confirm this, we investigate the multiple many-body interacting case which is known to exhibit the chaos effect. The result shows that the density of zeros actually takes finite values in the spin-glass phase, even on the real axis. These observations indicate that the density of zeros is more closely connected to the chaos effect than the replica symmetry breaking.Comment: 22 pages, 8 figure

Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration

We investigate the signal reconstruction performance of sparse linear regression in the presence of noise when piecewise continuous nonconvex penalties are used. Among such penalties, we focus on the SCAD penalty. The contributions of this study are three-fold: We first present a theoretical analysis of a typical reconstruction performance, using the replica method, under the assumption that each component of the design matrix is given as an independent and identically distributed (i.i.d.) Gaussian variable. This clarifies the superiority of the SCAD estimator compared with $\ell_1$ in a wide parameter range, although the nonconvex nature of the penalty tends to lead to solution multiplicity in certain regions. This multiplicity is shown to be connected to replica symmetry breaking in the spin-glass theory. We also show that the global minimum of the mean square error between the estimator and the true signal is located in the replica symmetric phase. Second, we develop an approximate formula efficiently computing the cross-validation error without actually conducting the cross-validation, which is also applicable to the non-i.i.d. design matrices. It is shown that this formula is only applicable to the unique solution region and tends to be unstable in the multiple solution region. We implement instability detection procedures, which allows the approximate formula to stand alone and resultantly enables us to draw phase diagrams for any specific dataset. Third, we propose an annealing procedure, called nonconvexity annealing, to obtain the solution path efficiently. Numerical simulations are conducted on simulated datasets to examine these results to verify the theoretical results consistency and the approximate formula efficiency. Another numerical experiment on a real-world dataset is conducted; its results are consistent with those of earlier studies using the $\ell_0$ formulation.Comment: 33 pages, 18 figures. MATLAB codes implementing the proposed method are distributed in https://github.com/T-Obuchi/SLRpackage_AcceleratedCV_matla

Accelerating Cross-Validation in Multinomial Logistic Regression with ℓ1\ell_1-Regularization

We develop an approximate formula for evaluating a cross-validation estimator of predictive likelihood for multinomial logistic regression regularized by an $\ell_1$-norm. This allows us to avoid repeated optimizations required for literally conducting cross-validation; hence, the computational time can be significantly reduced. The formula is derived through a perturbative approach employing the largeness of the data size and the model dimensionality. An extension to the elastic net regularization is also addressed. The usefulness of the approximate formula is demonstrated on simulated data and the ISOLET dataset from the UCI machine learning repository.Comment: 30 pages, 9 figures. MATLAB and python codes implementing the formula derived in the manuscript are distributed in https://github.com/T-Obuchi/AcceleratedCVonMLR_matlab and https://github.com/T-Obuchi/AcceleratedCVonMLR_pytho

Weight space structure and analysis using a finite replica number in the Ising perceptron

The weight space of the Ising perceptron in which a set of random patterns is stored is examined using the generating function of the partition function $\phi(n)=(1/N)\log [Z^n]$ as the dimension of the weight vector $N$ tends to infinity, where $Z$ is the partition function and $[ ... ]$ represents the configurational average. We utilize $\phi(n)$ for two purposes, depending on the value of the ratio $\alpha=M/N$, where $M$ is the number of random patterns. For $\alpha < \alpha_{\rm s}=0.833 ...$, we employ $\phi(n)$, in conjunction with Parisi's one-step replica symmetry breaking scheme in the limit of $n \to 0$, to evaluate the complexity that characterizes the number of disjoint clusters of weights that are compatible with a given set of random patterns, which indicates that, in typical cases, the weight space is equally dominated by a single large cluster of exponentially many weights and exponentially many small clusters of a single weight. For $\alpha > \alpha_{\rm s}$, on the other hand, $\phi(n)$ is used to assess the rate function of a small probability that a given set of random patterns is atypically separable by the Ising perceptrons. We show that the analyticity of the rate function changes at $\alpha = \alpha_{\rm GD}=1.245 ...$, which implies that the dominant configuration of the atypically separable patterns exhibits a phase transition at this critical ratio. Extensive numerical experiments are conducted to support the theoretical predictions.Comment: 21 pages, 11 figures, Added references, some comments, and corrections to minor error

Relative species abundance of replicator dynamics with sparse interactions

A theory of relative species abundance on sparsely-connected networks is presented by investigating the replicator dynamics with symmetric interactions. Sparseness of a network involves difficulty in analyzing the fixed points of the equation, and we avoid this problem by treating large self interaction $u$, which allows us to construct a perturbative expansion. Based on this perturbation, we find that the nature of the interactions is directly connected to the abundance distribution, and some characteristic behaviors, such as multiple peaks in the abundance distribution and all species coexistence at moderate values of $u$, are discovered in a wide class of the distribution of the interactions. The all species coexistence collapses at a critical value of $u$, $u_c$, and this collapsing is regarded as a phase transition. To get more quantitative information, we also construct a non-perturbative theory on random graphs based on techniques of statistical mechanics. The result shows those characteristic behaviors are sustained well even for not large $u$. For even smaller values of $u$, extinct species start to appear and the abundance distribution becomes rounded and closer to a standard functional form. Another interesting finding is the non-monotonic behavior of diversity, which quantifies the number of coexisting species, when changing the ratio of mutualistic relations $\Delta$. These results are examined by numerical simulations, and the multiple peaks in the abundance distribution are confirmed to be robust against a certain level of modifications of the problem. The numerical results also show that our theory is exact for the case without extinct species, but becomes less and less precise as the proportion of extinct species grows.Comment: 27 pages, 14 figure

Replica symmetry breaking, complexity and spin representation in the generalized random energy model

We study the random energy model with a hierarchical structure known as the generalized random energy model (GREM). In contrast to the original analysis by the microcanonical ensemble formalism, we investigate the GREM by the canonical ensemble formalism in conjunction with the replica method. In this analysis, spin-glass-order parameters are defined for respective hierarchy level, and all possible patterns of replica symmetry breaking (RSB) are taken into account. As a result, we find that the higher step RSB ansatz is useful for describing spin-glass phases in this system. For investigating the nature of the higher step RSB, we generalize the notion of complexity developed for the one-step RSB to the higher step and demonstrate how the GREM is characterized by the generalized complexity. In addition, we propose a novel mean-field spin-glass model with a hierarchical structure, which is equivalent to the GREM at a certain limit. We also show that the same hierarchical structure can be implemented to other mean-field spin models than the GREM. Such models with hierarchy exhibit phase transitions of multiple steps in common.Comment: 30 pages, 11 figures; minor change