Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization with Optimal Noise Scheduling

The graduated optimization approach is a heuristic method for finding globally optimal solutions for nonconvex functions and has been theoretically analyzed in several studies. This paper defines a new family of nonconvex functions for graduated optimization, discusses their sufficient conditions, and provides a convergence analysis of the graduated optimization algorithm for them. It shows that stochastic gradient descent (SGD) with mini-batch stochastic gradients has the effect of smoothing the function, the degree of which is determined by the learning rate and batch size. This finding provides theoretical insights on why large batch sizes fall into sharp local minima, why decaying learning rates and increasing batch sizes are superior to fixed learning rates and batch sizes, and what the optimal learning rate scheduling is. To the best of our knowledge, this is the first paper to provide a theoretical explanation for these aspects. Moreover, a new graduated optimization framework that uses a decaying learning rate and increasing batch size is analyzed and experimental results of image classification that support our theoretical findings are reported.Comment: The latest version was updated on Nov. 2

Nonlinear simulation of resistive ballooning modes in the Large Helical Device

Nonlinear simulations of a magnetohydrodynamic (MHD) plasma in full three-dimensional geometry of the Large Helical Device (LHD) [O. Motojima et al., Phys. Plasmas 6, 1843 (1999)] are conducted. A series of simulations shows growth of resistive ballooning instability, for which the growth rate is seen to be proportional to the one-third power of the resistivity. Nonlinear saturation of the excited mode and its slow decay are observed. Distinct ridge/valley structures in the pressure are formed in the course of the nonlinear evolution. The compressibility and the viscous heating, as well as the thermal conduction, are shown to be crucial to suppress the pressure deformations. Indication of a pressure-driven relaxation phenomenon that leads to an equilibrium with broader pressure profile is observed

Metal-free isotactic-specific radical polymerization of N-isopropylacrylamide with pyridine N-oxide derivatives : the effect of methyl substituents of pyridine N-oxide on the isotactic-specificity and the proposed mechanism for the isotactic-specific radical polymerization

The radical polymerizations of N-isopropylacrylamide (NIPAAm) in chloroform at low temperatures in the presence of pyridine N-oxide (PNO) derivatives were investigated. It was found that the methylation at meta-positions of PNO improved the isotactic-specificity induced by PNO, whereas the methylation at ortho-positions prevented the induction of the isotactic-specificity. NMR analysis revealed that NIPAAm and PNO derivatives formed predominantly 2:1 complex through a hydrogen bonding interaction. Furthermore, the induction of the isotactic-specificity was attributed to the conformationally-limited propagating radicals. Based on these findings, the mechanism of the isotactic-specific radical polymerization was discussed

Hydrogen-bond-assisted isotactic-specific radical polymerization of N-isopropylacrylamide with pyridine N-oxide

Radical polymerization of N-isopropylacrylamide (NIPAAm) in CHCl3 at low temperatures in the presence of pyridine N-oxide (PNO) was investigated. An isotactic poly(NIPAAm) with meso diad content of 61% was successfully prepared at –60°C in the presence of a twofold amount of PNO. Thermodynamic analysis suggested that the isotactic-specificity was entropically induced, probably due to conformational fixation near the propagating chain-end through coordination by PNO

Topological Quantum Walk with Discrete Time-Glide Symmetry

Discrete quantum walks are periodically driven systems with discrete time evolution. In contrast to ordinary Floquet systems, no microscopic Hamiltonian exists, and the one-period time evolution is given directly by a series of unitary operators. Regarding each constituent unitary operator as a discrete time step, we formulate discrete space-time symmetry in quantum walks and evaluate the corresponding symmetry protected topological phases. In particular, we study chiral and/or time-glide symmetric topological quantum walks in this formalism. Due to discrete nature of time evolution,the topological classification is found to be different from that in conventional Floquet systems. As a concrete example, we study a two-dimensional quantum walk having both chiral and time-glide symmetries, and identify the anomalous edge states protected by these symmetries.Comment: 15 pages, 7 figure