Multiple time-scale approach for a system of Brownian particles in a non-uniform temperature field

The Smoluchowsky equation for a system of interacting Brownian particles in a temperature gradient is derived from the Kramers equation by means of a multiple time-scale method. The interparticle interactions are assumed to be represented by a mean-field description. We present numerical results that compare well with the theoretical prediction together with an extensive discussion on the prescription of the Langevin equation in overdamped systems.Comment: 8 pages, 2 figure

Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks

Stochastic gradient descent (SGD) is widely believed to perform implicit regularization when used to train deep neural networks, but the precise manner in which this occurs has thus far been elusive. We prove that SGD minimizes an average potential over the posterior distribution of weights along with an entropic regularization term. This potential is however not the original loss function in general. So SGD does perform variational inference, but for a different loss than the one used to compute the gradients. Even more surprisingly, SGD does not even converge in the classical sense: we show that the most likely trajectories of SGD for deep networks do not behave like Brownian motion around critical points. Instead, they resemble closed loops with deterministic components. We prove that such "out-of-equilibrium" behavior is a consequence of highly non-isotropic gradient noise in SGD; the covariance matrix of mini-batch gradients for deep networks has a rank as small as 1% of its dimension. We provide extensive empirical validation of these claims, proven in the appendix

Nonlinear mean field Fokker-Planck equations. Application to the chemotaxis of biological populations

We study a general class of nonlinear mean field Fokker-Planck equations in relation with an effective generalized thermodynamical formalism. We show that these equations describe several physical systems such as: chemotaxis of bacterial populations, Bose-Einstein condensation in the canonical ensemble, porous media, generalized Cahn-Hilliard equations, Kuramoto model, BMF model, Burgers equation, Smoluchowski-Poisson system for self-gravitating Brownian particles, Debye-Huckel theory of electrolytes, two-dimensional turbulence... In particular, we show that nonlinear mean field Fokker-Planck equations can provide generalized Keller-Segel models describing the chemotaxis of biological populations. As an example, we introduce a new model of chemotaxis incorporating both effects of anomalous diffusion and exclusion principle (volume filling). Therefore, the notion of generalized thermodynamics can have applications for concrete physical systems. We also consider nonlinear mean field Fokker-Planck equations in phase space and show the passage from the generalized Kramers equation to the generalized Smoluchowski equation in a strong friction limit. Our formalism is simple and illustrated by several explicit examples corresponding to Boltzmann, Tsallis and Fermi-Dirac entropies among others

Finite sampling interval effects in Kramers-Moyal analysis

Large sampling intervals can affect reconstruction of Kramers-Moyal coefficients from data. A new method, which is direct, non-stochastic and exact up to numerical accuracy, can estimate these finite-time effects. For the first time, exact finite-time effects are described analytically for special cases; biologically inspired numerical examples are also worked through numerically. The approach developed here will permit better evaluation of Langevin or Fokker-Planck based models from data with large sampling intervals. It can also be used to predict the sampling intervals for which finite-time effects become significant.Comment: Preprin