Discrete-time thermodynamic uncertainty relation

We generalize the thermodynamic uncertainty relation, providing an entropic upper bound for average fluxes in time-continuous steady-state systems (Gingrich et al., Phys. Rev. Lett. 116, 120601 (2016)), to time-discrete Markov chains and to systems under time-symmetric, periodic driving

Integrable dissipative exclusion process: Correlation functions and physical properties

We study a one-parameter generalization of the symmetric simple exclusion process on a one dimensional lattice. In addition to the usual dynamics (where particles can hop with equal rates to the left or to the right with an exclusion constraint), annihilation and creation of pairs can occur. The system is driven out of equilibrium by two reservoirs at the boundaries. In this setting the model is still integrable: it is related to the open XXZ spin chain through a gauge transformation. This allows us to compute the full spectrum of the Markov matrix using Bethe equations. Then, we derive the spectral gap in the thermodynamical limit. We also show that the stationary state can be expressed in a matrix product form permitting to compute the multi-points correlation functions as well as the mean value of the lattice current and of the creation-annihilation current. Finally the variance of the lattice current is exactly computed for a finite size system. In the thermodynamical limit, it matches perfectly the value obtained from the associated macroscopic fluctuation theory. It provides a confirmation of the macroscopic fluctuation theory for dissipative system from a microscopic point of view.Comment: 31 pages, 7 figures ; introduction expanded, typos corrected and title change

Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks

Stochastic gradient descent (SGD) is widely believed to perform implicit regularization when used to train deep neural networks, but the precise manner in which this occurs has thus far been elusive. We prove that SGD minimizes an average potential over the posterior distribution of weights along with an entropic regularization term. This potential is however not the original loss function in general. So SGD does perform variational inference, but for a different loss than the one used to compute the gradients. Even more surprisingly, SGD does not even converge in the classical sense: we show that the most likely trajectories of SGD for deep networks do not behave like Brownian motion around critical points. Instead, they resemble closed loops with deterministic components. We prove that such "out-of-equilibrium" behavior is a consequence of highly non-isotropic gradient noise in SGD; the covariance matrix of mini-batch gradients for deep networks has a rank as small as 1% of its dimension. We provide extensive empirical validation of these claims, proven in the appendix

Macroscopic fluctuation theory

Stationary non-equilibrium states describe steady flows through macroscopic systems. Although they represent the simplest generalization of equilibrium states, they exhibit a variety of new phenomena. Within a statistical mechanics approach, these states have been the subject of several theoretical investigations, both analytic and numerical. The macroscopic fluctuation theory, based on a formula for the probability of joint space-time fluctuations of thermodynamic variables and currents, provides a unified macroscopic treatment of such states for driven diffusive systems. We give a detailed review of this theory including its main predictions and most relevant applications.Comment: Review article. Revised extended versio