Regularized Optimal Transport and the Rot Mover's Distance

This paper presents a unified framework for smooth convex regularization of discrete optimal transport problems. In this context, the regularized optimal transport turns out to be equivalent to a matrix nearness problem with respect to Bregman divergences. Our framework thus naturally generalizes a previously proposed regularization based on the Boltzmann-Shannon entropy related to the Kullback-Leibler divergence, and solved with the Sinkhorn-Knopp algorithm. We call the regularized optimal transport distance the rot mover's distance in reference to the classical earth mover's distance. We develop two generic schemes that we respectively call the alternate scaling algorithm and the non-negative alternate scaling algorithm, to compute efficiently the regularized optimal plans depending on whether the domain of the regularizer lies within the non-negative orthant or not. These schemes are based on Dykstra's algorithm with alternate Bregman projections, and further exploit the Newton-Raphson method when applied to separable divergences. We enhance the separable case with a sparse extension to deal with high data dimensions. We also instantiate our proposed framework and discuss the inherent specificities for well-known regularizers and statistical divergences in the machine learning and information geometry communities. Finally, we demonstrate the merits of our methods with experiments using synthetic data to illustrate the effect of different regularizers and penalties on the solutions, as well as real-world data for a pattern recognition application to audio scene classification

A Smoothed Dual Approach for Variational Wasserstein Problems

Variational problems that involve Wasserstein distances have been recently proposed to summarize and learn from probability measures. Despite being conceptually simple, such problems are computationally challenging because they involve minimizing over quantities (Wasserstein distances) that are themselves hard to compute. We show that the dual formulation of Wasserstein variational problems introduced recently by Carlier et al. (2014) can be regularized using an entropic smoothing, which leads to smooth, differentiable, convex optimization problems that are simpler to implement and numerically more stable. We illustrate the versatility of this approach by applying it to the computation of Wasserstein barycenters and gradient flows of spacial regularization functionals

Hamiltonian dynamics, nanosystems, and nonequilibrium statistical mechanics

An overview is given of recent advances in nonequilibrium statistical mechanics on the basis of the theory of Hamiltonian dynamical systems and in the perspective provided by the nanosciences. It is shown how the properties of relaxation toward a state of equilibrium can be derived from Liouville's equation for Hamiltonian dynamical systems. The relaxation rates can be conceived in terms of the so-called Pollicott-Ruelle resonances. In spatially extended systems, the transport coefficients can also be obtained from the Pollicott-Ruelle resonances. The Liouvillian eigenstates associated with these resonances are in general singular and present fractal properties. The singular character of the nonequilibrium states is shown to be at the origin of the positive entropy production of nonequilibrium thermodynamics. Furthermore, large-deviation dynamical relationships are obtained which relate the transport properties to the characteristic quantities of the microscopic dynamics such as the Lyapunov exponents, the Kolmogorov-Sinai entropy per unit time, and the fractal dimensions. We show that these large-deviation dynamical relationships belong to the same family of formulas as the fluctuation theorem, as well as a new formula relating the entropy production to the difference between an entropy per unit time of Kolmogorov-Sinai type and a time-reversed entropy per unit time. The connections to the nonequilibrium work theorem and the transient fluctuation theorem are also discussed. Applications to nanosystems are described.Comment: Lecture notes for the International Summer School Fundamental Problems in Statistical Physics XI (Leuven, Belgium, September 4-17, 2005

Predictability: a way to characterize Complexity

Different aspects of the predictability problem in dynamical systems are reviewed. The deep relation among Lyapunov exponents, Kolmogorov-Sinai entropy, Shannon entropy and algorithmic complexity is discussed. In particular, we emphasize how a characterization of the unpredictability of a system gives a measure of its complexity. Adopting this point of view, we review some developments in the characterization of the predictability of systems showing different kind of complexity: from low-dimensional systems to high-dimensional ones with spatio-temporal chaos and to fully developed turbulence. A special attention is devoted to finite-time and finite-resolution effects on predictability, which can be accounted with suitable generalization of the standard indicators. The problems involved in systems with intrinsic randomness is discussed, with emphasis on the important problems of distinguishing chaos from noise and of modeling the system. The characterization of irregular behavior in systems with discrete phase space is also considered.Comment: 142 Latex pgs. 41 included eps figures, submitted to Physics Reports. Related information at this http://axtnt2.phys.uniroma1.i

Large Scale Stochastic Dynamics

In focus are interacting stochastic systems with many components, ranging from stochastic partial differential equations to discrete systems as interacting particles on a lattice moving through random jumps. More specifically one wants to understand the large scale behavior, large in spatial extent but also over long time spans, as entailed by the characterization of stationary measures, effective macroscopic evolution laws, transport of conserved fields, homogenization, self-similar structure and scaling, critical dynamics, aging, dynamical phase transitions, large deviations, to mention only a few key items

The automatic solution of partial differential equations using a global spectral method

A spectral method for solving linear partial differential equations (PDEs) with variable coefficients and general boundary conditions defined on rectangular domains is described, based on separable representations of partial differential operators and the one-dimensional ultraspherical spectral method. If a partial differential operator is of splitting rank $2$, such as the operator associated with Poisson or Helmholtz, the corresponding PDE is solved via a generalized Sylvester matrix equation, and a bivariate polynomial approximation of the solution of degree $(n_x,n_y)$ is computed in $\mathcal{O}((n_x n_y)^{3/2})$ operations. Partial differential operators of splitting rank $\geq 3$ are solved via a linear system involving a block-banded matrix in $\mathcal{O}(\min(n_x^{3} n_y,n_x n_y^{3}))$ operations. Numerical examples demonstrate the applicability of our 2D spectral method to a broad class of PDEs, which includes elliptic and dispersive time-evolution equations. The resulting PDE solver is written in MATLAB and is publicly available as part of CHEBFUN. It can resolve solutions requiring over a million degrees of freedom in under $60$ seconds. An experimental implementation in the Julia language can currently perform the same solve in $10$ seconds.Comment: 22 page

An Optimization of Thermodynamic Efficiency vs. Capacity for Communications Systems

This work provides a fundamental view of the mechanisms which affect the power efficiency of communications processes along with a method for efficiency enhancement. Shannon\u27s work is the definitive source for analyzing information capacity of a communications system but his formulation does not predict an efficiency relationship suitable for calculating the power consumption of a system, particularly for practical signals which may only approach the capacity limit. This work leverages Shannon\u27s while providing additional insight through physical models which enable the calculation and improvement of efficiency for the encoding of signals. The proliferation of Mobile Communications platforms is challenging capacity of networks largely because of the ever increasing data rate at each node. This places significant power management demands on personal computing devices as well as cellular and WLAN terminals. The increased data throughput translates to shorter meantime between battery charging cycles and increased thermal footprint. Solutions are developed herein to counter this trend. Hardware was constructed to measure the efficiency of a prototypical Gaussian signal prior to efficiency enhancement. After an optimization was performed, the efficiency of the encoding apparatus increased from 3.125% to greater than 86% for a manageable investment of resources. Likewise several telecommunications standards based waveforms were also tested on the same hardware. The results reveal that the developed physical theories extrapolate in a very accurate manner to an electronics application, predicting the efficiency of single ended and differential encoding circuits before and after optimization

Dynamical Systems

Complex systems are pervasive in many areas of science integrated in our daily lives. Examples include financial markets, highway transportation networks, telecommunication networks, world and country economies, social networks, immunological systems, living organisms, computational systems and electrical and mechanical structures. Complex systems are often composed of a large number of interconnected and interacting entities, exhibiting much richer global scale dynamics than the properties and behavior of individual entities. Complex systems are studied in many areas of natural sciences, social sciences, engineering and mathematical sciences. This special issue therefore intends to contribute towards the dissemination of the multifaceted concepts in accepted use by the scientific community. We hope readers enjoy this pertinent selection of papers which represents relevant examples of the state of the art in present day research. [...