Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles

We present a canonical way to turn any smooth parametric family of probability distributions on an arbitrary search space $X$ into a continuous-time black-box optimization method on $X$, the \emph{information-geometric optimization} (IGO) method. Invariance as a design principle minimizes the number of arbitrary choices. The resulting \emph{IGO flow} conducts the natural gradient ascent of an adaptive, time-dependent, quantile-based transformation of the objective function. It makes no assumptions on the objective function to be optimized. The IGO method produces explicit IGO algorithms through time discretization. It naturally recovers versions of known algorithms and offers a systematic way to derive new ones. The cross-entropy method is recovered in a particular case, and can be extended into a smoothed, parametrization-independent maximum likelihood update (IGO-ML). For Gaussian distributions on $\mathbb{R}^d$, IGO is related to natural evolution strategies (NES) and recovers a version of the CMA-ES algorithm. For Bernoulli distributions on $\{0,1\}^d$, we recover the PBIL algorithm. From restricted Boltzmann machines, we obtain a novel algorithm for optimization on $\{0,1\}^d$. All these algorithms are unified under a single information-geometric optimization framework. Thanks to its intrinsic formulation, the IGO method achieves invariance under reparametrization of the search space $X$, under a change of parameters of the probability distributions, and under increasing transformations of the objective function. Theory strongly suggests that IGO algorithms have minimal loss in diversity during optimization, provided the initial diversity is high. First experiments using restricted Boltzmann machines confirm this insight. Thus IGO seems to provide, from information theory, an elegant way to spontaneously explore several valleys of a fitness landscape in a single run.Comment: Final published versio

Filling in CMB map missing data using constrained Gaussian realizations

For analyzing maps of the cosmic microwave background sky, it is necessary to mask out the region around the galactic equator where the parasitic foreground emission is strongest as well as the brightest compact sources. Since many of the analyses of the data, particularly those searching for non-Gaussianity of a primordial origin, are most straightforwardly carried out on full-sky maps, it is of great interest to develop efficient algorithms for filling in the missing information in a plausible way. We explore practical algorithms for filling in based on constrained Gaussian realizations. Although carrying out such realizations is in principle straightforward, for finely pixelized maps as will be required for the Planck analysis a direct brute force method is not numerically tractable. We present some concrete solutions to this problem, both on a spatially flat sky with periodic boundary conditions and on the pixelized sphere. One approach is to solve the linear system with an appropriately preconditioned conjugate gradient method. While this approach was successfully implemented on a rectangular domain with periodic boundary conditions and worked even for very wide masked regions, we found that the method failed on the pixelized sphere for reasons that we explain here. We present an approach that works for full-sky pixelized maps on the sphere involving a kernel-based multi-resolution Laplace solver followed by a series of conjugate gradient corrections near the boundary of the mask.Comment: 22 pages, 14 figures, minor changes, a few missing references adde

A scalable line-independent design algorithm for voltage and frequency control in AC islanded microgrids

We propose a decentralized control synthesis procedure for stabilizing voltage and frequency in AC Islanded microGrids (ImGs) composed of Distributed Generation Units (DGUs) and loads interconnected through power lines. The presented approach enables Plug-and-Play (PnP) operations, meaning that DGUs can be added or removed without compromising the overall ImG stability. The main feature of our approach is that the proposed design algorithm is line-independent. This implies that (i) the synthesis of each local controller requires only the parameters of the corresponding DGU and not the model of power lines connecting neighboring DGUs, and (ii) whenever a new DGU is plugged in, DGUs physically coupled with it do not have to retune their regulators because of the new power line connected to them. Moreover, we formally prove that stabilizing local controllers can be always computed, independently of the electrical parameters. Theoretical results are validated by simulating in PSCAD the behavior of a 10-DGUs ImG

Subdiffusive heat-kernel decay in four-dimensional i.i.d. random conductance models

We study the diagonal heat-kernel decay for the four-dimensional nearest-neighbor random walk (on $\Z^4$) among i.i.d. random conductances that are positive, bounded from above but can have arbitrarily heavy tails at zero. It has been known that the quenched return probability \cmss P_\omega^{2n}(0,0) after $2n$ steps is at most $C(\omega) n^{-2} \log n$, but the best lower bound till now has been $C(\omega) n^{-2}$. Here we will show that the $\log n$ term marks a real phenomenon by constructing an environment, for each sequence $\lambda_n\to\infty$, such that \cmss P_\omega^{2n}(0,0)\ge C(\omega)\log(n)n^{-2}/\lambda_n, with $C(\omega)>0$ a.s., along a deterministic subsequence of $n$'s. Notably, this holds simultaneously with a (non-degenerate) quenched invariance principle. As for the $d\ge5$ cases studied earlier, the source of the anomalous decay is a trapping phenomenon although the contribution is in this case collected from a whole range of spatial scales.Comment: 28 pages, version to appear in J. Lond. Math. So