Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles

We present a canonical way to turn any smooth parametric family of probability distributions on an arbitrary search space $X$ into a continuous-time black-box optimization method on $X$, the \emph{information-geometric optimization} (IGO) method. Invariance as a design principle minimizes the number of arbitrary choices. The resulting \emph{IGO flow} conducts the natural gradient ascent of an adaptive, time-dependent, quantile-based transformation of the objective function. It makes no assumptions on the objective function to be optimized. The IGO method produces explicit IGO algorithms through time discretization. It naturally recovers versions of known algorithms and offers a systematic way to derive new ones. The cross-entropy method is recovered in a particular case, and can be extended into a smoothed, parametrization-independent maximum likelihood update (IGO-ML). For Gaussian distributions on $\mathbb{R}^d$, IGO is related to natural evolution strategies (NES) and recovers a version of the CMA-ES algorithm. For Bernoulli distributions on $\{0,1\}^d$, we recover the PBIL algorithm. From restricted Boltzmann machines, we obtain a novel algorithm for optimization on $\{0,1\}^d$. All these algorithms are unified under a single information-geometric optimization framework. Thanks to its intrinsic formulation, the IGO method achieves invariance under reparametrization of the search space $X$, under a change of parameters of the probability distributions, and under increasing transformations of the objective function. Theory strongly suggests that IGO algorithms have minimal loss in diversity during optimization, provided the initial diversity is high. First experiments using restricted Boltzmann machines confirm this insight. Thus IGO seems to provide, from information theory, an elegant way to spontaneously explore several valleys of a fitness landscape in a single run.Comment: Final published versio

Non-Cognitive Skills: How Much Do They Matter for Earnings in Canada?

Evidence from different countries suggests that non-cognitive skills play an important role in wage determination and overall social outcomes, but studies for Canada are scarce. We contribute to filling this gap by estimating wage regressions with the Big Five traits using the Longitudinal and International Study of Adults. Our results indicate that conscientiousness is positively associated with wages, while agreeableness, extraversion, and neuroticism are associated with negative returns, with higher magnitudes on agreeableness and conscientiousness for females. Cognitive ability has the highest estimated wage return so, while significant, non-cognitive skills do not seem to be the most important wage determinant

DYAMOND: the DYnamics of the Atmospheric general circulation Modeled On Non-hydrostatic Domains

A review of the experimental protocol and motivation for DYAMOND, the first intercomparison project of global storm-resolving models, is presented. Nine models submitted simulation output for a 40-day (1 August–10 September 2016) intercomparison period. Eight of these employed a tiling of the sphere that was uniformly less than 5 km. By resolving the transient dynamics of convective storms in the tropics, global storm-resolving models remove the need to parameterize tropical deep convection, providing a fundamentally more sound representation of the climate system and a more natural link to commensurately high-resolution data from satellite-borne sensors. The models and some basic characteristics of their output are described in more detail, as is the availability and planned use of this output for future scientific study. Tropically and zonally averaged energy budgets, precipitable water distributions, and precipitation from the model ensemble are evaluated, as is their representation of tropical cyclones and the predictability of column water vapor, the latter being important for tropical weather

Tropical cyclones in global storm-resolving models

Recent progress in computing and model development has initiated the era of global storm-resolving modeling and with it the potential to transform weather and climate prediction. Within the general theme of vetting this new class of models, the present study evaluates nine global-storm resolving models in their ability to simulate tropical cyclones (TCs). Results show that, broadly speaking, the models produce realistic TCs and remove longstanding issues known from global models such as the deficiency to accurately simulate TC intensity. However, TCs are strongly affected by model formulation, and all models suffer from unique biases regarding the number of TCs, intensity, size, and structure. Some models simulated TCs better than others, but no single model was superior in every way. The overall results indicate that global storm-resolving models are able to open a new chapter in TC prediction, but they need to be improved to unleash their full potential

French Roadmap for complex Systems 2008-2009

This second issue of the French Complex Systems Roadmap is the outcome of the Entretiens de Cargese 2008, an interdisciplinary brainstorming session organized over one week in 2008, jointly by RNSC, ISC-PIF and IXXI. It capitalizes on the first roadmap and gathers contributions of more than 70 scientists from major French institutions. The aim of this roadmap is to foster the coordination of the complex systems community on focused topics and questions, as well as to present contributions and challenges in the complex systems sciences and complexity science to the public, political and industrial spheres

La modélisation numérique de l'atmosphère à l'échelle hectométrique: Comité Scientifique Consultatif de Météo-France

RAPPORT POUR LE COMITÉ SCIENTIFIQUE CONSULTATIF DE MÉTÉO-FRANC

La modélisation numérique de l'atmosphère à l'échelle hectométrique: Comité Scientifique Consultatif de Météo-France

RAPPORT POUR LE COMITÉ SCIENTIFIQUE CONSULTATIF DE MÉTÉO-FRANC

Stability of Constant and Variable Coefficient Semi-Implicit Schemes for the Fully Elastic System of Euler Equations in the Case of Steep Slopes

International audienceCrank–Nicolson temporal schemes are usually employed in most semi-implicit models commonly applied in numerical weather prediction. These schemes are approached by fixed points methods like the iterative centered implicit scheme, where a linear operator is introduced to invert the implicit problem. When this operator is spatially dependent the scheme is a variable coefficient one, and a constant coefficient one otherwise. Constant and variable coefficient temporal schemes are analyzed and tested on idealized test cases to improve numerical stability on the steepest slopes of the orography encountered at hectometric scales. These analyses and experiments are conducted for the fully elastic system of Euler equations for different slopes and different thermal residuals. Because of their low computational cost, several strategies can be easily tested. None of the strategies considered for constant coefficient schemes significantly improves numerical stability without worsening efficiency or accuracy. Hence, constant coefficient schemes are probably not the most suitable schemes for high-resolution computing. A successful strategy consists in using the same features of constant coefficient schemes except for the orographic terms that are implicitly treated, resulting in a variable coefficient scheme. In this case, slopes up to 70° can be easily reached, even in the case of a moderate thermal residual. Since estimates on the condition number of the implicit problem containing orographic terms remains low even in case of steep slopes, the implicit problem should be easily inverted by an iterative solver

Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles

We present a canonical way to turn any smooth parametric family of probability distributions on an arbitrary search space $X$ into a continuous-time black-box optimization method on $X$, the \emph{information-geometric optimization} (IGO) method. Invariance as a major design principle keeps the number of arbitrary choices to a minimum. The resulting \emph{IGO flow} is the flow of an ordinary differential equation conducting the natural gradient ascent of an adaptive, time-dependent transformation of the objective function. It makes no particular assumptions on the objective function to be optimized. The IGO method produces explicit IGO algorithms through time discretization. It naturally recovers versions of known algorithms and offers a systematic way to derive new ones. In continuous search spaces, IGO algorithms take a form related to natural evolution strategies (NES). The cross-entropy method is recovered in a particular case with a large time step, and can be extended into a smoothed, parametrization-independent maximum likelihood update (IGO-ML). When applied to the family of Gaussian distributions on $\R^d$, the IGO framework recovers a version of the well-known CMA-ES algorithm and of xNES. For the family of Bernoulli distributions on $\{0,1\}^d$, we recover the seminal PBIL algorithm. For the distributions of restricted Boltzmann machines, we naturally obtain a novel algorithm for discrete optimization on $\{0,1\}^d$. All these algorithms are natural instances of, and unified under, the single information-geometric optimization framework. The IGO method achieves, thanks to its intrinsic formulation, maximal invariance properties: invariance under reparametrization of the search space $X$, under a change of parameters of the probability distribution, and under increasing transformation of the function to be optimized. The latter is achieved through an adaptive formulation of the objective. Theoretical considerations strongly suggest that IGO algorithms are essentially characterized by a minimal change of the distribution over time. Therefore they have minimal loss in diversity through the course of optimization, provided the initial diversity is high. First experiments using restricted Boltzmann machines confirm this insight. As a simple consequence, IGO seems to provide, from information theory, an elegant way to spontaneously explore several valleys of a fitness landscape in a single run