Roman roads: The hierarchical endosymbiosis of cognitive modules

Serial endosymbiosis theory provides a unifying paradigm for examining the interaction of cognitive modules at vastly different scales of biological, social, and cultural organization. A trivial but not unimportant model associates a dual information source with a broad class of cognitive processes, and punctuated phenomena akin to phase transitions in physical systems, and associated coevolutionary processes, emerge as consequences of the homology between information source uncertainty and free energy density. The dynamics, including patterns of punctuation similar to ecosystem resilience transitions, are large dominated by the availability of 'Roman roads' constituting channels for the transmission of information between modules

Prediction and Generation of Binary Markov Processes: Can a Finite-State Fox Catch a Markov Mouse?

Understanding the generative mechanism of a natural system is a vital component of the scientific method. Here, we investigate one of the fundamental steps toward this goal by presenting the minimal generator of an arbitrary binary Markov process. This is a class of processes whose predictive model is well known. Surprisingly, the generative model requires three distinct topologies for different regions of parameter space. We show that a previously proposed generator for a particular set of binary Markov processes is, in fact, not minimal. Our results shed the first quantitative light on the relative (minimal) costs of prediction and generation. We find, for instance, that the difference between prediction and generation is maximized when the process is approximately independently, identically distributed.Comment: 12 pages, 12 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/gmc.ht

Unfolding the procedure of characterizing recorded ultra low frequency, kHZ and MHz electromagetic anomalies prior to the L'Aquila earthquake as pre-seismic ones. Part I

Ultra low frequency, kHz and MHz electromagnetic anomalies were recorded prior to the L'Aquila catastrophic earthquake that occurred on April 6, 2009. The main aims of this contribution are: (i) To suggest a procedure for the designation of detected EM anomalies as seismogenic ones. We do not expect to be possible to provide a succinct and solid definition of a pre-seismic EM emission. Instead, we attempt, through a multidisciplinary analysis, to provide elements of a definition. (ii) To link the detected MHz and kHz EM anomalies with equivalent last stages of the L'Aquila earthquake preparation process. (iii) To put forward physically meaningful arguments to support a way of quantifying the time to global failure and the identification of distinguishing features beyond which the evolution towards global failure becomes irreversible. The whole effort is unfolded in two consecutive parts. We clarify we try to specify not only whether or not a single EM anomaly is pre-seismic in itself, but mainly whether a combination of kHz, MHz, and ULF EM anomalies can be characterized as pre-seismic one

Regularized Optimal Transport and the Rot Mover's Distance

This paper presents a unified framework for smooth convex regularization of discrete optimal transport problems. In this context, the regularized optimal transport turns out to be equivalent to a matrix nearness problem with respect to Bregman divergences. Our framework thus naturally generalizes a previously proposed regularization based on the Boltzmann-Shannon entropy related to the Kullback-Leibler divergence, and solved with the Sinkhorn-Knopp algorithm. We call the regularized optimal transport distance the rot mover's distance in reference to the classical earth mover's distance. We develop two generic schemes that we respectively call the alternate scaling algorithm and the non-negative alternate scaling algorithm, to compute efficiently the regularized optimal plans depending on whether the domain of the regularizer lies within the non-negative orthant or not. These schemes are based on Dykstra's algorithm with alternate Bregman projections, and further exploit the Newton-Raphson method when applied to separable divergences. We enhance the separable case with a sparse extension to deal with high data dimensions. We also instantiate our proposed framework and discuss the inherent specificities for well-known regularizers and statistical divergences in the machine learning and information geometry communities. Finally, we demonstrate the merits of our methods with experiments using synthetic data to illustrate the effect of different regularizers and penalties on the solutions, as well as real-world data for a pattern recognition application to audio scene classification

On Tilted Losses in Machine Learning: Theory and Applications

Exponential tilting is a technique commonly used in fields such as statistics, probability, information theory, and optimization to create parametric distribution shifts. Despite its prevalence in related fields, tilting has not seen widespread use in machine learning. In this work, we aim to bridge this gap by exploring the use of tilting in risk minimization. We study a simple extension to ERM -- tilted empirical risk minimization (TERM) -- which uses exponential tilting to flexibly tune the impact of individual losses. The resulting framework has several useful properties: We show that TERM can increase or decrease the influence of outliers, respectively, to enable fairness or robustness; has variance-reduction properties that can benefit generalization; and can be viewed as a smooth approximation to the tail probability of losses. Our work makes rigorous connections between TERM and related objectives, such as Value-at-Risk, Conditional Value-at-Risk, and distributionally robust optimization (DRO). We develop batch and stochastic first-order optimization methods for solving TERM, provide convergence guarantees for the solvers, and show that the framework can be efficiently solved relative to common alternatives. Finally, we demonstrate that TERM can be used for a multitude of applications in machine learning, such as enforcing fairness between subgroups, mitigating the effect of outliers, and handling class imbalance. Despite the straightforward modification TERM makes to traditional ERM objectives, we find that the framework can consistently outperform ERM and deliver competitive performance with state-of-the-art, problem-specific approaches.Comment: arXiv admin note: substantial text overlap with arXiv:2007.0116