The Sampling-and-Learning Framework: A Statistical View of Evolutionary Algorithms

Evolutionary algorithms (EAs), a large class of general purpose optimization algorithms inspired from the natural phenomena, are widely used in various industrial optimizations and often show excellent performance. This paper presents an attempt towards revealing their general power from a statistical view of EAs. By summarizing a large range of EAs into the sampling-and-learning framework, we show that the framework directly admits a general analysis on the probable-absolute-approximate (PAA) query complexity. We particularly focus on the framework with the learning subroutine being restricted as a binary classification, which results in the sampling-and-classification (SAC) algorithms. With the help of the learning theory, we obtain a general upper bound on the PAA query complexity of SAC algorithms. We further compare SAC algorithms with the uniform search in different situations. Under the error-target independence condition, we show that SAC algorithms can achieve polynomial speedup to the uniform search, but not super-polynomial speedup. Under the one-side-error condition, we show that super-polynomial speedup can be achieved. This work only touches the surface of the framework. Its power under other conditions is still open

Kinematic Basis of Emergent Energetics of Complex Dynamics

Stochastic kinematic description of a complex dynamics is shown to dictate an energetic and thermodynamic structure. An energy function $\varphi(x)$ emerges as the limit of the generalized, nonequilibrium free energy of a Markovian dynamics with vanishing fluctuations. In terms of the $\nabla\varphi$ and its orthogonal field $\gamma(x)\perp\nabla\varphi$, a general vector field $b(x)$ can be decomposed into $-D(x)\nabla\varphi+\gamma$, where $\nabla\cdot\big(\omega(x)\gamma(x)\big)=$ $-\nabla\omega D(x)\nabla\varphi$. The matrix $D(x)$ and scalar $\omega(x)$, two additional characteristics to the $b(x)$ alone, represent the local geometry and density of states intrinsic to the statistical motion in the state space at $x$. $\varphi(x)$ and $\omega(x)$ are interpreted as the emergent energy and degeneracy of the motion, with an energy balance equation $d\varphi(x(t))/dt=\gamma D^{-1}\gamma-bD^{-1}b$, reflecting the geometrical $\|D\nabla\varphi\|^2+\|\gamma\|^2=\|b\|^2$. The partition function employed in statistical mechanics and J. W. Gibbs' method of ensemble change naturally arise; a fluctuation-dissipation theorem is established via the two leading-order asymptotics of entropy production as $\epsilon\to 0$. The present theory provides a mathematical basis for P. W. Anderson's emergent behavior in the hierarchical structure of complexity science.Comment: 7 page

ZOOpt: Toolbox for Derivative-Free Optimization

Recent advances of derivative-free optimization allow efficient approximating the global optimal solutions of sophisticated functions, such as functions with many local optima, non-differentiable and non-continuous functions. This article describes the ZOOpt (https://github.com/eyounx/ZOOpt) toolbox that provides efficient derivative-free solvers and are designed easy to use. ZOOpt provides a Python package for single-thread optimization, and a light-weighted distributed version with the help of the Julia language for Python described functions. ZOOpt toolbox particularly focuses on optimization problems in machine learning, addressing high-dimensional, noisy, and large-scale problems. The toolbox is being maintained toward ready-to-use tool in real-world machine learning tasks

Collaborative Inference of Coexisting Information Diffusions

Recently, \textit{diffusion history inference} has become an emerging research topic due to its great benefits for various applications, whose purpose is to reconstruct the missing histories of information diffusion traces according to incomplete observations. The existing methods, however, often focus only on single information diffusion trace, while in a real-world social network, there often coexist multiple information diffusions over the same network. In this paper, we propose a novel approach called Collaborative Inference Model (CIM) for the problem of the inference of coexisting information diffusions. By exploiting the synergism between the coexisting information diffusions, CIM holistically models multiple information diffusions as a sparse 4th-order tensor called Coexisting Diffusions Tensor (CDT) without any prior assumption of diffusion models, and collaboratively infers the histories of the coexisting information diffusions via a low-rank approximation of CDT with a fusion of heterogeneous constraints generated from additional data sources. To improve the efficiency, we further propose an optimal algorithm called Time Window based Parallel Decomposition Algorithm (TWPDA), which can speed up the inference without compromise on the accuracy by utilizing the temporal locality of information diffusions. The extensive experiments conducted on real world datasets and synthetic datasets verify the effectiveness and efficiency of CIM and TWPDA