Adaptive density estimation for stationary processes

We propose an algorithm to estimate the common density $s$ of a stationary process $X_1,...,X_n$. We suppose that the process is either $\beta$ or $\tau$-mixing. We provide a model selection procedure based on a generalization of Mallows' $C_p$ and we prove oracle inequalities for the selected estimator under a few prior assumptions on the collection of models and on the mixing coefficients. We prove that our estimator is adaptive over a class of Besov spaces, namely, we prove that it achieves the same rates of convergence as in the i.i.d framework

Covariance Matrix Adaptation Pareto Archived Evolution Strategy with Hypervolume-sorted Adaptive Grid Algorithm.

Real-world problems often involve the optimisation of multiple conflicting objectives. These problems, referred to as multi-objective optimisation problems, are especially challenging when more than three objectives are considered simultaneously. This paper proposes an algorithm to address this class of problems. The proposed algorithm is an evolutionary algorithm based on an evolution strategy framework, and more specifically, on the Covariance Matrix Adaptation Pareto Archived Evolution Strategy (CMA-PAES). A novel selection mechanism is introduced and integrated within the framework. This selection mechanism makes use of an adaptive grid to perform a local approximation of the hypervolume indicator which is then used as a selection criterion. The proposed implementation, named Covariance Matrix Adaptation Pareto Archived Evolution Strategy with Hypervolume-sorted Adaptive Grid Algorithm (CMA-PAES-HAGA), overcomes the limitation of CMA-PAES in handling more than two objectives and displays a remarkably good performance on a scalable test suite in five, seven, and ten-objective problems. The performance of CMA-PAES-HAGA has been compared with that of a competition winning meta-heuristic, representing the state-of-the-art in this sub-field of multi-objective optimisation. The proposed algorithm has been tested in a seven-objective real-world application, i.e. the design of an aircraft lateral control system. In this optimisation problem, CMA-PAES-HAGA greatly outperformed its competitors

Elastic-Net Regularization in Learning Theory

Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its consistency properties, we set up a suitable mathematical framework. Our setting is random-design regression where we allow the response variable to be vector-valued and we consider prediction functions which are linear combination of elements ({\em features}) in an infinite-dimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular ``{\em elastic-net representation}'' of the regression function such that, if the number of data increases, the elastic-net estimator is consistent not only for prediction but also for variable/feature selection. Our results include finite-sample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elastic-net solution which is different from the optimization procedure originally proposed by Zou and HastieComment: 32 pages, 3 figure

Online Selection of CMA-ES Variants

In the field of evolutionary computation, one of the most challenging topics is algorithm selection. Knowing which heuristics to use for which optimization problem is key to obtaining high-quality solutions. We aim to extend this research topic by taking a first step towards a selection method for adaptive CMA-ES algorithms. We build upon the theoretical work done by van Rijn \textit{et al.} [PPSN'18], in which the potential of switching between different CMA-ES variants was quantified in the context of a modular CMA-ES framework. We demonstrate in this work that their proposed approach is not very reliable, in that implementing the suggested adaptive configurations does not yield the predicted performance gains. We propose a revised approach, which results in a more robust fit between predicted and actual performance. The adaptive CMA-ES approach obtains performance gains on 18 out of 24 tested functions of the BBOB benchmark, with stable advantages of up to 23\%. An analysis of module activation indicates which modules are most crucial for the different phases of optimizing each of the 24 benchmark problems. The module activation also suggests that additional gains are possible when including the (B)IPOP modules, which we have excluded for this present work.Comment: To appear at Genetic and Evolutionary Computation Conference (GECCO'19) Appendix will be added in due tim