Optimal Parameter Choices Through Self-Adjustment: Applying the 1/5-th Rule in Discrete Settings

While evolutionary algorithms are known to be very successful for a broad range of applications, the algorithm designer is often left with many algorithmic choices, for example, the size of the population, the mutation rates, and the crossover rates of the algorithm. These parameters are known to have a crucial influence on the optimization time, and thus need to be chosen carefully, a task that often requires substantial efforts. Moreover, the optimal parameters can change during the optimization process. It is therefore of great interest to design mechanisms that dynamically choose best-possible parameters. An example for such an update mechanism is the one-fifth success rule for step-size adaption in evolutionary strategies. While in continuous domains this principle is well understood also from a mathematical point of view, no comparable theory is available for problems in discrete domains. In this work we show that the one-fifth success rule can be effective also in discrete settings. We regard the $(1+(\lambda,\lambda))$~GA proposed in [Doerr/Doerr/Ebel: From black-box complexity to designing new genetic algorithms, TCS 2015]. We prove that if its population size is chosen according to the one-fifth success rule then the expected optimization time on \textsc{OneMax} is linear. This is better than what \emph{any} static population size $\lambda$ can achieve and is asymptotically optimal also among all adaptive parameter choices.Comment: This is the full version of a paper that is to appear at GECCO 201

Solving rank structured Sylvester and Lyapunov equations

We consider the problem of efficiently solving Sylvester and Lyapunov equations of medium and large scale, in case of rank-structured data, i.e., when the coefficient matrices and the right-hand side have low-rank off-diagonal blocks. This comprises problems with banded data, recently studied by Haber and Verhaegen in "Sparse solution of the Lyapunov equation for large-scale interconnected systems", Automatica, 2016, and by Palitta and Simoncini in "Numerical methods for large-scale Lyapunov equations with symmetric banded data", SISC, 2018, which often arise in the discretization of elliptic PDEs. We show that, under suitable assumptions, the quasiseparable structure is guaranteed to be numerically present in the solution, and explicit novel estimates of the numerical rank of the off-diagonal blocks are provided. Efficient solution schemes that rely on the technology of hierarchical matrices are described, and several numerical experiments confirm the applicability and efficiency of the approaches. We develop a MATLAB toolbox that allows easy replication of the experiments and a ready-to-use interface for the solvers. The performances of the different approaches are compared, and we show that the new methods described are efficient on several classes of relevant problems

Generalized modularity matrices

Various modularity matrices appeared in the recent literature on network analysis and algebraic graph theory. Their purpose is to allow writing as quadratic forms certain combinatorial functions appearing in the framework of graph clustering problems. In this paper we put in evidence certain common traits of various modularity matrices and shed light on their spectral properties that are at the basis of various theoretical results and practical spectral-type algorithms for community detection

Randomized Local Model Order Reduction

In this paper we propose local approximation spaces for localized model order reduction procedures such as domain decomposition and multiscale methods. Those spaces are constructed from local solutions of the partial differential equation (PDE) with random boundary conditions, yield an approximation that converges provably at a nearly optimal rate, and can be generated at close to optimal computational complexity. In many localized model order reduction approaches like the generalized finite element method, static condensation procedures, and the multiscale finite element method local approximation spaces can be constructed by approximating the range of a suitably defined transfer operator that acts on the space of local solutions of the PDE. Optimal local approximation spaces that yield in general an exponentially convergent approximation are given by the left singular vectors of this transfer operator [I. Babu\v{s}ka and R. Lipton 2011, K. Smetana and A. T. Patera 2016]. However, the direct calculation of these singular vectors is computationally very expensive. In this paper, we propose an adaptive randomized algorithm based on methods from randomized linear algebra [N. Halko et al. 2011], which constructs a local reduced space approximating the range of the transfer operator and thus the optimal local approximation spaces. The adaptive algorithm relies on a probabilistic a posteriori error estimator for which we prove that it is both efficient and reliable with high probability. Several numerical experiments confirm the theoretical findings.Comment: 31 pages, 14 figures, 1 table, 1 algorith

Compositional uniformity, domain patterning and the mechanism underlying nano-chessboard arrays

We propose that systems exhibiting compositional patterning at the nanoscale, so far assumed to be due to some kind of ordered phase segregation, can be understood instead in terms of coherent, single phase ordering of minority motifs, caused by some constrained drive for uniformity. The essential features of this type of arrangements can be reproduced using a superspace construction typical of uniformity-driven orderings, which only requires the knowledge of the modulation vectors observed in the diffraction patterns. The idea is discussed in terms of a simple two dimensional lattice-gas model that simulates a binary system in which the dilution of the minority component is favored. This simple model already exhibits a hierarchy of arrangements similar to the experimentally observed nano-chessboard and nano-diamond patterns, which are described as occupational modulated structures with two independent modulation wave vectors and simple step-like occupation modulation functions.Comment: Preprint. 11 pages, 11 figure

Unsupervised Domain Adaptation by Backpropagation

Top-performing deep architectures are trained on massive amounts of labeled data. In the absence of labeled data for a certain task, domain adaptation often provides an attractive option given that labeled data of similar nature but from a different domain (e.g. synthetic images) are available. Here, we propose a new approach to domain adaptation in deep architectures that can be trained on large amount of labeled data from the source domain and large amount of unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of "deep" features that are (i) discriminative for the main learning task on the source domain and (ii) invariant with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a simple new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation. Overall, the approach can be implemented with little effort using any of the deep-learning packages. The method performs very well in a series of image classification experiments, achieving adaptation effect in the presence of big domain shifts and outperforming previous state-of-the-art on Office datasets