On Constructing Ensembles for Combinatorial Optimisation

Although the use of ensemble methods in machine-learning is ubiquitous due to their proven ability to outperform their constituent algorithms, ensembles of optimisation algorithms have received relatively little attention. Existing approaches lag behind machine-learning in both theory and practice, with no principled design guidelines available. In this paper, we address fundamental questions regarding ensemble composition in opti-misation using the domain of bin-packing as a example; in particular we investigate the trade-off between accuracy and diversity, and whether diversity metrics can be used as a proxy for constructing an ensemble, proposing a number of novel metrics for comparing algorithm diversity. We find that randomly composed ensembles can outperform ensembles of high-performing algorithms under certain conditions and that judicious choice of diversity metric is required to construct good ensembles. The method and findings can be generalised to any meta-heuristic ensemble, and lead to better understanding of how to undertake principled ensemble design

Molecular self-organisation in a developmental model for the evolution of large-scale artificial neural networks

We argue that molecular self-organisation during embryonic development allows evolution to perform highly nonlinear combinatorial optimisation. A structured approach to architectural optimisation of large-scale Artificial Neural Networks using this principle is presented. We also present simulation results demonstrating the evolution of an edge detecting retina using the proposed methodology

Ant system-based feature set partitioning algorithm for K-NN and LDA ensembles construction

Combination of several classifiers has been very useful in improving the prediction accuracy and in most situations multiple classifiers perform better than single classifier.However not all combining approaches are successful at producing multiple classifiers with good classification accuracy because there is no standard resolution in constructing diverse and accurate classifier ensemble.This paper proposes ant system-based feature set partitioning algorithm in constructing k-nearest neighbor (k-NN) and linear discriminant analysis (LDA) ensembles. Experiments were performed on several University California, Irvine datasets to test the performance of the proposed algorithm.Experimental results showed that the proposed algorithm has successfully constructed better classifier ensemble for k-NN and LDA

The Peculiar Phase Structure of Random Graph Bisection

The mincut graph bisection problem involves partitioning the n vertices of a graph into disjoint subsets, each containing exactly n/2 vertices, while minimizing the number of "cut" edges with an endpoint in each subset. When considered over sparse random graphs, the phase structure of the graph bisection problem displays certain familiar properties, but also some surprises. It is known that when the mean degree is below the critical value of 2 log 2, the cutsize is zero with high probability. We study how the minimum cutsize increases with mean degree above this critical threshold, finding a new analytical upper bound that improves considerably upon previous bounds. Combined with recent results on expander graphs, our bound suggests the unusual scenario that random graph bisection is replica symmetric up to and beyond the critical threshold, with a replica symmetry breaking transition possibly taking place above the threshold. An intriguing algorithmic consequence is that although the problem is NP-hard, we can find near-optimal cutsizes (whose ratio to the optimal value approaches 1 asymptotically) in polynomial time for typical instances near the phase transition.Comment: substantially revised section 2, changed figures 3, 4 and 6, made minor stylistic changes and added reference

Adapting Quality Assurance to Adaptive Systems: The Scenario Coevolution Paradigm

From formal and practical analysis, we identify new challenges that self-adaptive systems pose to the process of quality assurance. When tackling these, the effort spent on various tasks in the process of software engineering is naturally re-distributed. We claim that all steps related to testing need to become self-adaptive to match the capabilities of the self-adaptive system-under-test. Otherwise, the adaptive system's behavior might elude traditional variants of quality assurance. We thus propose the paradigm of scenario coevolution, which describes a pool of test cases and other constraints on system behavior that evolves in parallel to the (in part autonomous) development of behavior in the system-under-test. Scenario coevolution offers a simple structure for the organization of adaptive testing that allows for both human-controlled and autonomous intervention, supporting software engineering for adaptive systems on a procedural as well as technical level.Comment: 17 pages, published at ISOLA 201

Sample-efficient Multi-objective Molecular Optimization with GFlowNets

Many crucial scientific problems involve designing novel molecules with desired properties, which can be formulated as a black-box optimization problem over the discrete chemical space. In practice, multiple conflicting objectives and costly evaluations (e.g., wet-lab experiments) make the diversity of candidates paramount. Computational methods have achieved initial success but still struggle with considering diversity in both objective and search space. To fill this gap, we propose a multi-objective Bayesian optimization (MOBO) algorithm leveraging the hypernetwork-based GFlowNets (HN-GFN) as an acquisition function optimizer, with the purpose of sampling a diverse batch of candidate molecular graphs from an approximate Pareto front. Using a single preference-conditioned hypernetwork, HN-GFN learns to explore various trade-offs between objectives. We further propose a hindsight-like off-policy strategy to share high-performing molecules among different preferences in order to speed up learning for HN-GFN. We empirically illustrate that HN-GFN has adequate capacity to generalize over preferences. Moreover, experiments in various real-world MOBO settings demonstrate that our framework predominantly outperforms existing methods in terms of candidate quality and sample efficiency. The code is available at https://github.com/violet-sto/HN-GFN.Comment: NeurIPS 202