42,695 research outputs found
Let's Make Block Coordinate Descent Go Fast: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
Block coordinate descent (BCD) methods are widely-used for large-scale
numerical optimization because of their cheap iteration costs, low memory
requirements, amenability to parallelization, and ability to exploit problem
structure. Three main algorithmic choices influence the performance of BCD
methods: the block partitioning strategy, the block selection rule, and the
block update rule. In this paper we explore all three of these building blocks
and propose variations for each that can lead to significantly faster BCD
methods. We (i) propose new greedy block-selection strategies that guarantee
more progress per iteration than the Gauss-Southwell rule; (ii) explore
practical issues like how to implement the new rules when using "variable"
blocks; (iii) explore the use of message-passing to compute matrix or Newton
updates efficiently on huge blocks for problems with a sparse dependency
between variables; and (iv) consider optimal active manifold identification,
which leads to bounds on the "active set complexity" of BCD methods and leads
to superlinear convergence for certain problems with sparse solutions (and in
some cases finite termination at an optimal solution). We support all of our
findings with numerical results for the classic machine learning problems of
least squares, logistic regression, multi-class logistic regression, label
propagation, and L1-regularization
PasMoQAP: A Parallel Asynchronous Memetic Algorithm for solving the Multi-Objective Quadratic Assignment Problem
Multi-Objective Optimization Problems (MOPs) have attracted growing attention
during the last decades. Multi-Objective Evolutionary Algorithms (MOEAs) have
been extensively used to address MOPs because are able to approximate a set of
non-dominated high-quality solutions. The Multi-Objective Quadratic Assignment
Problem (mQAP) is a MOP. The mQAP is a generalization of the classical QAP
which has been extensively studied, and used in several real-life applications.
The mQAP is defined as having as input several flows between the facilities
which generate multiple cost functions that must be optimized simultaneously.
In this study, we propose PasMoQAP, a parallel asynchronous memetic algorithm
to solve the Multi-Objective Quadratic Assignment Problem. PasMoQAP is based on
an island model that structures the population by creating sub-populations. The
memetic algorithm on each island individually evolve a reduced population of
solutions, and they asynchronously cooperate by sending selected solutions to
the neighboring islands. The experimental results show that our approach
significatively outperforms all the island-based variants of the
multi-objective evolutionary algorithm NSGA-II. We show that PasMoQAP is a
suitable alternative to solve the Multi-Objective Quadratic Assignment Problem.Comment: 8 pages, 3 figures, 2 tables. Accepted at Conference on Evolutionary
Computation 2017 (CEC 2017
A geometric view of cryptographic equation solving
This paper considers the geometric properties of the Relinearisation algorithm and of the XL algorithm used in cryptology for equation solving. We give a formal description of each algorithm in terms of projective geometry, making particular use of the Veronese variety. We establish the fundamental geometrical connection between the two algorithms and show how both algorithms can be viewed as being equivalent to the problem of finding a matrix of low rank in the linear span of a collection of matrices, a problem sometimes known as the MinRank problem. Furthermore, we generalise the XL algorithm to a geometrically invariant algorithm, which we term the GeometricXL algorithm. The GeometricXL algorithm is a technique which can solve certain equation systems that are not easily soluble by the XL algorithm or by Groebner basis methods
Generic-case complexity, decision problems in group theory and random walks
We give a precise definition of ``generic-case complexity'' and show that for
a very large class of finitely generated groups the classical decision problems
of group theory - the word, conjugacy and membership problems - all have
linear-time generic-case complexity. We prove such theorems by using the theory
of random walks on regular graphs.Comment: Revised versio
- …