4,368 research outputs found
Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications
We extend the work of Narasimhan and Bilmes [30] for minimizing set functions
representable as a difference between submodular functions. Similar to [30],
our new algorithms are guaranteed to monotonically reduce the objective
function at every step. We empirically and theoretically show that the
per-iteration cost of our algorithms is much less than [30], and our algorithms
can be used to efficiently minimize a difference between submodular functions
under various combinatorial constraints, a problem not previously addressed. We
provide computational bounds and a hardness result on the mul- tiplicative
inapproximability of minimizing the difference between submodular functions. We
show, however, that it is possible to give worst-case additive bounds by
providing a polynomial time computable lower-bound on the minima. Finally we
show how a number of machine learning problems can be modeled as minimizing the
difference between submodular functions. We experimentally show the validity of
our algorithms by testing them on the problem of feature selection with
submodular cost features.Comment: 17 pages, 8 figures. A shorter version of this appeared in Proc.
Uncertainty in Artificial Intelligence (UAI), Catalina Islands, 201
OneMax in Black-Box Models with Several Restrictions
Black-box complexity studies lower bounds for the efficiency of
general-purpose black-box optimization algorithms such as evolutionary
algorithms and other search heuristics. Different models exist, each one being
designed to analyze a different aspect of typical heuristics such as the memory
size or the variation operators in use. While most of the previous works focus
on one particular such aspect, we consider in this work how the combination of
several algorithmic restrictions influence the black-box complexity. Our
testbed are so-called OneMax functions, a classical set of test functions that
is intimately related to classic coin-weighing problems and to the board game
Mastermind.
We analyze in particular the combined memory-restricted ranking-based
black-box complexity of OneMax for different memory sizes. While its isolated
memory-restricted as well as its ranking-based black-box complexity for bit
strings of length is only of order , the combined model does not
allow for algorithms being faster than linear in , as can be seen by
standard information-theoretic considerations. We show that this linear bound
is indeed asymptotically tight. Similar results are obtained for other memory-
and offspring-sizes. Our results also apply to the (Monte Carlo) complexity of
OneMax in the recently introduced elitist model, in which only the best-so-far
solution can be kept in the memory. Finally, we also provide improved lower
bounds for the complexity of OneMax in the regarded models.
Our result enlivens the quest for natural evolutionary algorithms optimizing
OneMax in iterations.Comment: This is the full version of a paper accepted to GECCO 201
Towards a Theory-Guided Benchmarking Suite for Discrete Black-Box Optimization Heuristics: Profiling EA Variants on OneMax and LeadingOnes
Theoretical and empirical research on evolutionary computation methods
complement each other by providing two fundamentally different approaches
towards a better understanding of black-box optimization heuristics. In
discrete optimization, both streams developed rather independently of each
other, but we observe today an increasing interest in reconciling these two
sub-branches. In continuous optimization, the COCO (COmparing Continuous
Optimisers) benchmarking suite has established itself as an important platform
that theoreticians and practitioners use to exchange research ideas and
questions. No widely accepted equivalent exists in the research domain of
discrete black-box optimization.
Marking an important step towards filling this gap, we adjust the COCO
software to pseudo-Boolean optimization problems, and obtain from this a
benchmarking environment that allows a fine-grained empirical analysis of
discrete black-box heuristics. In this documentation we demonstrate how this
test bed can be used to profile the performance of evolutionary algorithms.
More concretely, we study the optimization behavior of several EA
variants on the two benchmark problems OneMax and LeadingOnes. This comparison
motivates a refined analysis for the optimization time of the EA
on LeadingOnes
- …