690 research outputs found
Maximum Likelihood Associative Memories
Associative memories are structures that store data in such a way that it can
later be retrieved given only a part of its content -- a sort-of
error/erasure-resilience property. They are used in applications ranging from
caches and memory management in CPUs to database engines. In this work we study
associative memories built on the maximum likelihood principle. We derive
minimum residual error rates when the data stored comes from a uniform binary
source. Second, we determine the minimum amount of memory required to store the
same data. Finally, we bound the computational complexity for message
retrieval. We then compare these bounds with two existing associative memory
architectures: the celebrated Hopfield neural networks and a neural network
architecture introduced more recently by Gripon and Berrou
Complexity Theory for Discrete Black-Box Optimization Heuristics
A predominant topic in the theory of evolutionary algorithms and, more
generally, theory of randomized black-box optimization techniques is running
time analysis. Running time analysis aims at understanding the performance of a
given heuristic on a given problem by bounding the number of function
evaluations that are needed by the heuristic to identify a solution of a
desired quality. As in general algorithms theory, this running time perspective
is most useful when it is complemented by a meaningful complexity theory that
studies the limits of algorithmic solutions.
In the context of discrete black-box optimization, several black-box
complexity models have been developed to analyze the best possible performance
that a black-box optimization algorithm can achieve on a given problem. The
models differ in the classes of algorithms to which these lower bounds apply.
This way, black-box complexity contributes to a better understanding of how
certain algorithmic choices (such as the amount of memory used by a heuristic,
its selective pressure, or properties of the strategies that it uses to create
new solution candidates) influences performance.
In this chapter we review the different black-box complexity models that have
been proposed in the literature, survey the bounds that have been obtained for
these models, and discuss how the interplay of running time analysis and
black-box complexity can inspire new algorithmic solutions to well-researched
problems in evolutionary computation. We also discuss in this chapter several
interesting open questions for future work.Comment: This survey article is to appear (in a slightly modified form) in the
book "Theory of Randomized Search Heuristics in Discrete Search Spaces",
which will be published by Springer in 2018. The book is edited by Benjamin
Doerr and Frank Neumann. Missing numbers of pointers to other chapters of
this book will be added as soon as possibl
Faster Black-Box Algorithms Through Higher Arity Operators
We extend the work of Lehre and Witt (GECCO 2010) on the unbiased black-box
model by considering higher arity variation operators. In particular, we show
that already for binary operators the black-box complexity of \leadingones
drops from for unary operators to . For \onemax, the
unary black-box complexity drops to O(n) in the binary case.
For -ary operators, , the \onemax-complexity further decreases to
.Comment: To appear at FOGA 201
A sticky HDP-HMM with application to speaker diarization
We consider the problem of speaker diarization, the problem of segmenting an
audio recording of a meeting into temporal segments corresponding to individual
speakers. The problem is rendered particularly difficult by the fact that we
are not allowed to assume knowledge of the number of people participating in
the meeting. To address this problem, we take a Bayesian nonparametric approach
to speaker diarization that builds on the hierarchical Dirichlet process hidden
Markov model (HDP-HMM) of Teh et al. [J. Amer. Statist. Assoc. 101 (2006)
1566--1581]. Although the basic HDP-HMM tends to over-segment the audio
data---creating redundant states and rapidly switching among them---we describe
an augmented HDP-HMM that provides effective control over the switching rate.
We also show that this augmentation makes it possible to treat emission
distributions nonparametrically. To scale the resulting architecture to
realistic diarization problems, we develop a sampling algorithm that employs a
truncated approximation of the Dirichlet process to jointly resample the full
state sequence, greatly improving mixing rates. Working with a benchmark NIST
data set, we show that our Bayesian nonparametric architecture yields
state-of-the-art speaker diarization results.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS395 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Artificial Immune Systems for Combinatorial Optimisation: A Theoretical Investigation
We focus on the clonal selection inspired computational models of the immune system developed for general-purpose optimisation.
Our aim is to highlight when these artificial immune systems (AIS) are more efficient than evolutionary algorithms (EAs).
Compared to traditional EAs, AIS use considerably higher mutation rates (hypermutations) for variation, give higher selection probabilities to more recent solutions
and lower selection probabilities to older ones (ageing). We consider the standard Opt-IA that includes both of the AIS distinguishing features and argue why it is of greater applicability than other popular AIS. Our first result is the proof that the stop at first constructive mutation version of its hypermutation operator is essential.
Without it, the hypermutations cannot optimise any function with an arbitrary polynomial number of optima. Afterwards we show that the hypermutations are exponentially faster
than the standard bit mutation operator used in traditional EAs at escaping from local optima of standard benchmark function classes and of the NP-hard Partition problem.
If the basin of attraction of the local optima is not too large, then ageing allows even greater speed-ups. For the
Cliff benchmark function this can make the difference between exponential and quasi-linear expected time.
If the basin of attraction is too large, then ageing can implicitly detect the local optimum and escape it by automatically restarting the search process.
The described power of hypermutations and ageing allows us to prove that they guarantee (1+epsilon) approximations for Partition in expected polynomial time for any constant epsilon.
These features come at the expense of the hypermutations being a linear factor slower than EAs for standard unimodal benchmark functions and of eliminating the power of ageing at escaping local optima in the complete Opt-IA.
We show that hypermutating with inversely proportional rates mitigates such drawbacks at the expense of losing the explorative advantages of the standard operator.
We conclude the thesis by designing fast hypermutation operators that are provably a linear factor faster than the traditional ones for the unimodal benchmark functions and Partition, while maintaining explorative power and working in harmony together with ageing
- âŠ