12,653 research outputs found
Cognitive Medium Access: Exploration, Exploitation and Competition
This paper establishes the equivalence between cognitive medium access and
the competitive multi-armed bandit problem. First, the scenario in which a
single cognitive user wishes to opportunistically exploit the availability of
empty frequency bands in the spectrum with multiple bands is considered. In
this scenario, the availability probability of each channel is unknown to the
cognitive user a priori. Hence efficient medium access strategies must strike a
balance between exploring the availability of other free channels and
exploiting the opportunities identified thus far. By adopting a Bayesian
approach for this classical bandit problem, the optimal medium access strategy
is derived and its underlying recursive structure is illustrated via examples.
To avoid the prohibitive computational complexity of the optimal strategy, a
low complexity asymptotically optimal strategy is developed. The proposed
strategy does not require any prior statistical knowledge about the traffic
pattern on the different channels. Next, the multi-cognitive user scenario is
considered and low complexity medium access protocols, which strike the optimal
balance between exploration and exploitation in such competitive environments,
are developed. Finally, this formalism is extended to the case in which each
cognitive user is capable of sensing and using multiple channels
simultaneously.Comment: Submitted to IEEE/ACM Trans. on Networking, 14 pages, 2 figure
PAC-Bayesian Analysis of the Exploration-Exploitation Trade-off
We develop a coherent framework for integrative simultaneous analysis of the
exploration-exploitation and model order selection trade-offs. We improve over
our preceding results on the same subject (Seldin et al., 2011) by combining
PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a
combination is also of independent interest for studies of multiple
simultaneously evolving martingales.Comment: On-line Trading of Exploration and Exploitation 2 - ICML-2011
workshop. http://explo.cs.ucl.ac.uk/workshop
Superior Exploration-Exploitation Balance with Quantum-Inspired Hadamard Walks
This paper extends the analogies employed in the development of
quantum-inspired evolutionary algorithms by proposing quantum-inspired Hadamard
walks, called QHW. A novel quantum-inspired evolutionary algorithm, called
HQEA, for solving combinatorial optimization problems, is also proposed. The
novelty of HQEA lies in it's incorporation of QHW Remote Search and QHW Local
Search - the quantum equivalents of classical mutation and local search, that
this paper defines. The intuitive reasoning behind this approach, and the
exploration-exploitation balance thus occurring is explained. From the results
of the experiments carried out on the 0,1-knapsack problem, HQEA performs
significantly better than a conventional genetic algorithm, CGA, and two
quantum-inspired evolutionary algorithms - QEA and NQEA, in terms of
convergence speed and accuracy.Comment: 2 pages, 2 figures, 1 table, late-breakin
Better safe than sorry: Risky function exploitation through safe optimization
Exploration-exploitation of functions, that is learning and optimizing a
mapping between inputs and expected outputs, is ubiquitous to many real world
situations. These situations sometimes require us to avoid certain outcomes at
all cost, for example because they are poisonous, harmful, or otherwise
dangerous. We test participants' behavior in scenarios in which they have to
find the optimum of a function while at the same time avoid outputs below a
certain threshold. In two experiments, we find that Safe-Optimization, a
Gaussian Process-based exploration-exploitation algorithm, describes
participants' behavior well and that participants seem to care firstly whether
a point is safe and then try to pick the optimal point from all such safe
points. This means that their trade-off between exploration and exploitation
can be seen as an intelligent, approximate, and homeostasis-driven strategy.Comment: 6 pages, submitted to Cognitive Science Conferenc
- …