2,227 research outputs found
An Induced Natural Selection Heuristic for Finding Optimal Bayesian Experimental Designs
Bayesian optimal experimental design has immense potential to inform the
collection of data so as to subsequently enhance our understanding of a variety
of processes. However, a major impediment is the difficulty in evaluating
optimal designs for problems with large, or high-dimensional, design spaces. We
propose an efficient search heuristic suitable for general optimisation
problems, with a particular focus on optimal Bayesian experimental design
problems. The heuristic evaluates the objective (utility) function at an
initial, randomly generated set of input values. At each generation of the
algorithm, input values are "accepted" if their corresponding objective
(utility) function satisfies some acceptance criteria, and new inputs are
sampled about these accepted points. We demonstrate the new algorithm by
evaluating the optimal Bayesian experimental designs for the previously
considered death, pharmacokinetic and logistic regression models. Comparisons
to the current "gold-standard" method are given to demonstrate the proposed
algorithm as a computationally-efficient alternative for moderately-large
design problems (i.e., up to approximately 40-dimensions)
Dynamic Intrusion Detection in Resource-Constrained Cyber Networks
We consider a large-scale cyber network with N components (e.g., paths,
servers, subnets). Each component is either in a healthy state (0) or an
abnormal state (1). Due to random intrusions, the state of each component
transits from 0 to 1 over time according to certain stochastic process. At each
time, a subset of K (K < N) components are checked and those observed in
abnormal states are fixed. The objective is to design the optimal scheduling
for intrusion detection such that the long-term network cost incurred by all
abnormal components is minimized. We formulate the problem as a special class
of Restless Multi-Armed Bandit (RMAB) process. A general RMAB suffers from the
curse of dimensionality (PSPACE-hard) and numerical methods are often
inapplicable. We show that, for this class of RMAB, Whittle index exists and
can be obtained in closed form, leading to a low-complexity implementation of
Whittle index policy with a strong performance. For homogeneous components,
Whittle index policy is shown to have a simple structure that does not require
any prior knowledge on the intrusion processes. Based on this structure,
Whittle index policy is further shown to be optimal over a finite time horizon
with an arbitrary length. Beyond intrusion detection, these results also find
applications in queuing networks with finite-size buffers.Comment: 9 pages, 5 figure
An Online Approach to Dynamic Channel Access and Transmission Scheduling
Making judicious channel access and transmission scheduling decisions is
essential for improving performance as well as energy and spectral efficiency
in multichannel wireless systems. This problem has been a subject of extensive
study in the past decade, and the resulting dynamic and opportunistic channel
access schemes can bring potentially significant improvement over traditional
schemes. However, a common and severe limitation of these dynamic schemes is
that they almost always require some form of a priori knowledge of the channel
statistics. A natural remedy is a learning framework, which has also been
extensively studied in the same context, but a typical learning algorithm in
this literature seeks only the best static policy, with performance measured by
weak regret, rather than learning a good dynamic channel access policy. There
is thus a clear disconnect between what an optimal channel access policy can
achieve with known channel statistics that actively exploits temporal, spatial
and spectral diversity, and what a typical existing learning algorithm aims
for, which is the static use of a single channel devoid of diversity gain. In
this paper we bridge this gap by designing learning algorithms that track known
optimal or sub-optimal dynamic channel access and transmission scheduling
policies, thereby yielding performance measured by a form of strong regret, the
accumulated difference between the reward returned by an optimal solution when
a priori information is available and that by our online algorithm. We do so in
the context of two specific algorithms that appeared in [1] and [2],
respectively, the former for a multiuser single-channel setting and the latter
for a single-user multichannel setting. In both cases we show that our
algorithms achieve sub-linear regret uniform in time and outperforms the
standard weak-regret learning algorithms.Comment: 10 pages, to appear in MobiHoc 201
Characteristics of the polymer transport in ratchet systems
Molecules with complex internal structure in time-dependent periodic
potentials are studied by using short Rubinstein-Duke model polymers as an
example. We extend our earlier work on transport in stochastically varying
potentials to cover also deterministic potential switching mechanisms,
energetic efficiency and non-uniform charge distributions. We also use currents
in the non-equilibrium steady state to identify the dominating mechanisms that
lead to polymer transportation and analyze the evolution of the macroscopic
state (e.g., total and head-to-head lengths) of the polymers. Several numerical
methods are used to solve the master equations and nonlinear optimization
problems. The dominating transport mechanisms are found via graph optimization
methods. The results show that small changes in the molecule structure and the
environment variables can lead to large increases of the drift. The drift and
the coherence can be amplified by using deterministic flashing potentials and
customized polymer charge distributions. Identifying the dominating transport
mechanism by graph analysis tools is found to give insight in how the molecule
is transported by the ratchet effect.Comment: 35 pages, 17 figures, to appear in Phys. Rev.
Reinforcement Learning: A Survey
This paper surveys the field of reinforcement learning from a
computer-science perspective. It is written to be accessible to researchers
familiar with machine learning. Both the historical basis of the field and a
broad selection of current work are summarized. Reinforcement learning is the
problem faced by an agent that learns behavior through trial-and-error
interactions with a dynamic environment. The work described here has a
resemblance to work in psychology, but differs considerably in the details and
in the use of the word ``reinforcement.'' The paper discusses central issues of
reinforcement learning, including trading off exploration and exploitation,
establishing the foundations of the field via Markov decision theory, learning
from delayed reinforcement, constructing empirical models to accelerate
learning, making use of generalization and hierarchy, and coping with hidden
state. It concludes with a survey of some implemented systems and an assessment
of the practical utility of current methods for reinforcement learning.Comment: See http://www.jair.org/ for any accompanying file
The History of the Quantitative Methods in Finance Conference Series. 1992-2007
This report charts the history of the Quantitative Methods in Finance (QMF) conference from its beginning in 1993 to the 15th conference in 2007. It lists alphabetically the 1037 speakers who presented at all 15 conferences and the titles of their papers.
- …