21,400 research outputs found
Adversarially Robust Optimization with Gaussian Processes
In this paper, we consider the problem of Gaussian process (GP) optimization
with an added robustness requirement: The returned point may be perturbed by an
adversary, and we require the function value to remain as high as possible even
after this perturbation. This problem is motivated by settings in which the
underlying functions during optimization and implementation stages are
different, or when one is interested in finding an entire region of good inputs
rather than only a single point. We show that standard GP optimization
algorithms do not exhibit the desired robustness properties, and provide a
novel confidence-bound based algorithm StableOpt for this purpose. We
rigorously establish the required number of samples for StableOpt to find a
near-optimal point, and we complement this guarantee with an
algorithm-independent lower bound. We experimentally demonstrate several
potential applications of interest using real-world data sets, and we show that
StableOpt consistently succeeds in finding a stable maximizer where several
baseline methods fail.Comment: Corrected typo
Probabilistic Kernel Support Vector Machines
We propose a probabilistic enhancement of standard kernel Support Vector
Machines for binary classification, in order to address the case when, along
with given data sets, a description of uncertainty (e.g., error bounds) may be
available on each datum. In the present paper, we specifically consider
Gaussian distributions to model uncertainty. Thereby, our data consist of pairs
, , along with an indicator
to declare membership in one of two categories for each pair.
These pairs may be viewed to represent the mean and covariance, respectively,
of random vectors taking values in a suitable linear space (typically
). Thus, our setting may also be viewed as a modification of
Support Vector Machines to classify distributions, albeit, at present, only
Gaussian ones. We outline the formalism that allows computing suitable
classifiers via a natural modification of the standard "kernel trick." The main
contribution of this work is to point out a suitable kernel function for
applying Support Vector techniques to the setting of uncertain data for which a
detailed uncertainty description is also available (herein, "Gaussian points").Comment: 6 pages, 6 figure
Dynamic Control of Explore/Exploit Trade-Off In Bayesian Optimization
Bayesian optimization offers the possibility of optimizing black-box
operations not accessible through traditional techniques. The success of
Bayesian optimization methods such as Expected Improvement (EI) are
significantly affected by the degree of trade-off between exploration and
exploitation. Too much exploration can lead to inefficient optimization
protocols, whilst too much exploitation leaves the protocol open to strong
initial biases, and a high chance of getting stuck in a local minimum.
Typically, a constant margin is used to control this trade-off, which results
in yet another hyper-parameter to be optimized. We propose contextual
improvement as a simple, yet effective heuristic to counter this - achieving a
one-shot optimization strategy. Our proposed heuristic can be swiftly
calculated and improves both the speed and robustness of discovery of optimal
solutions. We demonstrate its effectiveness on both synthetic and real world
problems and explore the unaccounted for uncertainty in the pre-determination
of search hyperparameters controlling explore-exploit trade-off.Comment: Accepted for publication in the proceedings of 2018 Computing
Conferenc
- …