40,358 research outputs found
An informational approach to the global optimization of expensive-to-evaluate functions
In many global optimization problems motivated by engineering applications,
the number of function evaluations is severely limited by time or cost. To
ensure that each evaluation contributes to the localization of good candidates
for the role of global minimizer, a sequential choice of evaluation points is
usually carried out. In particular, when Kriging is used to interpolate past
evaluations, the uncertainty associated with the lack of information on the
function can be expressed and used to compute a number of criteria accounting
for the interest of an additional evaluation at any given point. This paper
introduces minimizer entropy as a new Kriging-based criterion for the
sequential choice of points at which the function should be evaluated. Based on
\emph{stepwise uncertainty reduction}, it accounts for the informational gain
on the minimizer expected from a new evaluation. The criterion is approximated
using conditional simulations of the Gaussian process model behind Kriging, and
then inserted into an algorithm similar in spirit to the \emph{Efficient Global
Optimization} (EGO) algorithm. An empirical comparison is carried out between
our criterion and \emph{expected improvement}, one of the reference criteria in
the literature. Experimental results indicate major evaluation savings over
EGO. Finally, the method, which we call IAGO (for Informational Approach to
Global Optimization) is extended to robust optimization problems, where both
the factors to be tuned and the function evaluations are corrupted by noise.Comment: Accepted for publication in the Journal of Global Optimization (This
is the revised version, with additional details on computational problems,
and some grammatical changes
Recommended from our members
The effect of missing values using genetic programming on evolvable diagnosis
Medical databases usually contain missing values due the policy of
reducing stress and harm to the patient. In practice missing values has been a
problem mainly due to the necessity to evaluate mathematical equations obtained
by genetic programming. The solution to this problem is to use fill in methods to
estimate the missing values. This paper analyses three fill in methods: (1) attribute
means, (2) conditional means, and (3) random number generation. The methods
are evaluated using sensitivity, specificity, and entropy to explain the exchange in
knowledge of the results. The results are illustrated based on the breast cancer
database. Conditional means produced the best fill in experimental results
An Information-Theoretic Analysis of Thompson Sampling
We provide an information-theoretic analysis of Thompson sampling that
applies across a broad range of online optimization problems in which a
decision-maker must learn from partial feedback. This analysis inherits the
simplicity and elegance of information theory and leads to regret bounds that
scale with the entropy of the optimal-action distribution. This strengthens
preexisting results and yields new insight into how information improves
performance
Uncertainty Reduction for Stochastic Processes on Complex Networks
Many real-world systems are characterized by stochastic dynamical rules where
a complex network of interactions among individual elements probabilistically
determines their state. Even with full knowledge of the network structure and
of the stochastic rules, the ability to predict system configurations is
generally characterized by a large uncertainty. Selecting a fraction of the
nodes and observing their state may help to reduce the uncertainty about the
unobserved nodes. However, choosing these points of observation in an optimal
way is a highly nontrivial task, depending on the nature of the stochastic
process and on the structure of the underlying interaction pattern. In this
paper, we introduce a computationally efficient algorithm to determine
quasioptimal solutions to the problem. The method leverages network sparsity to
reduce computational complexity from exponential to almost quadratic, thus
allowing the straightforward application of the method to mid-to-large-size
systems. Although the method is exact only for equilibrium stochastic processes
defined on trees, it turns out to be effective also for out-of-equilibrium
processes on sparse loopy networks.Comment: 5 pages, 2 figures + Supplemental Material. A python implementation
of the algorithm is available at
https://github.com/filrad/Maximum-Entropy-Samplin
- …