5,216 research outputs found
Constrained stochastic blackbox optimization using a progressive barrier and probabilistic estimates
This work introduces the StoMADS-PB algorithm for constrained stochastic
blackbox optimization, which is an extension of the mesh adaptive direct-search
(MADS) method originally developed for deterministic blackbox optimization
under general constraints. The values of the objective and constraint functions
are provided by a noisy blackbox, i.e., they can only be computed with random
noise whose distribution is unknown. As in MADS, constraint violations are
aggregated into a single constraint violation function. Since all functions
values are numerically unavailable, StoMADS-PB uses estimates and introduces
so-called probabilistic bounds for the violation. Such estimates and bounds
obtained from stochastic observations are required to be accurate and reliable
with high but fixed probabilities. The proposed method, which allows
intermediate infeasible iterates, accepts new points using sufficient decrease
conditions and imposing a threshold on the probabilistic bounds. Using Clarke
nonsmooth calculus and martingale theory, Clarke stationarity convergence
results for the objective and the violation function are derived with
probability one
Making Indefinite Kernel Learning Practical
In this paper we embed evolutionary computation into statistical learning theory. First, we outline the connection between large margin optimization and statistical learning and see why this paradigm is successful for many pattern recognition problems. We then embed evolutionary computation into the most prominent representative of this class of learning methods, namely into Support Vector Machines (SVM). In contrast to former applications of evolutionary algorithms to SVM we do not only optimize the method or kernel parameters. We rather use evolution strategies in order to directly solve the posed constrained optimization problem. Transforming the problem into the Wolfe dual reduces the total runtime and allows the usage of kernel functions just as for traditional SVM. We will show that evolutionary SVM are at least as accurate as their quadratic programming counterparts on eight real-world benchmark data sets in terms of generalization performance. They always outperform traditional approaches in terms of the original optimization problem. Additionally, the proposed algorithm is more generic than existing traditional solutions since it will also work for non-positive semidefinite or indefinite kernel functions. The evolutionary SVM variants frequently outperform their quadratic programming competitors in cases where such an indefinite Kernel function is used. --
Discrete-Continuous ADMM for Transductive Inference in Higher-Order MRFs
This paper introduces a novel algorithm for transductive inference in
higher-order MRFs, where the unary energies are parameterized by a variable
classifier. The considered task is posed as a joint optimization problem in the
continuous classifier parameters and the discrete label variables. In contrast
to prior approaches such as convex relaxations, we propose an advantageous
decoupling of the objective function into discrete and continuous subproblems
and a novel, efficient optimization method related to ADMM. This approach
preserves integrality of the discrete label variables and guarantees global
convergence to a critical point. We demonstrate the advantages of our approach
in several experiments including video object segmentation on the DAVIS data
set and interactive image segmentation
On the Use of Surrogate Functions for Mixed Variable Optimization of Simulated Systems
This research considers the efficient numerical solution of linearly constrained mixed variable programming (MVP) problems, in which the objective function is a black-box stochastic simulation, function evaluations may be computationally expensive, and derivative information is typically not available. MVP problems are those with a mixture of continuous, integer, and categorical variables, the latter of which may take on values only from a predefined list and may even be non-numeric. Mixed Variable Generalized Pattern Search with Ranking and Selection (MGPS-RS) is the only existing, provably convergent algorithm that can be applied to this class of problems. Present in this algorithm is an optional framework for constructing and managing less expensive surrogate functions as a means to reduce the number of true function evaluations that are required to find approximate solutions. In this research, the NOMADm software package, an implementation of pattern search for deterministic MVP problems, is modified to incorporate a sequential selection with memory (SSM) ranking and selection procedure for handling stochastic problems. In doing so, the underlying algorithm is modified to make the application of surrogates more efficient. A second class of surrogates based on the Nadaraya-Watson kernel regression estimator is also added to the software. Preliminary computational testing of the modified software is performed to characterize the relative efficiency of selected surrogate functions for mixed variable optimization in simulated systems
Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information
We consider variants of trust-region and cubic regularization methods for
non-convex optimization, in which the Hessian matrix is approximated. Under
mild conditions on the inexact Hessian, and using approximate solution of the
corresponding sub-problems, we provide iteration complexity to achieve -approximate second-order optimality which have shown to be tight.
Our Hessian approximation conditions constitute a major relaxation over the
existing ones in the literature. Consequently, we are able to show that such
mild conditions allow for the construction of the approximate Hessian through
various random sampling methods. In this light, we consider the canonical
problem of finite-sum minimization, provide appropriate uniform and non-uniform
sub-sampling strategies to construct such Hessian approximations, and obtain
optimal iteration complexity for the corresponding sub-sampled trust-region and
cubic regularization methods.Comment: 32 page
- …