29,127 research outputs found
Direct search based on probabilistic descent in reduced spaces
Derivative-free algorithms seek the minimum value of a given objective
function without using any derivative information. The performance of these
methods often worsen as the dimension increases, a phenomenon predicted by
their worst-case complexity guarantees. Nevertheless, recent algorithmic
proposals have shown that incorporating randomization into otherwise
deterministic frameworks could alleviate this effect for direct-search methods.
The best guarantees and practical performance are obtained when employing a
random vector and its negative, which amounts to drawing directions in a random
one-dimensional subspace. Unlike for other derivative-free schemes, however,
the properties of these subspaces have not been exploited.
In this paper, we study a generic direct-search algorithm in which the
polling directions are defined using random subspaces. Complexity guarantees
for such an approach are derived thanks to probabilistic properties related to
both the subspaces and the directions used within these subspaces. By
leveraging results on random subspace embeddings and sketching matrices, we
show that better complexity bounds are obtained for randomized instances of our
framework. A numerical investigation confirms the benefit of randomization,
particularly when done in subspaces, when solving problems of moderately large
dimension
Genetic algorithms with DNN-based trainable crossover as an example of partial specialization of general search
Universal induction relies on some general search procedure that is doomed to
be inefficient. One possibility to achieve both generality and efficiency is to
specialize this procedure w.r.t. any given narrow task. However, complete
specialization that implies direct mapping from the task parameters to
solutions (discriminative models) without search is not always possible. In
this paper, partial specialization of general search is considered in the form
of genetic algorithms (GAs) with a specialized crossover operator. We perform a
feasibility study of this idea implementing such an operator in the form of a
deep feedforward neural network. GAs with trainable crossover operators are
compared with the result of complete specialization, which is also represented
as a deep neural network. Experimental results show that specialized GAs can be
more efficient than both general GAs and discriminative models.Comment: AGI 2017 procedding, The final publication is available at
link.springer.co
Optimising Spatial and Tonal Data for PDE-based Inpainting
Some recent methods for lossy signal and image compression store only a few
selected pixels and fill in the missing structures by inpainting with a partial
differential equation (PDE). Suitable operators include the Laplacian, the
biharmonic operator, and edge-enhancing anisotropic diffusion (EED). The
quality of such approaches depends substantially on the selection of the data
that is kept. Optimising this data in the domain and codomain gives rise to
challenging mathematical problems that shall be addressed in our work.
In the 1D case, we prove results that provide insights into the difficulty of
this problem, and we give evidence that a splitting into spatial and tonal
(i.e. function value) optimisation does hardly deteriorate the results. In the
2D setting, we present generic algorithms that achieve a high reconstruction
quality even if the specified data is very sparse. To optimise the spatial
data, we use a probabilistic sparsification, followed by a nonlocal pixel
exchange that avoids getting trapped in bad local optima. After this spatial
optimisation we perform a tonal optimisation that modifies the function values
in order to reduce the global reconstruction error. For homogeneous diffusion
inpainting, this comes down to a least squares problem for which we prove that
it has a unique solution. We demonstrate that it can be found efficiently with
a gradient descent approach that is accelerated with fast explicit diffusion
(FED) cycles. Our framework allows to specify the desired density of the
inpainting mask a priori. Moreover, is more generic than other data
optimisation approaches for the sparse inpainting problem, since it can also be
extended to nonlinear inpainting operators such as EED. This is exploited to
achieve reconstructions with state-of-the-art quality.
We also give an extensive literature survey on PDE-based image compression
methods
Feature selection using genetic algorithms and probabilistic neural networks
Selection of input variables is a key stage in building
predictive models, and an important form of data mining. As exhaustive evaluation of potential input sets using full non-linear models is impractical, it is necessary to use simple fast-evaluating models and heuristic selection strategies. This paper discusses a fast, efficient, and powerful nonlinear input selection procedure using a combination of Probabilistic Neural Networks and repeated
bitwise gradient descent. The algorithm is compared
with forward elimination, backward elimination and genetic algorithms using a selection of real-world data sets. The algorithm has comparative performance and greatly reduced execution time with respect to these alternative approaches. It is demonstrated empirically that reliable results cannot be gained using any of these approaches without the use of resampling
Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
We present global convergence rates for a line-search method which is based
on random first-order models and directions whose quality is ensured only with
certain probability. We show that in terms of the order of the accuracy, the
evaluation complexity of such a method is the same as its counterparts that use
deterministic accurate models; the use of probabilistic models only increases
the complexity by a constant, which depends on the probability of the models
being good. We particularize and improve these results in the convex and
strongly convex case.
We also analyze a probabilistic cubic regularization variant that allows
approximate probabilistic second-order models and show improved complexity
bounds compared to probabilistic first-order methods; again, as a function of
the accuracy, the probabilistic cubic regularization bounds are of the same
(optimal) order as for the deterministic case
- …