107,702 research outputs found

    Escaping the Local Minima via Simulated Annealing: Optimization of Approximately Convex Functions

    Full text link
    We consider the problem of optimizing an approximately convex function over a bounded convex set in Rn\mathbb{R}^n using only function evaluations. The problem is reduced to sampling from an \emph{approximately} log-concave distribution using the Hit-and-Run method, which is shown to have the same O∗\mathcal{O}^* complexity as sampling from log-concave distributions. In addition to extend the analysis for log-concave distributions to approximate log-concave distributions, the implementation of the 1-dimensional sampler of the Hit-and-Run walk requires new methods and analysis. The algorithm then is based on simulated annealing which does not relies on first order conditions which makes it essentially immune to local minima. We then apply the method to different motivating problems. In the context of zeroth order stochastic convex optimization, the proposed method produces an ϵ\epsilon-minimizer after O∗(n7.5ϵ−2)\mathcal{O}^*(n^{7.5}\epsilon^{-2}) noisy function evaluations by inducing a O(ϵ/n)\mathcal{O}(\epsilon/n)-approximately log concave distribution. We also consider in detail the case when the "amount of non-convexity" decays towards the optimum of the function. Other applications of the method discussed in this work include private computation of empirical risk minimizers, two-stage stochastic programming, and approximate dynamic programming for online learning.Comment: 27 page

    Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs

    Get PDF
    The idea of computer vision as the Bayesian inverse problem to computer graphics has a long history and an appealing elegance, but it has proved difficult to directly implement. Instead, most vision tasks are approached via complex bottom-up processing pipelines. Here we show that it is possible to write short, simple probabilistic graphics programs that define flexible generative models and to automatically invert them to interpret real-world images. Generative probabilistic graphics programs consist of a stochastic scene generator, a renderer based on graphics software, a stochastic likelihood model linking the renderer's output and the data, and latent variables that adjust the fidelity of the renderer and the tolerance of the likelihood model. Representations and algorithms from computer graphics, originally designed to produce high-quality images, are instead used as the deterministic backbone for highly approximate and stochastic generative models. This formulation combines probabilistic programming, computer graphics, and approximate Bayesian computation, and depends only on general-purpose, automatic inference techniques. We describe two applications: reading sequences of degraded and adversarially obscured alphanumeric characters, and inferring 3D road models from vehicle-mounted camera images. Each of the probabilistic graphics programs we present relies on under 20 lines of probabilistic code, and supports accurate, approximately Bayesian inferences about ambiguous real-world images.Comment: The first two authors contributed equally to this wor

    On Optimization of Discontinuous Systems

    Get PDF
    In this paper stochastic programming techniques are adapted and further developed for applications to discrete event systems. We consider cases when the sample path of the system depend discontinuously on control parameters (e.g. modeling of failures, several competing processes), which could make the computation of estimates of the gradient difficult. Methods which use only samples of the performance criterion are developed, in particular finite differences with reduced variance and concurrent approximation and optimization algorithms. Optimization of the stationary behavior is also considered. Results of numerical experiments and convergence results are reported
    • …
    corecore