7,625 research outputs found

    A Fast Eigenvalue Approach for Solving the Trust Region Subproblem with an Additional Linear Inequality

    Full text link
    In this paper, we study the extended trust region subproblem (eTRS) in which the trust region intersects the unit ball with a single linear inequality constraint. By reformulating the Lagrangian dual of eTRS as a two-parameter linear eigenvalue problem, we state a necessary and sufficient condition for its strong duality in terms of an optimal solution of a linearly constrained bivariate concave maximization problem. This results in an efficient algorithm for solving eTRS of large size whenever the strong duality is detected. Finally, some numerical experiments are given to show the effectiveness of the proposed method

    Derivative-free optimization methods

    Full text link
    In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints

    Accelerated Inference in Markov Random Fields via Smooth Riemannian Optimization

    Full text link
    Markov Random Fields (MRFs) are a popular model for several pattern recognition and reconstruction problems in robotics and computer vision. Inference in MRFs is intractable in general and related work resorts to approximation algorithms. Among those techniques, semidefinite programming (SDP) relaxations have been shown to provide accurate estimates while scaling poorly with the problem size and being typically slow for practical applications. Our first contribution is to design a dual ascent method to solve standard SDP relaxations that takes advantage of the geometric structure of the problem to speed up computation. This technique, named Dual Ascent Riemannian Staircase (DARS), is able to solve large problem instances in seconds. Our second contribution is to develop a second and faster approach. The backbone of this second approach is a novel SDP relaxation combined with a fast and scalable solver based on smooth Riemannian optimization. We show that this approach, named Fast Unconstrained SEmidefinite Solver (FUSES), can solve large problems in milliseconds. Contrarily to local MRF solvers, e.g., loopy belief propagation, our approaches do not require an initial guess. Moreover, we leverage recent results from optimization theory to provide per-instance sub-optimality guarantees. We demonstrate the proposed approaches in multi-class image segmentation problems. Extensive experimental evidence shows that (i) FUSES and DARS produce near-optimal solutions, attaining an objective within 0.1% of the optimum, (ii) FUSES and DARS are remarkably faster than general-purpose SDP solvers, and FUSES is more than two orders of magnitude faster than DARS while attaining similar solution quality, (iii) FUSES is faster than local search methods while being a global solver.Comment: 16 page

    Data-Driven Combined State and Parameter Reduction for Extreme-Scale Inverse Problems

    Full text link
    In this contribution we present an accelerated optimization-based approach for combined state and parameter reduction of a parametrized linear control system which is then used as a surrogate model in a Bayesian inverse setting. Following the basic ideas presented in [Lieberman, Willcox, Ghattas. Parameter and state model reduction for large-scale statistical inverse settings, SIAM J. Sci. Comput., 32(5):2523-2542, 2010], our approach is based on a generalized data-driven optimization functional in the construction process of the surrogate model and the usage of a trust-region-type solution strategy that results in an additional speed-up of the overall method. In principal, the model reduction procedure is based on the offline construction of appropriate low-dimensional state and parameter spaces and an online inversion step based on the resulting surrogate model that is obtained through projection of the underlying control system onto the reduced spaces. The generalization and enhancements presented in this work are shown to decrease overall computational time and increase accuracy of the reduced order model and thus allow an application to extreme-scale problems. Numerical experiments for a generic model and a fMRI connectivity model are presented in order to compare the computational efficiency of our improved method with the original approach.Comment: Preprin

    Constrained Optimization for Liquid Crystal Equilibria: Extended Results

    Full text link
    This paper investigates energy-minimization finite-element approaches for the computation of nematic liquid crystal equilibrium configurations. We compare the performance of these methods when the necessary unit-length constraint is enforced by either continuous Lagrange multipliers or a penalty functional. Building on previous work in [1,2], the penalty method is derived and the linearizations within the nonlinear iteration are shown to be well-posed under certain assumptions. In addition, the paper discusses the effects of tailored trust-region methods and nested iteration for both formulations. Such methods are aimed at increasing the efficiency and robustness of each algorithms' nonlinear iterations. Three representative, free-elastic, equilibrium problems are considered to examine each method's performance. The first two configurations have analytical solutions and, therefore, convergence to the true solution is considered. The third problem considers more complicated boundary conditions, relevant in ongoing research, simulating surface nano-patterning. A multigrid approach is introduced and tested for a flexoelectrically coupled model to establish scalability for highly complicated applications. The Lagrange multiplier method is found to outperform the penalty method in a number of measures, trust regions are shown to improve robustness, and nested iteration proves highly effective at reducing computational costs.Comment: 28 Pages, 8 Figures, 15 Tables, 5 Procedures. Added and removed references, as well as some minor figure rearrangemen

    A new alternating direction trust region method based on conic model for solving unconstrained optimization

    Full text link
    In this paper, a new alternating direction trust region method based on conic model is used to solve unconstrained optimization problems. By use of the alternating direction method, the new conic model trust region subproblem is solved by two steps in two orthogonal directions. This new idea overcomes the shortcomings of conic model subproblem which is difficult to solve. Then the global convergence of the method under some reasonable conditions is established. Numerical experiment shows that this method may be better than the dogleg method to solve the subproblem, especially for large-scale problems.Comment: 18 pages,3 table

    Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey

    Full text link
    Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimization problem could potentially exist. In fact, the process of sequential computation of optimal component weights that maximize the portfolio's expected return subject to a certain risk budget can be reformulated as a discrete-time Markov Decision Process (MDP) and hence as a stochastic optimal control, where the system being controlled is a portfolio consisting of multiple investment components, and the control is its component weights. Consequently, the problem could be solved using model-free Reinforcement Learning (RL) without knowing specific component dynamics. By examining existing methods of both value-based and policy-based model-free RL for the portfolio optimization problem, we identify some of the key unresolved questions and difficulties facing today's portfolio managers of applying model-free RL to their investment portfolios

    A New Class of Scaling Matrices for Scaled Trust Region Algorithms

    Full text link
    A new class of affine scaling matrices for the interior point Newton-type methods is considered to solve the nonlinear systems with simple bounds. We review the essential properties of a scaling matrix and consider several well-known scaling matrices proposed in the literature. We define a new scaling matrix that is the convex combination of these matrices. The proposed scaling matrix inherits those interesting properties of the individual matrices and satisfies additional desired requirements. The numerical experiments demonstrate the superiority of the new scaling matrix in solving several important test problems

    High-order convergent Finite-Elements Direct Transcription Method for Constrained Optimal Control Problems

    Full text link
    In this paper we present a finite element method for the direct transcription of constrained non-linear optimal control problems. We prove that our method converges of high order under mild assumptions. Our analysis uses a regularized penalty-barrier functional. The convergence result is obtained from local strict convexity and Lipschitz-continuity of this functional in the finite-element space. The method is very flexible. Each component of the numerical solution can be discretized with a different mesh. General differential-algebraic constraints of arbitrary index can be treated easily with this new method. From the discretization results an unconstrained non-linear programming problem (NLP) with penalty- and barrier-terms. The derivatives of the NLP functions have a sparsity pattern that can be analysed and tailored in terms of the chosen finite-element bases in an easy way. We discuss how to treat the resulting NLP in a practical way with general-purpose software for constrained non-linear programming.Comment: 23 page

    A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications

    Full text link
    Bilevel optimization is defined as a mathematical program, where an optimization problem contains another optimization problem as a constraint. These problems have received significant attention from the mathematical programming community. Only limited work exists on bilevel problems using evolutionary computation techniques; however, recently there has been an increasing interest due to the proliferation of practical applications and the potential of evolutionary algorithms in tackling these problems. This paper provides a comprehensive review on bilevel optimization from the basic principles to solution strategies; both classical and evolutionary. A number of potential application problems are also discussed. To offer the readers insights on the prominent developments in the field of bilevel optimization, we have performed an automated text-analysis of an extended list of papers published on bilevel optimization to date. This paper should motivate evolutionary computation researchers to pay more attention to this practical yet challenging area
    • …
    corecore