7,625 research outputs found
A Fast Eigenvalue Approach for Solving the Trust Region Subproblem with an Additional Linear Inequality
In this paper, we study the extended trust region subproblem (eTRS) in which
the trust region intersects the unit ball with a single linear inequality
constraint. By reformulating the Lagrangian dual of eTRS as a two-parameter
linear eigenvalue problem, we state a necessary and sufficient condition for
its strong duality in terms of an optimal solution of a linearly constrained
bivariate concave maximization problem. This results in an efficient algorithm
for solving eTRS of large size whenever the strong duality is detected.
Finally, some numerical experiments are given to show the effectiveness of the
proposed method
Derivative-free optimization methods
In many optimization problems arising from scientific, engineering and
artificial intelligence applications, objective and constraint functions are
available only as the output of a black-box or simulation oracle that does not
provide derivative information. Such settings necessitate the use of methods
for derivative-free, or zeroth-order, optimization. We provide a review and
perspectives on developments in these methods, with an emphasis on highlighting
recent developments and on unifying treatment of such problems in the
non-linear optimization and machine learning literature. We categorize methods
based on assumed properties of the black-box functions, as well as features of
the methods. We first overview the primary setting of deterministic methods
applied to unconstrained, non-convex optimization problems where the objective
function is defined by a deterministic black-box oracle. We then discuss
developments in randomized methods, methods that assume some additional
structure about the objective (including convexity, separability and general
non-smooth compositions), methods for problems where the output of the
black-box oracle is stochastic, and methods for handling different types of
constraints
Accelerated Inference in Markov Random Fields via Smooth Riemannian Optimization
Markov Random Fields (MRFs) are a popular model for several pattern
recognition and reconstruction problems in robotics and computer vision.
Inference in MRFs is intractable in general and related work resorts to
approximation algorithms. Among those techniques, semidefinite programming
(SDP) relaxations have been shown to provide accurate estimates while scaling
poorly with the problem size and being typically slow for practical
applications. Our first contribution is to design a dual ascent method to solve
standard SDP relaxations that takes advantage of the geometric structure of the
problem to speed up computation. This technique, named Dual Ascent Riemannian
Staircase (DARS), is able to solve large problem instances in seconds. Our
second contribution is to develop a second and faster approach. The backbone of
this second approach is a novel SDP relaxation combined with a fast and
scalable solver based on smooth Riemannian optimization. We show that this
approach, named Fast Unconstrained SEmidefinite Solver (FUSES), can solve large
problems in milliseconds. Contrarily to local MRF solvers, e.g., loopy belief
propagation, our approaches do not require an initial guess. Moreover, we
leverage recent results from optimization theory to provide per-instance
sub-optimality guarantees. We demonstrate the proposed approaches in
multi-class image segmentation problems. Extensive experimental evidence shows
that (i) FUSES and DARS produce near-optimal solutions, attaining an objective
within 0.1% of the optimum, (ii) FUSES and DARS are remarkably faster than
general-purpose SDP solvers, and FUSES is more than two orders of magnitude
faster than DARS while attaining similar solution quality, (iii) FUSES is
faster than local search methods while being a global solver.Comment: 16 page
Data-Driven Combined State and Parameter Reduction for Extreme-Scale Inverse Problems
In this contribution we present an accelerated optimization-based approach
for combined state and parameter reduction of a parametrized linear control
system which is then used as a surrogate model in a Bayesian inverse setting.
Following the basic ideas presented in [Lieberman, Willcox, Ghattas. Parameter
and state model reduction for large-scale statistical inverse settings, SIAM J.
Sci. Comput., 32(5):2523-2542, 2010], our approach is based on a generalized
data-driven optimization functional in the construction process of the
surrogate model and the usage of a trust-region-type solution strategy that
results in an additional speed-up of the overall method. In principal, the
model reduction procedure is based on the offline construction of appropriate
low-dimensional state and parameter spaces and an online inversion step based
on the resulting surrogate model that is obtained through projection of the
underlying control system onto the reduced spaces. The generalization and
enhancements presented in this work are shown to decrease overall computational
time and increase accuracy of the reduced order model and thus allow an
application to extreme-scale problems. Numerical experiments for a generic
model and a fMRI connectivity model are presented in order to compare the
computational efficiency of our improved method with the original approach.Comment: Preprin
Constrained Optimization for Liquid Crystal Equilibria: Extended Results
This paper investigates energy-minimization finite-element approaches for the
computation of nematic liquid crystal equilibrium configurations. We compare
the performance of these methods when the necessary unit-length constraint is
enforced by either continuous Lagrange multipliers or a penalty functional.
Building on previous work in [1,2], the penalty method is derived and the
linearizations within the nonlinear iteration are shown to be well-posed under
certain assumptions. In addition, the paper discusses the effects of tailored
trust-region methods and nested iteration for both formulations. Such methods
are aimed at increasing the efficiency and robustness of each algorithms'
nonlinear iterations. Three representative, free-elastic, equilibrium problems
are considered to examine each method's performance. The first two
configurations have analytical solutions and, therefore, convergence to the
true solution is considered. The third problem considers more complicated
boundary conditions, relevant in ongoing research, simulating surface
nano-patterning. A multigrid approach is introduced and tested for a
flexoelectrically coupled model to establish scalability for highly complicated
applications. The Lagrange multiplier method is found to outperform the penalty
method in a number of measures, trust regions are shown to improve robustness,
and nested iteration proves highly effective at reducing computational costs.Comment: 28 Pages, 8 Figures, 15 Tables, 5 Procedures. Added and removed
references, as well as some minor figure rearrangemen
A new alternating direction trust region method based on conic model for solving unconstrained optimization
In this paper, a new alternating direction trust region method based on conic
model is used to solve unconstrained optimization problems. By use of the
alternating direction method, the new conic model trust region subproblem is
solved by two steps in two orthogonal directions. This new idea overcomes the
shortcomings of conic model subproblem which is difficult to solve. Then the
global convergence of the method under some reasonable conditions is
established. Numerical experiment shows that this method may be better than the
dogleg method to solve the subproblem, especially for large-scale problems.Comment: 18 pages,3 table
Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey
Financial portfolio management is one of the problems that are most
frequently encountered in the investment industry. Nevertheless, it is not
widely recognized that both Kelly Criterion and Risk Parity collapse into Mean
Variance under some conditions, which implies that a universal solution to the
portfolio optimization problem could potentially exist. In fact, the process of
sequential computation of optimal component weights that maximize the
portfolio's expected return subject to a certain risk budget can be
reformulated as a discrete-time Markov Decision Process (MDP) and hence as a
stochastic optimal control, where the system being controlled is a portfolio
consisting of multiple investment components, and the control is its component
weights. Consequently, the problem could be solved using model-free
Reinforcement Learning (RL) without knowing specific component dynamics. By
examining existing methods of both value-based and policy-based model-free RL
for the portfolio optimization problem, we identify some of the key unresolved
questions and difficulties facing today's portfolio managers of applying
model-free RL to their investment portfolios
A New Class of Scaling Matrices for Scaled Trust Region Algorithms
A new class of affine scaling matrices for the interior point Newton-type
methods is considered to solve the nonlinear systems with simple bounds. We
review the essential properties of a scaling matrix and consider several
well-known scaling matrices proposed in the literature. We define a new scaling
matrix that is the convex combination of these matrices. The proposed scaling
matrix inherits those interesting properties of the individual matrices and
satisfies additional desired requirements. The numerical experiments
demonstrate the superiority of the new scaling matrix in solving several
important test problems
High-order convergent Finite-Elements Direct Transcription Method for Constrained Optimal Control Problems
In this paper we present a finite element method for the direct transcription
of constrained non-linear optimal control problems.
We prove that our method converges of high order under mild assumptions. Our
analysis uses a regularized penalty-barrier functional. The convergence result
is obtained from local strict convexity and Lipschitz-continuity of this
functional in the finite-element space.
The method is very flexible. Each component of the numerical solution can be
discretized with a different mesh. General differential-algebraic constraints
of arbitrary index can be treated easily with this new method.
From the discretization results an unconstrained non-linear programming
problem (NLP) with penalty- and barrier-terms. The derivatives of the NLP
functions have a sparsity pattern that can be analysed and tailored in terms of
the chosen finite-element bases in an easy way. We discuss how to treat the
resulting NLP in a practical way with general-purpose software for constrained
non-linear programming.Comment: 23 page
A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications
Bilevel optimization is defined as a mathematical program, where an
optimization problem contains another optimization problem as a constraint.
These problems have received significant attention from the mathematical
programming community. Only limited work exists on bilevel problems using
evolutionary computation techniques; however, recently there has been an
increasing interest due to the proliferation of practical applications and the
potential of evolutionary algorithms in tackling these problems. This paper
provides a comprehensive review on bilevel optimization from the basic
principles to solution strategies; both classical and evolutionary. A number of
potential application problems are also discussed. To offer the readers
insights on the prominent developments in the field of bilevel optimization, we
have performed an automated text-analysis of an extended list of papers
published on bilevel optimization to date. This paper should motivate
evolutionary computation researchers to pay more attention to this practical
yet challenging area
- …