190,410 research outputs found
Best Subset Selection via a Modern Optimization Lens
In the last twenty-five years (1990-2014), algorithmic advances in integer
optimization combined with hardware improvements have resulted in an
astonishing 200 billion factor speedup in solving Mixed Integer Optimization
(MIO) problems. We present a MIO approach for solving the classical best subset
selection problem of choosing out of features in linear regression
given observations. We develop a discrete extension of modern first order
continuous optimization methods to find high quality feasible solutions that we
use as warm starts to a MIO solver that finds provably optimal solutions. The
resulting algorithm (a) provides a solution with a guarantee on its
suboptimality even if we terminate the algorithm early, (b) can accommodate
side constraints on the coefficients of the linear regression and (c) extends
to finding best subset solutions for the least absolute deviation loss
function. Using a wide variety of synthetic and real datasets, we demonstrate
that our approach solves problems with in the 1000s and in the 100s in
minutes to provable optimality, and finds near optimal solutions for in the
100s and in the 1000s in minutes. We also establish via numerical
experiments that the MIO approach performs better than {\texttt {Lasso}} and
other popularly used sparse learning procedures, in terms of achieving sparse
solutions with good predictive power.Comment: This is a revised version (May, 2015) of the first submission in June
201
Immersion-based model predictive control of constrained nonlinear systems: Polyflow approximation
In the framework of Model Predictive Control (MPC), the control input is
typically computed by solving optimization problems repeatedly online. For
general nonlinear systems, the online optimization problems are non-convex and
computationally expensive or even intractable. In this paper, we propose to
circumvent this issue by computing a high-dimensional linear embedding of
discrete-time nonlinear systems. The computation relies on an algebraic
condition related to the immersibility property of nonlinear systems and can be
implemented offline. With the high-dimensional linear model, we then define and
solve a convex online MPC problem. We also provide an interpretation of our
approach under the Koopman operator framework.Comment: Accepted to the European Control Conferenc
A discrete approximation of Blake & Zisserman energy in image denoising and optimal choice of regularization parameters
We consider a multi-scale approach for the discrete approximation of a functional proposed by Bake and Zisserman (BZ) for solving image denoising and segmentation problems. The proposed method is based on simple and effective higher order varia-tional model. It consists of building linear discrete energies family which Γ-converges to the non-linear BZ functional. The key point of the approach is the construction of the diffusion operators in the discrete energies within a finite element adaptive procedure which approximate in the Γ-convergence sense the initial energy including the singular parts. The resulting model preserves the singularities of the image and of its gradient while keeping a simple structure of the underlying PDEs, hence efficient numerical method for solving the problem under consideration. A new point to make this approach work is to deal with constrained optimization problems that we circumvent through a Lagrangian formulation. We present some numerical experiments to show that the proposed approach allows us to detect first and second-order singularities. We also consider and implement to enhance the algorithms and convergence properties, an augmented Lagrangian method using the alternating direction method of Multipliers (ADMM)
Qualitative Characteristics and Quantitative Measures of Solution's Reliability in Discrete Optimization: Traditional Analytical Approaches, Innovative Computational Methods and Applicability
The purpose of this thesis is twofold. The first and major part is devoted to
sensitivity analysis of various discrete optimization problems while the second
part addresses methods applied for calculating measures of solution stability
and solving multicriteria discrete optimization problems.
Despite numerous approaches to stability analysis of discrete optimization
problems two major directions can be single out: quantitative and qualitative.
Qualitative sensitivity analysis is conducted for multicriteria discrete optimization
problems with minisum, minimax and minimin partial criteria. The main
results obtained here are necessary and sufficient conditions for different stability
types of optimal solutions (or a set of optimal solutions) of the considered
problems.
Within the framework of quantitative direction various measures of solution
stability are investigated. A formula for a quantitative characteristic called
stability radius is obtained for the generalized equilibrium situation invariant
to changes of game parameters in the case of the H¨older metric. Quality of the
problem solution can also be described in terms of robustness analysis. In this
work the concepts of accuracy and robustness tolerances are presented for a
strategic game with a finite number of players where initial coefficients (costs)
of linear payoff functions are subject to perturbations.
Investigation of stability radius also aims to devise methods for its calculation.
A new metaheuristic approach is derived for calculation of stability
radius of an optimal solution to the shortest path problem. The main advantage
of the developed method is that it can be potentially applicable for
calculating stability radii of NP-hard problems.
The last chapter of the thesis focuses on deriving innovative methods based
on interactive optimization approach for solving multicriteria combinatorial
optimization problems. The key idea of the proposed approach is to utilize
a parameterized achievement scalarizing function for solution calculation and
to direct interactive procedure by changing weighting coefficients of this function.
In order to illustrate the introduced ideas a decision making process is
simulated for three objective median location problem.
The concepts, models, and ideas collected and analyzed in this thesis create
a good and relevant grounds for developing more complicated and integrated
models of postoptimal analysis and solving the most computationally challenging
problems related to it.Siirretty Doriast
Direct Policy Optimization using Deterministic Sampling and Collocation
We present an approach for approximately solving discrete-time stochastic
optimal-control problems by combining direct trajectory optimization,
deterministic sampling, and policy optimization. Our feedback motion-planning
algorithm uses a quasi-Newton method to simultaneously optimize a reference
trajectory, a set of deterministically chosen sample trajectories, and a
parameterized policy. We demonstrate that this approach exactly recovers LQR
policies in the case of linear dynamics, quadratic objective, and Gaussian
disturbances. We also demonstrate the algorithm on several nonlinear,
underactuated robotic systems to highlight its performance and ability to
handle control limits, safely avoid obstacles, and generate robust plans in the
presence of unmodeled dynamics.Comment: revisions for RA-L 202
Direct and inverse elastic scattering problems for diffraction gratings
This paper is concerned with the direct and inverse scattering of time-harmonic plane elastic
waves by unbounded periodic structures (diffraction gratings). We present a variational approach
to the forward scattering problems with Lipschitz grating profiles and give a survey of recent
uniqueness and existence results. We also report on recent global uniqueness results within the
class of piecewise linear grating profiles for the corresponding inverse elastic scattering problems. Moreover, a discrete Galerkin method is presented to efficiently approximate solutions of direct
scattering problems via an integral equation approach. Finally, an optimization method for solving
the inverse problem of recovering a 2D periodic structure from scattered elastic waves measured
above the structure is discussed
Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality Differences
In this chapter, an efficient computation approach is proposed for solving a general class of discrete-time optimal control problems. In our approach, a simplified optimal control model, which is adding the adjusted parameters into the model used, is solved iteratively. In this way, the differences between the real plant and the model used are calculated, in turn, to update the optimal solution of the model used. During the computation procedure, the equivalent optimization problem is formulated, where the conjugate gradient algorithm is applied in solving the optimization problem. On this basis, the optimal solution of the modified model-based optimal control problem is obtained repeatedly. Once the convergence is achieved, the iterative solution approximates to the correct optimal solution of the original optimal control problem, in spite of model-reality differences. For illustration, both linear and nonlinear examples are demonstrated to show the performance of the approach proposed. In conclusion, the efficiency of the approach proposed is highly presented
Scaling Up Exact Neural Network Compression by ReLU Stability
We can compress a rectifier network while exactly preserving its underlying functionality with respect to a given input domain if some of its neurons are stable. However, current approaches to determine the stability of neurons with Rectified Linear Unit (ReLU) activations require solving or finding a good approximation to multiple discrete optimization problems. In this work, we introduce an algorithm based on solving a single optimization problem to identify all stable neurons. Our approach is on median 183 times faster than the state-of-art method on CIFAR-10, which allows us to explore exact compression on deeper (5 x 100) and wider (2 x 800) networks within minutes. For classifiers trained under an amount of L1 regularization that does not worsen accuracy, we can remove up to 56% of the connections on the CIFAR-10 dataset. The code is available at the following link, https://github.com/yuxwind/ExactCompression
- …