54,841 research outputs found
A Computational Framework for Multivariate Convex Regression and Its Variants
We study the nonparametric least squares estimator (LSE) of a multivariate convex regression function. The LSE, given as the solution to a quadratic program with O(n²) linear constraints (n being the sample size), is difficult to compute for large problems. Exploiting problem specific structure, we propose a scalable algorithmic framework based on the augmented Lagrangian method to compute the LSE. We develop a novel approach to obtain smooth convex approximations to the fitted (piecewise affine) convex LSE and provide formal bounds on the quality of approximation. When the number of samples is not too large compared to the dimension of the predictor, we propose a regularization scheme—Lipschitz convex regression—where we constrain the norm of the subgradients, and study the rates of convergence of the obtained LSE. Our algorithmic framework is simple and flexible and can be easily adapted to handle variants: estimation of a nondecreasing/nonincreasing convex/concave (with or without a Lipschitz bound) function. We perform numerical studies illustrating the scalability of the proposed algorithm—on some instances our proposal leads to more than a 10,000-fold improvement in runtime when compared to off-the-shelf interior point solvers for problems with n = 500. Keywords: Augmented Lagrangian method; Lipschitz convex regression; Non parametric least squares estimator; Scalable quadratic programming; Smooth convex regressionUnited States. Office of Naval Research (Grant N00014-15-1-2342
Using a conic bundle method to accelerate both phases of a quadratic convex reformulation
We present algorithm MIQCR-CB that is an advancement of method
MIQCR~(Billionnet, Elloumi and Lambert, 2012). MIQCR is a method for solving
mixed-integer quadratic programs and works in two phases: the first phase
determines an equivalent quadratic formulation with a convex objective function
by solving a semidefinite problem , and, in the second phase, the
equivalent formulation is solved by a standard solver. As the reformulation
relies on the solution of a large-scale semidefinite program, it is not
tractable by existing semidefinite solvers, already for medium sized problems.
To surmount this difficulty, we present in MIQCR-CB a subgradient algorithm
within a Lagrangian duality framework for solving that substantially
speeds up the first phase. Moreover, this algorithm leads to a reformulated
problem of smaller size than the one obtained by the original MIQCR method
which results in a shorter time for solving the second phase.
We present extensive computational results to show the efficiency of our
algorithm
Large-scale Binary Quadratic Optimization Using Semidefinite Relaxation and Applications
In computer vision, many problems such as image segmentation, pixel
labelling, and scene parsing can be formulated as binary quadratic programs
(BQPs). For submodular problems, cuts based methods can be employed to
efficiently solve large-scale problems. However, general nonsubmodular problems
are significantly more challenging to solve. Finding a solution when the
problem is of large size to be of practical interest, however, typically
requires relaxation. Two standard relaxation methods are widely used for
solving general BQPs--spectral methods and semidefinite programming (SDP), each
with their own advantages and disadvantages. Spectral relaxation is simple and
easy to implement, but its bound is loose. Semidefinite relaxation has a
tighter bound, but its computational complexity is high, especially for large
scale problems. In this work, we present a new SDP formulation for BQPs, with
two desirable properties. First, it has a similar relaxation bound to
conventional SDP formulations. Second, compared with conventional SDP methods,
the new SDP formulation leads to a significantly more efficient and scalable
dual optimization approach, which has the same degree of complexity as spectral
methods. We then propose two solvers, namely, quasi-Newton and smoothing Newton
methods, for the dual problem. Both of them are significantly more efficiently
than standard interior-point methods. In practice, the smoothing Newton solver
is faster than the quasi-Newton solver for dense or medium-sized problems,
while the quasi-Newton solver is preferable for large sparse/structured
problems. Our experiments on a few computer vision applications including
clustering, image segmentation, co-segmentation and registration show the
potential of our SDP formulation for solving large-scale BQPs.Comment: Fixed some typos. 18 pages. Accepted to IEEE Transactions on Pattern
Analysis and Machine Intelligenc
Portfolio selection problems in practice: a comparison between linear and quadratic optimization models
Several portfolio selection models take into account practical limitations on
the number of assets to include and on their weights in the portfolio. We
present here a study of the Limited Asset Markowitz (LAM), of the Limited Asset
Mean Absolute Deviation (LAMAD) and of the Limited Asset Conditional
Value-at-Risk (LACVaR) models, where the assets are limited with the
introduction of quantity and cardinality constraints. We propose a completely
new approach for solving the LAM model, based on reformulation as a Standard
Quadratic Program and on some recent theoretical results. With this approach we
obtain optimal solutions both for some well-known financial data sets used by
several other authors, and for some unsolved large size portfolio problems. We
also test our method on five new data sets involving real-world capital market
indices from major stock markets. Our computational experience shows that,
rather unexpectedly, it is easier to solve the quadratic LAM model with our
algorithm, than to solve the linear LACVaR and LAMAD models with CPLEX, one of
the best commercial codes for mixed integer linear programming (MILP) problems.
Finally, on the new data sets we have also compared, using out-of-sample
analysis, the performance of the portfolios obtained by the Limited Asset
models with the performance provided by the unconstrained models and with that
of the official capital market indices
- …