Algorithms for the continuous nonlinear resource allocation problem---new implementations and numerical studies

Patriksson (2008) provided a then up-to-date survey on the continuous,separable, differentiable and convex resource allocation problem with a single resource constraint. Since the publication of that paper the interest in the problem has grown: several new applications have arisen where the problem at hand constitutes a subproblem, and several new algorithms have been developed for its efficient solution. This paper therefore serves three purposes. First, it provides an up-to-date extension of the survey of the literature of the field, complementing the survey in Patriksson (2008) with more then 20 books and articles. Second, it contributes improvements of some of these algorithms, in particular with an improvement of the pegging (that is, variable fixing) process in the relaxation algorithm, and an improved means to evaluate subsolutions. Third, it numerically evaluates several relaxation (primal) and breakpoint (dual) algorithms, incorporating a variety of pegging strategies, as well as a quasi-Newton method. Our conclusion is that our modification of the relaxation algorithm performs the best. At least for problem sizes up to 30 million variables the practical time complexity for the breakpoint and relaxation algorithms is linear

Portfolio selection problems in practice: a comparison between linear and quadratic optimization models

Several portfolio selection models take into account practical limitations on the number of assets to include and on their weights in the portfolio. We present here a study of the Limited Asset Markowitz (LAM), of the Limited Asset Mean Absolute Deviation (LAMAD) and of the Limited Asset Conditional Value-at-Risk (LACVaR) models, where the assets are limited with the introduction of quantity and cardinality constraints. We propose a completely new approach for solving the LAM model, based on reformulation as a Standard Quadratic Program and on some recent theoretical results. With this approach we obtain optimal solutions both for some well-known financial data sets used by several other authors, and for some unsolved large size portfolio problems. We also test our method on five new data sets involving real-world capital market indices from major stock markets. Our computational experience shows that, rather unexpectedly, it is easier to solve the quadratic LAM model with our algorithm, than to solve the linear LACVaR and LAMAD models with CPLEX, one of the best commercial codes for mixed integer linear programming (MILP) problems. Finally, on the new data sets we have also compared, using out-of-sample analysis, the performance of the portfolios obtained by the Limited Asset models with the performance provided by the unconstrained models and with that of the official capital market indices

Nonlinear Integer Programming

Research efforts of the past fifty years have led to a development of linear integer programming as a mature discipline of mathematical optimization. Such a level of maturity has not been reached when one considers nonlinear systems subject to integrality requirements for the variables. This chapter is dedicated to this topic. The primary goal is a study of a simple version of general nonlinear integer problems, where all constraints are still linear. Our focus is on the computational complexity of the problem, which varies significantly with the type of nonlinear objective function in combination with the underlying combinatorial structure. Numerous boundary cases of complexity emerge, which sometimes surprisingly lead even to polynomial time algorithms. We also cover recent successful approaches for more general classes of problems. Though no positive theoretical efficiency results are available, nor are they likely to ever be available, these seem to be the currently most successful and interesting approaches for solving practical problems. It is our belief that the study of algorithms motivated by theoretical considerations and those motivated by our desire to solve practical instances should and do inform one another. So it is with this viewpoint that we present the subject, and it is in this direction that we hope to spark further research.Comment: 57 pages. To appear in: M. J\"unger, T. Liebling, D. Naddef, G. Nemhauser, W. Pulleyblank, G. Reinelt, G. Rinaldi, and L. Wolsey (eds.), 50 Years of Integer Programming 1958--2008: The Early Years and State-of-the-Art Surveys, Springer-Verlag, 2009, ISBN 354068274

Sum-Rate Maximization in Two-Way AF MIMO Relaying: Polynomial Time Solutions to a Class of DC Programming Problems

Sum-rate maximization in two-way amplify-and-forward (AF) multiple-input multiple-output (MIMO) relaying belongs to the class of difference-of-convex functions (DC) programming problems. DC programming problems occur as well in other signal processing applications and are typically solved using different modifications of the branch-and-bound method. This method, however, does not have any polynomial time complexity guarantees. In this paper, we show that a class of DC programming problems, to which the sum-rate maximization in two-way MIMO relaying belongs, can be solved very efficiently in polynomial time, and develop two algorithms. The objective function of the problem is represented as a product of quadratic ratios and parameterized so that its convex part (versus the concave part) contains only one (or two) optimization variables. One of the algorithms is called POlynomial-Time DC (POTDC) and is based on semi-definite programming (SDP) relaxation, linearization, and an iterative search over a single parameter. The other algorithm is called RAte-maximization via Generalized EigenvectorS (RAGES) and is based on the generalized eigenvectors method and an iterative search over two (or one, in its approximate version) optimization variables. We also derive an upper-bound for the optimal values of the corresponding optimization problem and show by simulations that this upper-bound can be achieved by both algorithms. The proposed methods for maximizing the sum-rate in the two-way AF MIMO relaying system are shown to be superior to other state-of-the-art algorithms.Comment: 35 pages, 10 figures, Submitted to the IEEE Trans. Signal Processing in Nov. 201

An Exponential Lower Bound on the Complexity of Regularization Paths

For a variety of regularized optimization problems in machine learning, algorithms computing the entire solution path have been developed recently. Most of these methods are quadratic programs that are parameterized by a single parameter, as for example the Support Vector Machine (SVM). Solution path algorithms do not only compute the solution for one particular value of the regularization parameter but the entire path of solutions, making the selection of an optimal parameter much easier. It has been assumed that these piecewise linear solution paths have only linear complexity, i.e. linearly many bends. We prove that for the support vector machine this complexity can be exponential in the number of training points in the worst case. More strongly, we construct a single instance of n input points in d dimensions for an SVM such that at least \Theta(2^{n/2}) = \Theta(2^d) many distinct subsets of support vectors occur as the regularization parameter changes.Comment: Journal version, 28 Pages, 5 Figure

A Convex Feasibility Approach to Anytime Model Predictive Control

This paper proposes to decouple performance optimization and enforcement of asymptotic convergence in Model Predictive Control (MPC) so that convergence to a given terminal set is achieved independently of how much performance is optimized at each sampling step. By embedding an explicit decreasing condition in the MPC constraints and thanks to a novel and very easy-to-implement convex feasibility solver proposed in the paper, it is possible to run an outer performance optimization algorithm on top of the feasibility solver and optimize for an amount of time that depends on the available CPU resources within the current sampling step (possibly going open-loop at a given sampling step in the extreme case no resources are available) and still guarantee convergence to the terminal set. While the MPC setup and the solver proposed in the paper can deal with quite general classes of functions, we highlight the synthesis method and show numerical results in case of linear MPC and ellipsoidal and polyhedral terminal sets.Comment: 8 page

Certification of Bounds of Non-linear Functions: the Templates Method

The aim of this work is to certify lower bounds for real-valued multivariate functions, defined by semialgebraic or transcendental expressions. The certificate must be, eventually, formally provable in a proof system such as Coq. The application range for such a tool is widespread; for instance Hales' proof of Kepler's conjecture yields thousands of inequalities. We introduce an approximation algorithm, which combines ideas of the max-plus basis method (in optimal control) and of the linear templates method developed by Manna et al. (in static analysis). This algorithm consists in bounding some of the constituents of the function by suprema of quadratic forms with a well chosen curvature. This leads to semialgebraic optimization problems, solved by sum-of-squares relaxations. Templates limit the blow up of these relaxations at the price of coarsening the approximation. We illustrate the efficiency of our framework with various examples from the literature and discuss the interfacing with Coq.Comment: 16 pages, 3 figures, 2 table