A novel iterative convex approximation method

Abstract—In this paper, we propose a novel iterative convex approximation algorithm to efficiently compute stationary points for a large class of possibly nonconvex optimization problems. The stationary points are obtained by solving a sequence of successively refined approximate problems, each of which is much easier to solve than the original problem. To achieve convergence, the approximate problem only needs to exhibit a weak form of convexity, namely, pseudo-convexity. We show that the proposed framework not only includes as special cases a number of existing methods, for example, the gradient method and the Jacobi algorithm, but also leads to new algorithms which enjoy easier implementation and faster convergence speed. We also propose a novel line search method for nondifferentiable optimization problems, which is carried out over a properly constructed differentiable function with the benefit of a simpli-fied implementation as compared to state-of-the-art line search techniques that directly operate on the original nondifferentiable objective function. The advantages of the proposed algorithm are shown, both theoretically and numerically, by several ex-ample applications, namely, MIMO broadcast channel capacity computation and LASSO in sparse signal recovery

A Unified Successive Pseudo-Convex Approximation Framework

In this paper, we propose a successive pseudo-convex approximation algorithm to efficiently compute stationary points for a large class of possibly nonconvex optimization problems. The stationary points are obtained by solving a sequence of successively refined approximate problems, each of which is much easier to solve than the original problem. To achieve convergence, the approximate problem only needs to exhibit a weak form of convexity, namely, pseudo-convexity. We show that the proposed framework not only includes as special cases a number of existing methods, for example, the gradient method and the Jacobi algorithm, but also leads to new algorithms which enjoy easier implementation and faster convergence speed. We also propose a novel line search method for nondifferentiable optimization problems, which is carried out over a properly constructed differentiable function with the benefit of a simplified implementation as compared to state-of-the-art line search techniques that directly operate on the original nondifferentiable objective function. The advantages of the proposed algorithm are shown, both theoretically and numerically, by several example applications, namely, MIMO broadcast channel capacity computation, energy efficiency maximization in massive MIMO systems and LASSO in sparse signal recovery.Comment: submitted to IEEE Transactions on Signal Processing; original title: A Novel Iterative Convex Approximation Metho

Partially distributed outer approximation

This paper presents a novel partially distributed outer approximation algorithm, named PaDOA, for solving a class of structured mixed integer convex programming problems to global optimality. The proposed scheme uses an iterative outer approximation method for coupled mixed integer optimization problems with separable convex objective functions, affine coupling constraints, and compact domain. PaDOA proceeds by alternating between solving large-scale structured mixed-integer linear programming problems and partially decoupled mixed-integer nonlinear programming subproblems that comprise much fewer integer variables. We establish conditions under which PaDOA converges to global minimizers after a finite number of iterations and verify these properties with an application to thermostatically controlled loads and to mixed-integer regression

Multi-view Metric Learning in Vector-valued Kernel Spaces

We consider the problem of metric learning for multi-view data and present a novel method for learning within-view as well as between-view metrics in vector-valued kernel spaces, as a way to capture multi-modal structure of the data. We formulate two convex optimization problems to jointly learn the metric and the classifier or regressor in kernel feature spaces. An iterative three-step multi-view metric learning algorithm is derived from the optimization problems. In order to scale the computation to large training sets, a block-wise Nystr{\"o}m approximation of the multi-view kernel matrix is introduced. We justify our approach theoretically and experimentally, and show its performance on real-world datasets against relevant state-of-the-art methods

Iterative learning control for constrained linear systems

This paper considers iterative learning control for linear systems with convex control input constraints. First, the constrained ILC problem is formulated in a novel successive projection framework. Then, based on this projection method, two algorithms are proposed to solve this constrained ILC problem. The results show that, when perfect tracking is possible, both algorithms can achieve perfect tracking. The two algorithms differ however in that one algorithm needs much less computation than the other. When perfect tracking is not possible, both algorithms can exhibit a form of practical convergence to a "best approximation". The effect of weighting matrices on the performance of the algorithms is also discussed and finally, numerical simulations are given to demonstrate the e®ectiveness of the proposed methods

Spectrum optimization in multi-user multi-carrier systems with iterative convex and nonconvex approximation methods

Several practical multi-user multi-carrier communication systems are characterized by a multi-carrier interference channel system model where the interference is treated as noise. For these systems, spectrum optimization is a promising means to mitigate interference. This however corresponds to a challenging nonconvex optimization problem. Existing iterative convex approximation (ICA) methods consist in solving a series of improving convex approximations and are typically implemented in a per-user iterative approach. However they do not take this typical iterative implementation into account in their design. This paper proposes a novel class of iterative approximation methods that focuses explicitly on the per-user iterative implementation, which allows to relax the problem significantly, dropping joint convexity and even convexity requirements for the approximations. A systematic design framework is proposed to construct instances of this novel class, where several new iterative approximation methods are developed with improved per-user convex and nonconvex approximations that are both tighter and simpler to solve (in closed-form). As a result, these novel methods display a much faster convergence speed and require a significantly lower computational cost. Furthermore, a majority of the proposed methods can tackle the issue of getting stuck in bad locally optimal solutions, and hence improve solution quality compared to existing ICA methods.Comment: 33 pages, 7 figures. This work has been submitted for possible publicatio

Toward Energy Efficient Multiuser IRS-Assisted URLLC Systems: A Novel Rank Relaxation Method

This paper proposes an energy efficient resource allocation design algorithm for an intelligent reflecting surface (IRS)-assisted downlink ultra-reliable low-latency communication (URLLC) network. This setup features a multi-antenna base station (BS) transmitting data traffic to a group of URLLC users with short packet lengths. We maximize the total network's energy efficiency (EE) through the optimization of active beamformers at the BS and passive beamformers (a.k.a. phase shifts) at the IRS. The main non-convex problem is divided into two sub-problems. An alternating optimization (AO) approach is then used to solve the problem. Through the use of the successive convex approximation (SCA) with a novel iterative rank relaxation method, we construct a concave-convex objective function for each sub-problem. The first sub-problem is a fractional program that is solved using the Dinkelbach method and a penalty-based approach. The second sub-problem is then solved based on semi-definite programming (SDP) and the penalty-based approach. The iterative solution gradually approaches the rank-one for both the active beamforming and unit modulus IRS phase-shift sub-problems. Our results demonstrate the efficacy of the proposed solution compared to existing benchmarks

Multicast Multigroup Beamforming for Per-antenna Power Constrained Large-scale Arrays

Large in the number of transmit elements, multi-antenna arrays with per-element limitations are in the focus of the present work. In this context, physical layer multigroup multicasting under per-antenna power constrains, is investigated herein. To address this complex optimization problem low-complexity alternatives to semi-definite relaxation are proposed. The goal is to optimize the per-antenna power constrained transmitter in a maximum fairness sense, which is formulated as a non-convex quadratically constrained quadratic problem. Therefore, the recently developed tool of feasible point pursuit and successive convex approximation is extended to account for practical per-antenna power constraints. Interestingly, the novel iterative method exhibits not only superior performance in terms of approaching the relaxed upper bound but also a significant complexity reduction, as the dimensions of the optimization variables increase. Consequently, multicast multigroup beamforming for large-scale array transmitters with per-antenna dedicated amplifiers is rendered computationally efficient and accurate. A preliminary performance evaluation in large-scale systems for which the semi-definite relaxation constantly yields non rank-1 solutions is presented.Comment: submitted to IEEE SPAWC 2015. arXiv admin note: substantial text overlap with arXiv:1406.755

Energy efficiency optimization in MIMO interference channels: A successive pseudoconvex approximation approach

In this paper, we consider the (global and sum) energy efficiency optimization problem in downlink multi-input multi-output multi-cell systems, where all users suffer from multi-user interference. This is a challenging problem due to several reasons: 1) it is a nonconvex fractional programming problem, 2) the transmission rate functions are characterized by (complex-valued) transmit covariance matrices, and 3) the processing-related power consumption may depend on the transmission rate. We tackle this problem by the successive pseudoconvex approximation approach, and we argue that pseudoconvex optimization plays a fundamental role in designing novel iterative algorithms, not only because every locally optimal point of a pseudoconvex optimization problem is also globally optimal, but also because a descent direction is easily obtained from every optimal point of a pseudoconvex optimization problem. The proposed algorithms have the following advantages: 1) fast convergence as the structure of the original optimization problem is preserved as much as possible in the approximate problem solved in each iteration, 2) easy implementation as each approximate problem is suitable for parallel computation and its solution has a closed-form expression, and 3) guaranteed convergence to a stationary point or a Karush-Kuhn-Tucker point. The advantages of the proposed algorithm are also illustrated numerically.Comment: submitted to IEEE Transactions on Signal Processin