17,622 research outputs found
SOCP relaxation bounds for the optimal subset selection problem applied to robust linear regression
This paper deals with the problem of finding the globally optimal subset of h
elements from a larger set of n elements in d space dimensions so as to
minimize a quadratic criterion, with an special emphasis on applications to
computing the Least Trimmed Squares Estimator (LTSE) for robust regression. The
computation of the LTSE is a challenging subset selection problem involving a
nonlinear program with continuous and binary variables, linked in a highly
nonlinear fashion. The selection of a globally optimal subset using the branch
and bound (BB) algorithm is limited to problems in very low dimension,
tipically d<5, as the complexity of the problem increases exponentially with d.
We introduce a bold pruning strategy in the BB algorithm that results in a
significant reduction in computing time, at the price of a negligeable accuracy
lost. The novelty of our algorithm is that the bounds at nodes of the BB tree
come from pseudo-convexifications derived using a linearization technique with
approximate bounds for the nonlinear terms. The approximate bounds are computed
solving an auxiliary semidefinite optimization problem. We show through a
computational study that our algorithm performs well in a wide set of the most
difficult instances of the LTSE problem.Comment: 12 pages, 3 figures, 2 table
A Unified Successive Pseudo-Convex Approximation Framework
In this paper, we propose a successive pseudo-convex approximation algorithm
to efficiently compute stationary points for a large class of possibly
nonconvex optimization problems. The stationary points are obtained by solving
a sequence of successively refined approximate problems, each of which is much
easier to solve than the original problem. To achieve convergence, the
approximate problem only needs to exhibit a weak form of convexity, namely,
pseudo-convexity. We show that the proposed framework not only includes as
special cases a number of existing methods, for example, the gradient method
and the Jacobi algorithm, but also leads to new algorithms which enjoy easier
implementation and faster convergence speed. We also propose a novel line
search method for nondifferentiable optimization problems, which is carried out
over a properly constructed differentiable function with the benefit of a
simplified implementation as compared to state-of-the-art line search
techniques that directly operate on the original nondifferentiable objective
function. The advantages of the proposed algorithm are shown, both
theoretically and numerically, by several example applications, namely, MIMO
broadcast channel capacity computation, energy efficiency maximization in
massive MIMO systems and LASSO in sparse signal recovery.Comment: submitted to IEEE Transactions on Signal Processing; original title:
A Novel Iterative Convex Approximation Metho
Sum-Rate Maximization in Two-Way AF MIMO Relaying: Polynomial Time Solutions to a Class of DC Programming Problems
Sum-rate maximization in two-way amplify-and-forward (AF) multiple-input
multiple-output (MIMO) relaying belongs to the class of difference-of-convex
functions (DC) programming problems. DC programming problems occur as well in
other signal processing applications and are typically solved using different
modifications of the branch-and-bound method. This method, however, does not
have any polynomial time complexity guarantees. In this paper, we show that a
class of DC programming problems, to which the sum-rate maximization in two-way
MIMO relaying belongs, can be solved very efficiently in polynomial time, and
develop two algorithms. The objective function of the problem is represented as
a product of quadratic ratios and parameterized so that its convex part (versus
the concave part) contains only one (or two) optimization variables. One of the
algorithms is called POlynomial-Time DC (POTDC) and is based on semi-definite
programming (SDP) relaxation, linearization, and an iterative search over a
single parameter. The other algorithm is called RAte-maximization via
Generalized EigenvectorS (RAGES) and is based on the generalized eigenvectors
method and an iterative search over two (or one, in its approximate version)
optimization variables. We also derive an upper-bound for the optimal values of
the corresponding optimization problem and show by simulations that this
upper-bound can be achieved by both algorithms. The proposed methods for
maximizing the sum-rate in the two-way AF MIMO relaying system are shown to be
superior to other state-of-the-art algorithms.Comment: 35 pages, 10 figures, Submitted to the IEEE Trans. Signal Processing
in Nov. 201
Probabilistic Clustering Using Maximal Matrix Norm Couplings
In this paper, we present a local information theoretic approach to
explicitly learn probabilistic clustering of a discrete random variable. Our
formulation yields a convex maximization problem for which it is NP-hard to
find the global optimum. In order to algorithmically solve this optimization
problem, we propose two relaxations that are solved via gradient ascent and
alternating maximization. Experiments on the MSR Sentence Completion Challenge,
MovieLens 100K, and Reuters21578 datasets demonstrate that our approach is
competitive with existing techniques and worthy of further investigation.Comment: Presented at 56th Annual Allerton Conference on Communication,
Control, and Computing, 201
- …