Approximate Nonnegative Matrix Factorization via Alternating Minimization

In this paper we consider the Nonnegative Matrix Factorization (NMF) problem: given an (elementwise) nonnegative matrix $V \in \R_+^{m\times n}$ find, for assigned $k$, nonnegative matrices $W\in\R_+^{m\times k}$ and $H\in\R_+^{k\times n}$ such that $V=WH$. Exact, non trivial, nonnegative factorizations do not always exist, hence it is interesting to pose the approximate NMF problem. The criterion which is commonly employed is I-divergence between nonnegative matrices. The problem becomes that of finding, for assigned $k$, the factorization $WH$ closest to $V$ in I-divergence. An iterative algorithm, EM like, for the construction of the best pair $(W, H)$ has been proposed in the literature. In this paper we interpret the algorithm as an alternating minimization procedure \`a la Csisz\'ar-Tusn\'ady and investigate some of its stability properties. NMF is widespreading as a data analysis method in applications for which the positivity constraint is relevant. There are other data analysis methods which impose some form of nonnegativity: we discuss here the connections between NMF and Archetypal Analysis. An interesting system theoretic application of NMF is to the problem of approximate realization of Hidden Markov Models

Joint Source and Relay Precoding Designs for MIMO Two-Way Relaying Based on MSE Criterion

Properly designed precoders can significantly improve the spectral efficiency of multiple-input multiple-output (MIMO) relay systems. In this paper, we investigate joint source and relay precoding design based on the mean-square-error (MSE) criterion in MIMO two-way relay systems, where two multi-antenna source nodes exchange information via a multi-antenna amplify-and-forward relay node. This problem is non-convex and its optimal solution remains unsolved. Aiming to find an efficient way to solve the problem, we first decouple the primal problem into three tractable sub-problems, and then propose an iterative precoding design algorithm based on alternating optimization. The solution to each sub-problem is optimal and unique, thus the convergence of the iterative algorithm is guaranteed. Secondly, we propose a structured precoding design to lower the computational complexity. The proposed precoding structure is able to parallelize the channels in the multiple access (MAC) phase and broadcast (BC) phase. It thus reduces the precoding design to a simple power allocation problem. Lastly, for the special case where only a single data stream is transmitted from each source node, we present a source-antenna-selection (SAS) based precoding design algorithm. This algorithm selects only one antenna for transmission from each source and thus requires lower signalling overhead. Comprehensive simulation is conducted to evaluate the effectiveness of all the proposed precoding designs.Comment: 32 pages, 10 figure

A fast algorithm for matrix balancing

As long as a square nonnegative matrix A contains sufficient nonzero elements, then the matrix can be balanced, that is we can find a diagonal scaling of A that is doubly stochastic. A number of algorithms have been proposed to achieve the balancing, the most well known of these being Sinkhorn-Knopp. In this paper we derive new algorithms based on inner-outer iteration schemes. We show that Sinkhorn-Knopp belongs to this family, but other members can converge much more quickly. In particular, we show that while stationary iterative methods offer little or no improvement in many cases, a scheme using a preconditioned conjugate gradient method as the inner iteration can give quadratic convergence at low cost

Error estimators and their analysis for CG, Bi-CG and GMRES

We present an analysis of the uncertainty in the convergence of iterative linear solvers when using relative residue as a stopping criterion, and the resulting over/under computation for a given tolerance in error. This shows that error estimation is indispensable for efficient and accurate solution of moderate to high conditioned linear systems ($\kappa>100$), where $\kappa$ is the condition number of the matrix. An $\mathcal{O}(1)$ error estimator for iterations of the CG (Conjugate Gradient) algorithm was proposed more than two decades ago. Recently, an $\mathcal{O}(k^2)$ error estimator was described for the GMRES (Generalized Minimal Residual) algorithm which allows for non-symmetric linear systems as well, where $k$ is the iteration number. We suggest a minor modification in this GMRES error estimation for increased stability. In this work, we also propose an $\mathcal{O}(n)$ error estimator for A-norm and $l_{2}$ norm of the error vector in Bi-CG (Bi-Conjugate Gradient) algorithm. The robust performance of these estimates as a stopping criterion results in increased savings and accuracy in computation, as condition number and size of problems increase