A literature survey of matrix methods for data science

Efficient numerical linear algebra is a core ingredient in many applications across almost all scientific and industrial disciplines. With this survey we want to illustrate that numerical linear algebra has played and is playing a crucial role in enabling and improving data science computations with many new developments being fueled by the availability of data and computing resources. We highlight the role of various different factorizations and the power of changing the representation of the data as well as discussing topics such as randomized algorithms, functions of matrices, and high-dimensional problems. We briefly touch upon the role of techniques from numerical linear algebra used within deep learning

On algorithmization of Janashia-Lagvilava matrix spectral factorization method

We consider three different ways of algorithmization of the Janashia-Lagvilava spectral factorization method. The first algorithm is faster than the second one, however, it is only suitable for matrices of low dimension. The second algorithm, on the other hand, can be applied to matrices of substantially larger dimension. The third algorithm is a superfast implementation of the method, but only works in the polynomial case under the additional restriction that the zeros of the determinant are not too close to the boundary. All three algorithms fully utilize the advantage of the method which carries out spectral factorization of leading principal submatrices step-by-step. The corresponding results of numerical simulations are reported in order to describe the characteristic features of each algorithm and compare them to other existing algorithms.Comment: 18 page

A Survey on Multi-View Clustering

With advances in information acquisition technologies, multi-view data become ubiquitous. Multi-view learning has thus become more and more popular in machine learning and data mining fields. Multi-view unsupervised or semi-supervised learning, such as co-training, co-regularization has gained considerable attention. Although recently, multi-view clustering (MVC) methods have been developed rapidly, there has not been a survey to summarize and analyze the current progress. Therefore, this paper reviews the common strategies for combining multiple views of data and based on this summary we propose a novel taxonomy of the MVC approaches. We further discuss the relationships between MVC and multi-view representation, ensemble clustering, multi-task clustering, multi-view supervised and semi-supervised learning. Several representative real-world applications are elaborated. To promote future development of MVC, we envision several open problems that may require further investigation and thorough examination.Comment: 17 pages, 4 figure

On the estimation of the convergence rate in the Janashia-Lagvilava spectral factorization algorithm

In the present paper, we estimate the convergence rate in the Janashia-Lagvilava spectral factorization algorithm (see Studia Mathematica, 137, 1999, 93-100) under the restriction on a spectral density matrix that its inverse is integrable.Comment: 9 page

A New Method of Matrix Spectral Factorization

A new method of matrix spectral factorization is proposed which reliably computes an approximate spectral factor of any matrix spectral density that admits spectral factorizationComment: 23 page

Unmixing of Hyperspectral Data Using Robust Statistics-based NMF

Mixed pixels are presented in hyperspectral images due to low spatial resolution of hyperspectral sensors. Spectral unmixing decomposes mixed pixels spectra into endmembers spectra and abundance fractions. In this paper using of robust statistics-based nonnegative matrix factorization (RNMF) for spectral unmixing of hyperspectral data is investigated. RNMF uses a robust cost function and iterative updating procedure, so is not sensitive to outliers. This method has been applied to simulated data using USGS spectral library, AVIRIS and ROSIS datasets. Unmixing results are compared to traditional NMF method based on SAD and AAD measures. Results demonstrate that this method can be used efficiently for hyperspectral unmixing purposes.Comment: 4 pages, conferenc

Machine Learning on Graphs: A Model and Comprehensive Taxonomy

There has been a surge of recent interest in learning representations for graph-structured data. Graph representation learning methods have generally fallen into three main categories, based on the availability of labeled data. The first, network embedding (such as shallow graph embedding or graph auto-encoders), focuses on learning unsupervised representations of relational structure. The second, graph regularized neural networks, leverages graphs to augment neural network losses with a regularization objective for semi-supervised learning. The third, graph neural networks, aims to learn differentiable functions over discrete topologies with arbitrary structure. However, despite the popularity of these areas there has been surprisingly little work on unifying the three paradigms. Here, we aim to bridge the gap between graph neural networks, network embedding and graph regularization models. We propose a comprehensive taxonomy of representation learning methods for graph-structured data, aiming to unify several disparate bodies of work. Specifically, we propose a Graph Encoder Decoder Model (GRAPHEDM), which generalizes popular algorithms for semi-supervised learning on graphs (e.g. GraphSage, Graph Convolutional Networks, Graph Attention Networks), and unsupervised learning of graph representations (e.g. DeepWalk, node2vec, etc) into a single consistent approach. To illustrate the generality of this approach, we fit over thirty existing methods into this framework. We believe that this unifying view both provides a solid foundation for understanding the intuition behind these methods, and enables future research in the area

The Why and How of Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) has become a widely used tool for the analysis of high-dimensional data as it automatically extracts sparse and meaningful features from a set of nonnegative data vectors. We first illustrate this property of NMF on three applications, in image processing, text mining and hyperspectral imaging --this is the why. Then we address the problem of solving NMF, which is NP-hard in general. We review some standard NMF algorithms, and also present a recent subclass of NMF problems, referred to as near-separable NMF, that can be solved efficiently (that is, in polynomial time), even in the presence of noise --this is the how. Finally, we briefly describe some problems in mathematics and computer science closely related to NMF via the nonnegative rank.Comment: 25 pages, 5 figures. Some typos and errors corrected, Section 3.2 reorganize

Introduction to Nonnegative Matrix Factorization

In this paper, we introduce and provide a short overview of nonnegative matrix factorization (NMF). Several aspects of NMF are discussed, namely, the application in hyperspectral imaging, geometry and uniqueness of NMF solutions, complexity, algorithms, and its link with extended formulations of polyhedra. In order to put NMF into perspective, the more general problem class of constrained low-rank matrix approximation problems is first briefly introduced.Comment: 18 pages, 4 figure

An Analytic Proof of the Matrix Spectral Factorization Theorem

An analytic proof is proposed of Wiener's theorem on factorization of positive definite matrix-functions.Comment: 11 page