Solving mixed sparse-dense linear least-squares problems by preconditioned iterative methods

The efficient solution of large linear least-squares problems in which the system matrix A contains rows with very different densities is challenging. Previous work has focused on direct methods for problems in which A has a few relatively dense rows. These rows are initially ignored, a factorization of the sparse part is computed using a sparse direct solver, and then the solution is updated to take account of the omitted dense rows. In some practical applications the number of dense rows can be significant and for very large problems, using a direct solver may not be feasible. We propose processing rows that are identified as dense separately within a conjugate gradient method using an incomplete factorization preconditioner combined with the factorization of a dense matrix of size equal to the number of dense rows. Numerical experiments on large-scale problems from real applications are used to illustrate the effectiveness of our approach. The results demonstrate that we can efficiently solve problems that could not be solved by a preconditioned conjugate gradient method without exploiting the dense rows

Sparse stretching for solving sparse-dense linear least-squares problems

Large-scale linear least-squares problems arise in a wide range of practical applications. In some cases, the system matrix contains a small number of dense rows. These make the problem significantly harder to solve because their presence limits the direct applicability of sparse matrix techniques. In particular, the normal matrix is (close to) dense, so that forming it is impractical. One way to help overcome the dense row problem is to employ matrix stretching. Stretching is a sparse matrix technique that improves sparsity by making the least-squares problem larger. We show that standard stretching can still result in the normal matrix for the stretched problem having an unacceptably large amount of fill. This motivates us to propose a new sparse stretching strategy that performs the stretching so as to limit the fill in the normal matrix and its Cholesky factor. Numerical examples from real problems are used to illustrate the potential gains

A computational study of using black-box QR solvers for large-scale sparse-dense linear least squares problems

Large-scale overdetermined linear least squares problems arise in many practical applications. One popular solution method is based on the backward stable QR factorization of the system matrix A . This article focuses on sparse-dense least squares problems in which A is sparse except from a small number of rows that are considered dense. For large-scale problems, the direct application of a QR solver either fails because of insufficient memory or is unacceptably slow. We study several solution approaches based on using a sparse QR solver without modification, focussing on the case that the sparse part of A is rank deficient. We discuss partial matrix stretching and regularization and propose extending the augmented system formulation with iterative refinement for sparse problems to sparse-dense problems, optionally incorporating multi-precision arithmetic. In summary, our computational study shows that, before applying a black-box QR factorization, a check should be made for rows that are classified as dense and, if such rows are identified, then A should be split into sparse and dense blocks; a number of ways to use a black-box QR factorization to exploit this splitting are possible, with no single method found to be the best in all cases

Strengths and limitations of stretching for least-squares problems with some dense rows

We recently introduced a sparse stretching strategy for handling dense rows that can arise in large-scale linear least-squares problems and make such problems challenging to solve. Sparse stretching is designed to limit the amount of fill within the stretched normal matrix and hence within the subsequent Cholesky factorization. While preliminary results demonstrated that sparse stretching performs significantly better than standard stretching, it has a number of limitations. In this paper, we discuss and illustrate these limitations and propose new strategies that are designed to overcome them. Numerical experiments on problems arising from practical applications are used to demonstrate the effectiveness of these new ideas. We consider both direct and preconditioned iterative solvers

Застосування методів на графах для зведення матриць до блочно-діагональної форми при схемотехнічному моделюванні

Робота виконана на 115 сторінках, містить 33 ілюстрації, 23 таблиці. При підготовці використовувалася література з 35 джерел. Актуальність. У наш час виникає досить глибока проблема паралельного вирішення СЛАР, бо це займає багато часу, тому важливо знайти швидкий та дієвий спосіб спростити цю задачу. На мою думку, слід почати з обернення матриці до блочно-діагональної форми. Це допоможе пришвидшити зведення та мінімізує складність часу. Мета. Знайти дієвий і найправильніший спосіб змінити таким чином симетричну матрицю щоб вона стала блочно-діагональною з обрамленням. Завдання. Для досягнення поставленої мети необхідно розв’язати наступні завдання: проаналізувати існуючі типи матриць для того, щоб обрати найбільш підходящу для вхідних даних; розібрати методи зведення матриць до блочно-діагональної форми; проаналізувати обрані методи на графах; розробити програмний продукт як приклад зведення; розробити стартап, який допоможе прорахувати усі доцільні витрати, та зробити продукт спроможним змагатися з іншими на рівні. Об’єкт дослідження. Симетричні та блочно-діагональні матриці, гіперграфи. Предмет дослідження. Взаємодія з матрицями та гіперграфами. Наукова новизна. Наукова новизна роботи полягає в дослідженні методів на графах для зведення матриць до блочно-діагональної та пошук способів мінімізації часу зведення. Практична цінність. Практична цінність роботи полягає у подальшому її використанні для паралельного вирішення СЛАР.The work is done on 115 pages, contains 33 illustrations, 23 tables. In preparation, literature from 35 resources was used. Topicality. Nowadays, a rather deep problem of parallel solution of SLAEs arises, since it takes a lot of time, so it is important to find quick and effective way to simplify this task. In my opinion, one should start by converting the matrix to a block-diagonal form. This will help speed up casting and minimize time complexity. Purpose. Find an efficient and correct way to change a symmetric matrix in such a way that it becomes block-diagonal with a border. Task. To achieve this goal, it is necessary to solve the following tasks: analyze the existing types of matrices in order to choose the most suitable for the input parameters; analyze the methods of reducing matrices to block-diagonal form; analyze the selected methods on graphs; develop a software product as an example of casting; develop a startup project that will help calculate all the necessary costs and make the product able to compete with its environment at the level. Object of research. Symmetric and block-diagonal matrices, hypergraphs. Subject of research. Interaction with matrices and hypergraphs. Scientific novelty. The scientific novelty of the work lies in the study of methods on graphs for reducing matrices to a block-diagonal form and the search for ways to minimize the reduction time. Practical value of research. The practical value of the work lies in its further use for the parallel solution of the SLAE