Reconciling alternate methods for the determination of charge distributions: A probabilistic approach to high-dimensional least-squares approximations

We propose extensions and improvements of the statistical analysis of distributed multipoles (SADM) algorithm put forth by Chipot et al. in [6] for the derivation of distributed atomic multipoles from the quantum-mechanical electrostatic potential. The method is mathematically extended to general least-squares problems and provides an alternative approximation method in cases where the original least-squares problem is computationally not tractable, either because of its ill-posedness or its high-dimensionality. The solution is approximated employing a Monte Carlo method that takes the average of a random variable defined as the solutions of random small least-squares problems drawn as subsystems of the original problem. The conditions that ensure convergence and consistency of the method are discussed, along with an analysis of the computational cost in specific instances

An Arrow-Hurwicz-Uzawa type flow as least squares solver for network linear equations

We study the approach to obtaining least squares solutions to systems of linear algebraic equations over networks by using distributed algorithms. Each node has access to one of the linear equations and holds a dynamic state. The aim for the node states is to reach a consensus as a least squares solution of the linear equations by exchanging their states with neighbors over an underlying interaction graph. A continuous-time distributed least squares solver over networks is developed in the form of the famous Arrow–Hurwicz–Uzawa flow. A necessary and sufficient condition is established on the graph Laplacian for the continuous-time distributed algorithm to give the least squares solution in the limit, with an exponentially fast convergence rate. The feasibility of different fundamental graphs is discussed including path graph and random graph. Moreover, a discrete-time distributed algorithm is developed by Euler’s method, converging exponentially to the least squares solution at the node states with suitable step size and graph conditions.This work was supported by the DAAD with funds of the German Federal Ministry of Education and Research (BMBF), by the Australian Research Council (ARC) under grants DP-130103610 and DP-160104500

A stochastic mirror-descent algorithm for solving AXB=CAXB=C over an multi-agent system

summary:In this paper, we consider a distributed stochastic computation of $AXB=C$ with local set constraints over an multi-agent system, where each agent over the network only knows a few rows or columns of matrixes. Through formulating an equivalent distributed optimization problem for seeking least-squares solutions of $AXB=C$, we propose a distributed stochastic mirror-descent algorithm for solving the equivalent distributed problem. Then, we provide the sublinear convergence of the proposed algorithm. Moreover, a numerical example is also given to illustrate the effectiveness of the proposed algorithm

Distributed estimation from relative measurements of heterogeneous and uncertain quality

This paper studies the problem of estimation from relative measurements in a graph, in which a vector indexed over the nodes has to be reconstructed from pairwise measurements of differences between its components associated to nodes connected by an edge. In order to model heterogeneity and uncertainty of the measurements, we assume them to be affected by additive noise distributed according to a Gaussian mixture. In this original setup, we formulate the problem of computing the Maximum-Likelihood (ML) estimates and we design two novel algorithms, based on Least Squares regression and Expectation-Maximization (EM). The first algorithm (LS- EM) is centralized and performs the estimation from relative measurements, the soft classification of the measurements, and the estimation of the noise parameters. The second algorithm (Distributed LS-EM) is distributed and performs estimation and soft classification of the measurements, but requires the knowledge of the noise parameters. We provide rigorous proofs of convergence of both algorithms and we present numerical experiments to evaluate and compare their performance with classical solutions. The experiments show the robustness of the proposed methods against different kinds of noise and, for the Distributed LS-EM, against errors in the knowledge of noise parameters.Comment: Submitted to IEEE transaction

RSSI-Based Self-Localization with Perturbed Anchor Positions

We consider the problem of self-localization by a resource-constrained mobile node given perturbed anchor position information and distance estimates from the anchor nodes. We consider normally-distributed noise in anchor position information. The distance estimates are based on the log-normal shadowing path-loss model for the RSSI measurements. The available solutions to this problem are based on complex and iterative optimization techniques such as semidefinite programming or second-order cone programming, which are not suitable for resource-constrained environments. In this paper, we propose a closed-form weighted least-squares solution. We calculate the weights by taking into account the statistical properties of the perturbations in both RSSI and anchor position information. We also estimate the bias of the proposed solution and subtract it from the proposed solution. We evaluate the performance of the proposed algorithm considering a set of arbitrary network topologies in comparison to an existing algorithm that is based on a similar approach but only accounts for perturbations in the RSSI measurements. We also compare the results with the corresponding Cramer-Rao lower bound. Our experimental evaluation shows that the proposed algorithm can substantially improve the localization performance in terms of both root mean square error and bias.Comment: Accepted for publication in 28th Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (IEEE PIMRC 2017

Novel Monte Carlo Methods for Large-Scale Linear Algebra Operations

Linear algebra operations play an important role in scientific computing and data analysis. With increasing data volume and complexity in the Big Data era, linear algebra operations are important tools to process massive datasets. On one hand, the advent of modern high-performance computing architectures with increasing computing power has greatly enhanced our capability to deal with a large volume of data. One the other hand, many classical, deterministic numerical linear algebra algorithms have difficulty to scale to handle large data sets. Monte Carlo methods, which are based on statistical sampling, exhibit many attractive properties in dealing with large volume of datasets, including fast approximated results, memory efficiency, reduced data accesses, natural parallelism, and inherent fault tolerance. In this dissertation, we present new Monte Carlo methods to accommodate a set of fundamental and ubiquitous large-scale linear algebra operations, including solving large-scale linear systems, constructing low-rank matrix approximation, and approximating the extreme eigenvalues/ eigenvectors, across modern distributed and parallel computing architectures. First of all, we revisit the classical Ulam-von Neumann Monte Carlo algorithm and derive the necessary and sufficient condition for its convergence. To support a broad family of linear systems, we develop Krylov subspace Monte Carlo solvers that go beyond the use of Neumann series. New algorithms used in the Krylov subspace Monte Carlo solvers include (1) a Breakdown-Free Block Conjugate Gradient algorithm to address the potential rank deficiency problem occurred in block Krylov subspace methods; (2) a Block Conjugate Gradient for Least Squares algorithm to stably approximate the least squares solutions of general linear systems; (3) a BCGLS algorithm with deflation to gain convergence acceleration; and (4) a Monte Carlo Generalized Minimal Residual algorithm based on sampling matrix-vector products to provide fast approximation of solutions. Secondly, we design a rank-revealing randomized Singular Value Decomposition (R3SVD) algorithm for adaptively constructing low-rank matrix approximations to satisfy application-specific accuracy. Thirdly, we study the block power method on Markov Chain Monte Carlo transition matrices and find that the convergence is actually depending on the number of independent vectors in the block. Correspondingly, we develop a sliding window power method to find stationary distribution, which has demonstrated success in modeling stochastic luminal Calcium release site. Fourthly, we take advantage of hybrid CPU-GPU computing platforms to accelerate the performance of the Breakdown-Free Block Conjugate Gradient algorithm and the randomized Singular Value Decomposition algorithm. Finally, we design a Gaussian variant of Freivalds’ algorithm to efficiently verify the correctness of matrix-matrix multiplication while avoiding undetectable fault patterns encountered in deterministic algorithms