On the exponential convergence of the Kaczmarz algorithm

The Kaczmarz algorithm (KA) is a popular method for solving a system of linear equations. In this note we derive a new exponential convergence result for the KA. The key allowing us to establish the new result is to rewrite the KA in such a way that its solution path can be interpreted as the output from a particular dynamical system. The asymptotic stability results of the corresponding dynamical system can then be leveraged to prove exponential convergence of the KA. The new bound is also compared to existing bounds

Decentralized Convex Optimization for Wireless Sensor Networks

Many real-world applications arising in domains such as large-scale machine learning, wired and wireless networks can be formulated as distributed linear least-squares over a large network. These problems often have their data naturally distributed. For instance applications such as seismic imaging, smart grid have the sensors geographically distributed and the current algorithms to analyze these data rely on centralized approach. The data is either gathered manually, or relayed by expensive broadband stations, and then processed at a base station. This approach is time-consuming (weeks to months) and hazardous as the task involves manual data gathering in extreme conditions. To obtain the solution in real-time, we require decentralized algorithms that do not rely on a fusion center, cluster heads, or multi-hop communication. In this thesis, we propose several decentralized least squares optimization algorithm that are suitable for performing real-time seismic imaging in a sensor network. The algorithms are evaluated and tested using both synthetic and real-data traces. The results validate that our distributed algorithm is able to obtain a satisfactory image similar to centralized computation under constraints of network resources, while distributing the computational burden to sensor nodes

Accelerating Random Kaczmarz Algorithm Based on Clustering Information

Kaczmarz algorithm is an efficient iterative algorithm to solve overdetermined consistent system of linear equations. During each updating step, Kaczmarz chooses a hyperplane based on an individual equation and projects the current estimate for the exact solution onto that space to get a new estimate. Many vairants of Kaczmarz algorithms are proposed on how to choose better hyperplanes. Using the property of randomly sampled data in high-dimensional space, we propose an accelerated algorithm based on clustering information to improve block Kaczmarz and Kaczmarz via Johnson-Lindenstrauss lemma. Additionally, we theoretically demonstrate convergence improvement on block Kaczmarz algorithm