CoCoA: A General Framework for Communication-Efficient Distributed Optimization

The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate with an extensive set of experiments on real distributed datasets

Parallel unstructured solvers for linear partial differential equations

This thesis presents the development of a parallel algorithm to solve symmetric systems of linear equations and the computational implementation of a parallel partial differential equations solver for unstructured meshes. The proposed method, called distributive conjugate gradient - DCG, is based on a single-level domain decomposition method and the conjugate gradient method to obtain a highly scalable parallel algorithm. An overview on methods for the discretization of domains and partial differential equations is given. The partition and refinement of meshes is discussed and the formulation of the weighted residual method for two- and three-dimensions presented. Some of the methods to solve systems of linear equations are introduced, highlighting the conjugate gradient method and domain decomposition methods. A parallel unstructured PDE solver is proposed and its actual implementation presented. Emphasis is given to the data partition adopted and the scheme used for communication among adjacent subdomains is explained. A series of experiments in processor scalability is also reported. The derivation and parallelization of DCG are presented and the method validated throughout numerical experiments. The method capabilities and limitations were investigated by the solution of the Poisson equation with various source terms. The experimental results obtained using the parallel solver developed as part of this work show that the algorithm presented is accurate and highly scalable, achieving roughly linear parallel speed-up in many of the cases tested

The Investigation of Efficiency of Physical Phenomena Modelling Using Differential Equations on Distributed Systems

This work is dedicated to development of mathematical modelling software. In this dissertation numerical methods and algorithms are investigated in software making context. While applying a numerical method it is important to take into account the limited computer resources, the architecture of these resources and how do methods affect software robustness. Three main aspects of this investigation are that software implementation must be efficient, robust and be able to utilize specific hardware resources. The hardware specificity in this work is related to distributed computations of different types: single CPU with multiple cores, multiple CPUs with multiple cores and highly parallel multithreaded GPU device. The investigation is done in three directions: GPU usage for 3D FDTD calculations, FVM method usage to implement efficient calculations of a very specific heat transferring problem, and development of special techniques for software for specific bacteria self organization problem when the results are sensitive to numerical methods, initial data and even computer round-off errors. All these directions are dedicated to create correct technological components that make a software implementation robust and efficient. The time prediction model for 3D FDTD calculations is proposed, which lets to evaluate the efficiency of different GPUs. A reasonable speedup with GPU comparing to CPU is obtained. For FVM implementation the OpenFOAM open source software is selected as a basis for implementation of calculations and a few algorithms and their modifications to solve efficiency issues are proposed. The FVM parallel solver is implemented and analyzed, it is adapted to heterogeneous cluster Vilkas. To create robust software for simulation of bacteria self organization mathematically robust methods are applied and results are analyzed, the algorithm is modified for parallel computations

Coupling different discretizations for fluid structure interaction in a monolithic approach

In this thesis we present a monolithic coupling approach for the simulation of phenomena involving interacting fluid and structure using different discretizations for the subproblems. For many applications in fluid dynamics, the Finite Volume method is the first choice in simulation science. Likewise, for the simulation of structural mechanics the Finite Element method is one of the most, if not the most, popular discretization method. However, despite the advantages of these discretizations in their respective application domains, monolithic coupling schemes have so far been restricted to a single discretization for both subproblems. We present a fluid structure coupling scheme based on a mixed Finite Volume/Finite Element method that combines the benefits of these discretizations. An important challenge in coupling fluid and structure is the transfer of forces and velocities at the fluidstructure interface in a stable and efficient way. In our approach this is achieved by means of a fully implicit formulation, i.e., the transfer of forces and displacements is carried out in a common set of equations for fluid and structure. We assemble the two different discretizations for the fluid and structure subproblems as well as the coupling conditions for forces and displacements into a single large algebraic system. Since we simulate real world problems, as a consequence of the complexity of the considered geometries, we end up with algebraic systems with a large number of degrees of freedom. This necessitates the use of parallel solution techniques. Our work covers the design and implementation of the proposed heterogeneous monolithic coupling approach as well as the efficient solution of the arising large nonlinear systems on distributed memory supercomputers. We apply Newton’s method to linearize the fully implicit coupled nonlinear fluid structure interaction problem. The resulting linear system is solved with a Krylov subspace correction method. For the preconditioning of the iterative solver we propose the use of multilevel methods. Specifically, we study a multigrid as well as a two-level restricted additive Schwarz method. We illustrate the performance of our method on a benchmark example and compare the afore mentioned different preconditioning strategies for the parallel solution of the monolithic coupled system