Preconditioned Spectral Clustering for Stochastic Block Partition Streaming Graph Challenge

Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) is demonstrated to efficiently solve eigenvalue problems for graph Laplacians that appear in spectral clustering. For static graph partitioning, 10-20 iterations of LOBPCG without preconditioning result in ~10x error reduction, enough to achieve 100% correctness for all Challenge datasets with known truth partitions, e.g., for graphs with 5K/.1M (50K/1M) Vertices/Edges in 2 (7) seconds, compared to over 5,000 (30,000) seconds needed by the baseline Python code. Our Python code 100% correctly determines 98 (160) clusters from the Challenge static graphs with 0.5M (2M) vertices in 270 (1,700) seconds using 10GB (50GB) of memory. Our single-precision MATLAB code calculates the same clusters at half time and memory. For streaming graph partitioning, LOBPCG is initiated with approximate eigenvectors of the graph Laplacian already computed for the previous graph, in many cases reducing 2-3 times the number of required LOBPCG iterations, compared to the static case. Our spectral clustering is generic, i.e. assuming nothing specific of the block model or streaming, used to generate the graphs for the Challenge, in contrast to the base code. Nevertheless, in 10-stage streaming comparison with the base code for the 5K graph, the quality of our clusters is similar or better starting at stage 4 (7) for emerging edging (snowballing) streaming, while the computations are over 100-1000 faster.Comment: 6 pages. To appear in Proceedings of the 2017 IEEE High Performance Extreme Computing Conference. Student Innovation Award Streaming Graph Challenge: Stochastic Block Partition, see http://graphchallenge.mit.edu/champion

A geometric convergence theory for the preconditioned steepest descent iteration

Preconditioned gradient iterations for very large eigenvalue problems are efficient solvers with growing popularity. However, only for the simplest preconditioned eigensolver, namely the preconditioned gradient iteration (or preconditioned inverse iteration) with fixed step size, sharp non-asymptotic convergence estimates are known and these estimates require an ideally scaled preconditioner. In this paper a new sharp convergence estimate is derived for the preconditioned steepest descent iteration which combines the preconditioned gradient iteration with the Rayleigh-Ritz procedure for optimal line search convergence acceleration. The new estimate always improves that of the fixed step size iteration. The practical importance of this new estimate is that arbitrarily scaled preconditioners can be used. The Rayleigh-Ritz procedure implicitly computes the optimal scaling.Comment: 17 pages, 6 figure

Convergence Analysis of Extended LOBPCG for Computing Extreme Eigenvalues

This paper is concerned with the convergence analysis of an extended variation of the locally optimal preconditioned conjugate gradient method (LOBPCG) for the extreme eigenvalue of a Hermitian matrix polynomial which admits some extended form of Rayleigh quotient. This work is a generalization of the analysis by Ovtchinnikov (SIAM J. Numer. Anal., 46(5):2567-2592, 2008). As instances, the algorithms for definite matrix pairs and hyperbolic quadratic matrix polynomials are shown to be globally convergent and to have an asymptotically local convergence rate. Also, numerical examples are given to illustrate the convergence.Comment: 21 pages, 2 figure

Riemannian Acceleration with Preconditioning for symmetric eigenvalue problems

In this paper, we propose a Riemannian Acceleration with Preconditioning (RAP) for symmetric eigenvalue problems, which is one of the most important geodesically convex optimization problem on Riemannian manifold, and obtain the acceleration. Firstly, the preconditioning for symmetric eigenvalue problems from the Riemannian manifold viewpoint is discussed. In order to obtain the local geodesic convexity, we develop the leading angle to measure the quality of the preconditioner for symmetric eigenvalue problems. A new Riemannian acceleration, called Locally Optimal Riemannian Accelerated Gradient (LORAG) method, is proposed to overcome the local geodesic convexity for symmetric eigenvalue problems. With similar techniques for RAGD and analysis of local convex optimization in Euclidean space, we analyze the convergence of LORAG. Incorporating the local geodesic convexity of symmetric eigenvalue problems under preconditioning with the LORAG, we propose the Riemannian Acceleration with Preconditioning (RAP) and prove its acceleration. Additionally, when the Schwarz preconditioner, especially the overlapping or non-overlapping domain decomposition method, is applied for elliptic eigenvalue problems, we also obtain the rate of convergence as $1-C\kappa^{-1/2}$, where $C$ is a constant independent of the mesh sizes and the eigenvalue gap, $\kappa=\kappa_{\nu}\lambda_{2}/(\lambda_{2}-\lambda_{1})$, $\kappa_{\nu}$ is the parameter from the stable decomposition, $\lambda_{1}$ and $\lambda_{2}$ are the smallest two eigenvalues of the elliptic operator. Numerical results show the power of Riemannian acceleration and preconditioning.Comment: Due to the limit in abstract of arXiv, the abstract here is shorter than in PD

Simultaneous-FETI and Block-FETI: robust domain decomposition with multiple search directions.

International audienceDomain Decomposition methods often exhibit very poor performance when applied to engineering problems with large heterogeneities. In particular for heterogeneities along domain interfaces the iterative techniques to solve the interface problem are lacking an efficient preconditioner. Recently a robust approach, named FETI-Geneo, was proposed where troublesome modes are precomputed and deflated from the interface problem. The cost of the FETI-Geneo is however high. We propose in this paper techniques that share similar ideas with FETI-Geneo but where no pre-processing is needed and that can be easily and efficiently implemented as an alternative to standard Domain Decomposition methods. In the block iterative approaches presented in this paper, the search space at every iteration on the interface problem contains as many directions as there are domains in the decomposition. Those search directions originate either from the domain-wise preconditioner (in the Simultaneous FETI method) or from the block structure of the right-hand side of the interface problem (Block FETI). We show on 2D structural examples that both methods are robust and provide good convergence in the presence of high heterogeneities, even when the interface is jagged or when the domains have a bad aspect ratio. The Simultaneous FETI was also efficiently implemented in an optimized parallel code and exhibited excellent performance compared to the regular FETI method

Physics-based balancing domain decomposition by constraints for multi-material problems

The final publication is available at Springer via http://dx.doi.org/10.1007/s10915-018-0870-zIn this work, we present a new variant of the balancing domain decomposition by constraints preconditioner that is robust for multi-material problems. We start with a well-balanced subdomain partition, and based on an aggregation of elements according to their physical coefficients, we end up with a finer physics-based (PB) subdomain partition. Next, we define corners, edges, and faces for this PB partition, and select some of them to enforce subdomain continuity (primal faces/edges/corners). When the physical coefficient in each PB subdomain is constant and the set of selected primal faces/edges/corners satisfy a mild condition on the existence of acceptable paths, we can show both theoretically and numerically that the condition number does not depend on the contrast of the coefficient across subdomains. An extensive set of numerical experiments for 2D and 3D for the Poisson and linear elasticity problems is provided to support our findings. In particular, we show robustness and weak scalability of the new preconditioner variant up to 8232 cores when applied to 3D multi-material problems with the contrast of the physical coefficient up to 108 and more than half a billion degrees of freedom. For the scalability analysis, we have exploited a highly scalable advanced inter-level overlapped implementation of the preconditioner that deals very efficiently with the coarse problem computation. The proposed preconditioner is compared against a state-of-the-art implementation of an adaptive BDDC method in PETSc for thermal and mechanical multi-material problems.Peer ReviewedPostprint (author's final draft