Hugues, Maxime

Petiton, Serge

INRIA a CCSD electronic archive server

HAL Id: inria-00581707https://hal.inria.fr/inria-00581707Submitted on 31 Mar 2011HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Optimized Sparse Matrix Formats on GT200 and FermiGPUsMaxime Hugues, Serge PetitonTo cite this version:Maxime Hugues, Serge Petiton. Optimized Sparse Matrix Formats on GT200 and Fermi GPUs.International Conference On Preconditioning Techniques For Scientific And Industrial Applications,Preconditioning 2011, May 2011, Bordeaux, France. ￿inria-00581707￿Optimized Sparse Matrix Formats on GT200 and Fermi GPUsList of authors:M. Hugues 1 and S. PetitonCNRS/LIFL, Laboratoire d’Informatique Fondamentale de Lille, Francemaxime.hugues@lifl.fr, serge.petiton@lifl.fr 2The rise of computing power now goes through the multiplication of cores on a single-chipbecause of the frequencies limitation. Multicore processors will compose future peta and exas-cale supercomputers which will allow applications to sustain many petaflops and thus to makebreakthrough on scientific challenge problems. One massively multicore processor already avail-able on the market is the GPU and has been making the rebirth of data parallel programmingwhich is focused on data mapping and movements. GPU is a high peak performance archi-tecture which gives the opportunity to accelerate scientific applications. Dense linear algebrahas already shown that it could benefit from this architecture [1] and some libraries such asCUFFT or CUBLAS are available to easily develop high performance applications. In oppo-site, sparse matrices have irregular data structures which generate a memory bottleneck byaccessing data in a non-contiguous fashion. The main optimization strategy is to get the mostefficient memory accesses by arranging data in a well-suited manner depending on the matrixstructure. Sparse formats define this data arrangement in memory. The sparse matrix vectorproduct (SpMV or Ax = y) is one of the most important kernel in scientific computing. Thiskernel is intensively used and is the main performance key of iterative methods, such as Con-jugate Gradient or GMRES, for solving partial differential equations, large linear systems oreigenvalues problems. Thereby, many researches have been conducted to optimize this kernelimplementation through various sparse formats on many-core processors. On multicore proces-sors, the Block Compressed Sparse Row (BCSR) format has been evaluated and optimized ondifferent platforms like Cell BE, Intel, AMD and Sun processors [2]. It clearly achieves the bestperformances by reusing sparse matrix elements in the different cache levels thanks to the reg-ister and cache blocking techniques. On massively parallel architectures, some implementationsof SpMV kernel have been evaluated by using various sparse formats on NVIDIA GPUs [3].They have shown that their HYB Format, a mix between the COO and Ellpack/ITpack format,achieves the best results on a set of matrices previously used on multicore processors and theDIA and Ellpack/ITpack is the most efficient format for a finite differences discretization of theLaplacian. In [4], authors have also proposed an optimized implementation of the CompressedSparse Row (CSR) format by using vectorization and padding which increase the efficiency andoutperform the implementation of Nvidia. A sparse linear solver [5] method has been devel-oped on GPU and has proposed the implementation of the SpMV by using the BCSR formatfor register blocking. In spite the fact, GPU is called data parallel architecture, like ConnectionMachine, the past results obtained on this type of architecture are not fully reusable as is.121We propose in this talk to study some implementations on GPU of sparse format which has beensuccessful on data parallel machine (CSC, SGP [6]) and present a performance comparison ofcommon sparse formats for the SpMV kernel with our optimizations. This work also comparesthe performance achievement on the GT200 and the new Nvidia architecture, Fermi, throughtwo evaluation methodologies The first method is based on a set of matrices from the Universityof Florida Sparse Matrix Collection and a finite difference discretization of the Laplace operatorin 2D with different stencil in order to compare our results to previous of [3]. The second oneis focused on the sparse matrix format performances following the distribution of the nonzeros values in the matrix by using a matrix pattern, named C-Diagonal q-perturbed. Thesepatterns was introduced to evaluate sparse format performances on data parallel supercomputersfollowing communications which depends on the distribution of nnz [7]. The C-Diagonal matrixis defined as a matrix with nc diagonals from the main diagonal to the right. The C-Diagonalq-perturbed matrix is built from a C-Diagonal matrix by modifying the data structure with arandom perturbation. Each element of the matrix has a perturbation called q which defineswhether the value has to be set at a random distance from the main diagonal, thus the elementcolumn index is modified.On the set of matrices and the GT200, we demonstrate Ellpack/ITpack format with a column-major order in memory is the best format for common matrices by achieving 18.7Gflop/s insingle precision and 11Gflop/s in double precision. Our variant of Ellpack/ITpack with arow-major order performs also well, depending on the matrix structure. For matrices with annz per row lower than 16, this implementation and all vectorized formats, such as the CSRimplementation, perform poorly because they can not realize a coalescing with a half-warp. Theproposed vectorized version of BCSR format achieves also good performance on matrices witha sufficient nnz to have an optimized blocking. Results on compressed column formats such asCSC, Ellpack/ITpack with column compression and SGP show that type of format perform lessbetter than compressed row format. This comes from the summing of all elements belonging tothe same row which makes irregular data access and degrades performances. The performancestudy of sparse formats following the distribution of nnz also shows interesting results about thechoice of format for different matrix structure. Ellpack/ITpack implementation with column-major order stays the most efficient sparse format by performing 21Gflop/s in single precisionand 13.8Gflop/s in double precision with q = 0. Nevertheless, we have seen for sparse matriceswith a strong distribution of nnz (q > 0.5) that the various formats achieve approximatelythe same performance by reaching 2.0 Gflop/s in single and double precision. This can beinteresting for iterative methods which require high precision to keep stability and convergefaster without loosing performance. On Fermi, the performance behavior observed is the samewith an improvement for some sparse formats such as CSR and BCSR. The first one benefits ofa 1.7 SpeedUp in comparison to the GT200 by achieving 17Gflop/s and the second one realizes25Gflop/s with a SpeedUp of 2. BCSR outperform Ellpack/ITpack format on Fermi in singleand double precision by using the new cache levels on the GPU. The C-diagonal q-perturbedbenchmark shows that Ellpack/ITpack is still the best because as the distribution of nnz growsthe number of Block for BCSR also grows and creates extra zeros to compute. Moreover, wecan notice the performance are quiet the same but they decline faster when q is growing than2on the GT200 architecture and finally reach less figures for q = 1.0 in single precision. In doubleprecision, we observe a SpeedUp of 1.5 when the distribution of nnz increases and demonstratesthe effort of Nvidia on Fermi to optimize double precision computations is clearly significant.References[1] V. Volkov and J. Demmel, Benchmarking GPUs to Tune Dense Linear Algebra, SC’08:Proceedings of the 2008 ACM/IEEE conference on Supercomputing, 31, 2008.[2] S. Williams, L. Oliker, R. W. Vuduc, J. Shalf, K. A. Yelick and J. Demmel, Optimization ofSparse Matrix-Vector Multiplication on Emerging Multicore Platforms, SC’07: Proceedingsof the ACM/IEEE Conference on High Performance Networking and Computing, 38, 2007.[3] N. Bell and M. Garland, Implementing Sparse Matrix-Vector Multiplication on Throughput-Oriented Processors, SC ’09: Proceedings of the Conference on High Performance Comput-ing Networking, Storage and Analysis, 1–11, 2009.[4] M. Baskaran and R. Bordawekar, Optimizing Sparse Matrix-Vector Multiplication on GPUs.IBM Technical Report, 2009.[5] L. Buatois, G. Caumon and B. Levy, Concurrent Number Cruncher: a GPU Implementationof a General Sparse Linear Solver, Int. J. Parallel Emerg. Distrib. Syst., 24:205–223, 2009.[6] W. R. Ferng, S. G. Petiton and Kesheng Wu, Basic Sparse Computations on Data ParallelComputers, PPSC, 462–466, 1993.[7] S. G. Petiton and C. Weill-Duflo, Massively Parallel Preconditioners for the Sparse Conju-gate Gradient Method, Parallel Processing: CONPAR 92 - VAPP V, Second Joint Inter-national Conference on Vector and Parallel Processing, 373-378, 1992.3

Optimized Sparse Matrix Formats on GT200 and Fermi GPUs

HAL - Lille 3

Optimized Sparse Matrix Formats on GT200 and FermiGPUsMaxime Hugues, Serge PetitonTo cite this version:Maxime Hugues, Serge Petiton. Optimized Sparse Matrix Formats on GT200 and Fermi GPUs.International Conference On Preconditioning Techniques For Scientific And Industrial Appli-cations, Preconditioning 2011, May 2011, Bordeaux, France. <inria-00581707>HAL Id: inria-00581707https://hal.inria.fr/inria-00581707Submitted on 31 Mar 2011HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestine´e au de´poˆt et a` la diffusion de documentsscientifiques de niveau recherche, publie´s ou non,e´manant des e´tablissements d’enseignement et derecherche franc¸ais ou e´trangers, des laboratoirespublics ou prive´s.Optimized Sparse Matrix Formats on GT200 and Fermi GPUsList of authors:M. Hugues 1 and S. PetitonCNRS/LIFL, Laboratoire d’Informatique Fondamentale de Lille, Francemaxime.hugues@lifl.fr, serge.petiton@lifl.fr 2The rise of computing power now goes through the multiplication of cores on a single-chipbecause of the frequencies limitation. Multicore processors will compose future peta and exas-cale supercomputers which will allow applications to sustain many petaflops and thus to makebreakthrough on scientific challenge problems. One massively multicore processor already avail-able on the market is the GPU and has been making the rebirth of data parallel programmingwhich is focused on data mapping and movements. GPU is a high peak performance archi-tecture which gives the opportunity to accelerate scientific applications. Dense linear algebrahas already shown that it could benefit from this architecture [1] and some libraries such asCUFFT or CUBLAS are available to easily develop high performance applications. In oppo-site, sparse matrices have irregular data structures which generate a memory bottleneck byaccessing data in a non-contiguous fashion. The main optimization strategy is to get the mostefficient memory accesses by arranging data in a well-suited manner depending on the matrixstructure. Sparse formats define this data arrangement in memory. The sparse matrix vectorproduct (SpMV or Ax = y) is one of the most important kernel in scientific computing. Thiskernel is intensively used and is the main performance key of iterative methods, such as Con-jugate Gradient or GMRES, for solving partial differential equations, large linear systems oreigenvalues problems. Thereby, many researches have been conducted to optimize this kernelimplementation through various sparse formats on many-core processors. On multicore proces-sors, the Block Compressed Sparse Row (BCSR) format has been evaluated and optimized ondifferent platforms like Cell BE, Intel, AMD and Sun processors [2]. It clearly achieves the bestperformances by reusing sparse matrix elements in the different cache levels thanks to the reg-ister and cache blocking techniques. On massively parallel architectures, some implementationsof SpMV kernel have been evaluated by using various sparse formats on NVIDIA GPUs [3].They have shown that their HYB Format, a mix between the COO and Ellpack/ITpack format,achieves the best results on a set of matrices previously used on multicore processors and theDIA and Ellpack/ITpack is the most efficient format for a finite differences discretization of theLaplacian. In [4], authors have also proposed an optimized implementation of the CompressedSparse Row (CSR) format by using vectorization and padding which increase the efficiency andoutperform the implementation of Nvidia. A sparse linear solver [5] method has been devel-oped on GPU and has proposed the implementation of the SpMV by using the BCSR formatfor register blocking. In spite the fact, GPU is called data parallel architecture, like ConnectionMachine, the past results obtained on this type of architecture are not fully reusable as is.121We propose in this talk to study some implementations on GPU of sparse format which has beensuccessful on data parallel machine (CSC, SGP [6]) and present a performance comparison ofcommon sparse formats for the SpMV kernel with our optimizations. This work also comparesthe performance achievement on the GT200 and the new Nvidia architecture, Fermi, throughtwo evaluation methodologies The first method is based on a set of matrices from the Universityof Florida Sparse Matrix Collection and a finite difference discretization of the Laplace operatorin 2D with different stencil in order to compare our results to previous of [3]. The second oneis focused on the sparse matrix format performances following the distribution of the nonzeros values in the matrix by using a matrix pattern, named C-Diagonal q-perturbed. Thesepatterns was introduced to evaluate sparse format performances on data parallel supercomputersfollowing communications which depends on the distribution of nnz [7]. The C-Diagonal matrixis defined as a matrix with nc diagonals from the main diagonal to the right. The C-Diagonalq-perturbed matrix is built from a C-Diagonal matrix by modifying the data structure with arandom perturbation. Each element of the matrix has a perturbation called q which defineswhether the value has to be set at a random distance from the main diagonal, thus the elementcolumn index is modified.On the set of matrices and the GT200, we demonstrate Ellpack/ITpack format with a column-major order in memory is the best format for common matrices by achieving 18.7Gflop/s insingle precision and 11Gflop/s in double precision. Our variant of Ellpack/ITpack with arow-major order performs also well, depending on the matrix structure. For matrices with annz per row lower than 16, this implementation and all vectorized formats, such as the CSRimplementation, perform poorly because they can not realize a coalescing with a half-warp. Theproposed vectorized version of BCSR format achieves also good performance on matrices witha sufficient nnz to have an optimized blocking. Results on compressed column formats such asCSC, Ellpack/ITpack with column compression and SGP show that type of format perform lessbetter than compressed row format. This comes from the summing of all elements belonging tothe same row which makes irregular data access and degrades performances. The performancestudy of sparse formats following the distribution of nnz also shows interesting results about thechoice of format for different matrix structure. Ellpack/ITpack implementation with column-major order stays the most efficient sparse format by performing 21Gflop/s in single precisionand 13.8Gflop/s in double precision with q = 0. Nevertheless, we have seen for sparse matriceswith a strong distribution of nnz (q > 0.5) that the various formats achieve approximatelythe same performance by reaching 2.0 Gflop/s in single and double precision. This can beinteresting for iterative methods which require high precision to keep stability and convergefaster without loosing performance. On Fermi, the performance behavior observed is the samewith an improvement for some sparse formats such as CSR and BCSR. The first one benefits ofa 1.7 SpeedUp in comparison to the GT200 by achieving 17Gflop/s and the second one realizes25Gflop/s with a SpeedUp of 2. BCSR outperform Ellpack/ITpack format on Fermi in singleand double precision by using the new cache levels on the GPU. The C-diagonal q-perturbedbenchmark shows that Ellpack/ITpack is still the best because as the distribution of nnz growsthe number of Block for BCSR also grows and creates extra zeros to compute. Moreover, wecan notice the performance are quiet the same but they decline faster when q is growing than2on the GT200 architecture and finally reach less figures for q = 1.0 in single precision. In doubleprecision, we observe a SpeedUp of 1.5 when the distribution of nnz increases and demonstratesthe effort of Nvidia on Fermi to optimize double precision computations is clearly significant.References[1] V. Volkov and J. Demmel, Benchmarking GPUs to Tune Dense Linear Algebra, SC’08:Proceedings of the 2008 ACM/IEEE conference on Supercomputing, 31, 2008.[2] S. Williams, L. Oliker, R. W. Vuduc, J. Shalf, K. A. Yelick and J. Demmel, Optimization ofSparse Matrix-Vector Multiplication on Emerging Multicore Platforms, SC’07: Proceedingsof the ACM/IEEE Conference on High Performance Networking and Computing, 38, 2007.[3] N. Bell and M. Garland, Implementing Sparse Matrix-Vector Multiplication on Throughput-Oriented Processors, SC ’09: Proceedings of the Conference on High Performance Comput-ing Networking, Storage and Analysis, 1–11, 2009.[4] M. Baskaran and R. Bordawekar, Optimizing Sparse Matrix-Vector Multiplication on GPUs.IBM Technical Report, 2009.[5] L. Buatois, G. Caumon and B. Levy, Concurrent Number Cruncher: a GPU Implementationof a General Sparse Linear Solver, Int. J. Parallel Emerg. Distrib. Syst., 24:205–223, 2009.[6] W. R. Ferng, S. G. Petiton and Kesheng Wu, Basic Sparse Computations on Data ParallelComputers, PPSC, 462–466, 1993.[7] S. G. Petiton and C. Weill-Duflo, Massively Parallel Preconditioners for the Sparse Conju-gate Gradient Method, Parallel Processing: CONPAR 92 - VAPP V, Second Joint Inter-national Conference on Vector and Parallel Processing, 373-378, 1992.3

https://hal.inria.fr/inria-00581707/file/preconditioning2011_submission_33.pdf

Optimized Sparse Matrix Formats on GT200 and Fermi GPUs

Abstract

Similar works

Full text

Available Versions

INRIA a CCSD electronic archive server

HAL - Lille 3