Anzt, Hartwig

Götz, Markus

English

KITopen

Steinbuch Centre for Computing (SCC)KIT – The Research University in the Helmholtz Association www.kit.eduMachine Learning-Aided Numerical Linear Algebra: Convolutional Neural Networks for the Efficient Preconditioner GenerationMarkus Götz, Hartwig AnztScalA’18: 9th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems, WS at SC18Steinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraJacobi method based on diagonal scaling:Can be used asIterative solverPreconditionerMotivation: Block Jacobi Preconditioning11/11/182,Steinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraJacobi method based on diagonal scaling:Can be used asIterative solverPreconditionerExtension: block-Jacobi:Set of diagonal blocksTreat each block as linear systemLarger blocksbetter convergence, more expensive to computeMotivation: Block Jacobi Preconditioning11/11/183,Steinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraJacobi method based on diagonal scaling:Can be used asIterative solverPreconditionerExtension: block-Jacobi:Set of diagonal blocksTreat each block as linear systemLarger blocksbetter convergence, more expensive to computeMotivation: Block Jacobi Preconditioning11/11/184,Steinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraMotivation: Block-Jacobi Preconditioning11/11/185Determination of block necessaryUnproblematic if information about systemis known aprioriCluster AnalysisWorks wellHighly expensive to computeSteinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraMotivation: Block-Jacobi Preconditioning11/11/186Determination of block necessaryUnproblematic if information about systemis known aprioriCluster AnalysisWorks wellHighly expensive to computeSupervariable Agglomeration (SVA)State-of-the-artResults are okaySequential by definitionSteinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraMotivation: Block-Jacobi Preconditioning11/11/187Determination of block necessaryUnproblematic if information about systemis known aprioriCluster AnalysisWorks wellHighly expensive to computeSupervariable Agglomeration (SVA)State-of-the-artResults are okaySequential by definition„Looking at it“HeuristicFeeling for natural blocksSteinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraMimics behaviour of biological eyesEfficient in detecting recurring patternsCrash course: Convolutional Neural Networks (CNN)11/11/188© https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/Steinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraInterpret matrix as imageMainly interested in non-zero patternsclose to main diagonalIdea – Let CNN „look at“ sparsity pattern11/11/189Steinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraInterpret matrix as imageMainly interested in non-zero patternsclose to main diagonalNeed dataset with labelsOther algorithms set upper boundNo credit card for Amazon MturkIdea – Let CNN „look at“ sparsity pattern11/11/18100 20 40 60 80 100 120020Convolution	of	dimension	(2w+1)	x	(2k+1).	.	.	Padding	of	size	kSteinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraInterpret matrix as imageMainly interested in non-zero patternsclose to main diagonalNeed dataset with labelsOther algorithms set upper boundNo credit card for Amazon MturkGenerate artificial dataBoundaries known3000 matricesUniform size, 128x128𝑤 = 10Random block + gaussian noise (𝜇 = 10)Idea – Let CNN „look at“ sparsity pattern11/11/18110 20 40 60 80 100 120020Convolution	of	dimension	(2w+1)	x	(2k+1).	.	.	Padding	of	size	kSteinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraSingle block ResNet architecture4 Convolutional layers1 Fully-connected prediction layerDropout and L2 regularizationOpen-source script in Pythonhttps://github.com/Markus-Goetz/block-predictionKeras + TensorFlow for CNNHDF5 I/OProcessing performanceSingle nVidia K80Batch size=1, 3000 matrices in 2.5sBatch size=1500, 3000 matrices in 0.6msImplementation11/11/18120 20 40 60 80 100 120020Convolution	of	dimension	(2w+1)	x	(2k+1).	.	.	Padding	of	size	kSamples of convolved matricesdenoisednoise patternblock candidatesSteinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraLoss function:CNN Training Process11/11/1813Steinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraPrediction Performance Evaluation11/11/1814LabelsPredictionTrue-positivesFalse-positivesFalse-negativesCNNSVA10SVA25Steinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraHigh-level Performance Analysis11/11/1815Used predicted block boundariesin a Jacobi preconditioner600 test matricesAverage iteration count~22% less iteration with CNN compared to no blocksSteinbuch Centre for Computing (SCC)Markus Götz – Machine Learning-Aided Numerical Linear AlgebraSummary and Outlook11/11/1816SummaryUsed CNN to predict blocks for Jacobi block preconditionersReduction in solver iterationsParallel and fast prediction of blocks, usable on GPUsNext StepsManually label matricesAdaptability to other preconditioners (ILU/ILUT)Robustness study of CNN architecture and input data

Machine learning-aided numerical linear algebra: Convolutional neural network for the efficient preconditioner generation

https://publikationen.bibliothek.kit.edu/1000086923/36140557

Machine learning-aided numerical linear algebra: Convolutional neural network for the efficient preconditioner generation

Abstract

Similar works

Full text

Available Versions

KITopen