An Optimized and Scalable Eigensolver for Sequences of Eigenvalue Problems

In many scientific applications the solution of non-linear differential equations are obtained through the set-up and solution of a number of successive eigenproblems. These eigenproblems can be regarded as a sequence whenever the solution of one problem fosters the initialization of the next. In addition, in some eigenproblem sequences there is a connection between the solutions of adjacent eigenproblems. Whenever it is possible to unravel the existence of such a connection, the eigenproblem sequence is said to be correlated. When facing with a sequence of correlated eigenproblems the current strategy amounts to solving each eigenproblem in isolation. We propose a alternative approach which exploits such correlation through the use of an eigensolver based on subspace iteration and accelerated with Chebyshev polynomials (ChFSI). The resulting eigensolver is optimized by minimizing the number of matrix-vector multiplications and parallelized using the Elemental library framework. Numerical results show that ChFSI achieves excellent scalability and is competitive with current dense linear algebra parallel eigensolvers.Comment: 23 Pages, 6 figures. First revision of an invited submission to special issue of Concurrency and Computation: Practice and Experienc

Large Scale Electronic Structure Studies on the Energetics of Dislocations in Al-Mg Materials System and Its Connection to Mesoscale Models

Computational modeling of dislocation behavior is vital for designing new lightweight metallic alloys. However, extraordinary challenges are posed by the multiscale physics ranging over a vast span of interacting length-scales from electronic-structure and atomic-scale effects at the dislocation core ($< 10^{-9} {rm m}$) to long-ranged elastic interactions at the continuum scale ($sim 10 upmu$). In particular, quantification of the energetics associated with electronic-structure effects inside the dislocation core and its interaction with the external macroscopic elastic fields have not been explored due to limitations of current electronic-structure methods based on the widely used plane-wave based discretization. This thesis seeks to address the above challenges by developing computational methodologies to conduct large-scale real-space electronic-structure studies of energetics of dislocations in Aluminum and Magnesium, and use these results to develop phenomenological connections to mesoscale models of plasticity like discrete dislocation dynamics (DDD), which study the collective behavior of the dislocations at longer length scales ($sim$ 1--15 $upmu$). First, a local real-space formulation of orbital-free Density Functional Theory is developed based on prior work, and implemented using finite-element discretization. The local real-space formulation coupled with bulk Dirichlet boundary conditions enables a direct computation of the isolated dislocation core energy. Studies on dislocations in Aluminum and Magnesium suggest that the core-size---region with significant contribution of electronic effects to dislocation energetics---is around seven to eleven times the magnitude of the Burgers vector. This is in stark contrast to prior displacement field based core size estimates of one to three times the magnitude of the Burgers vector. Interestingly, our study further indicates that the core-energy of the dislocations in both Aluminum and Magnesium is strongly dependent on external macroscopic strains with a non-zero slope at zero external strain. Next, the computed dislocation core energetics is used to develop a continuum model for an arbitrary aggregate of dislocations in an infinite isotropic elastic continua. This model, which accounts for the core energy dependence on macroscopic deformation provides a phenomenological approach to incorporate the electronic structure effects into mesoscale DDD simulations. Application of this model to derive nodal forces in a discrete dislocation network, leads to additional configurational forces beyond those considered in existing DDD models. Using case studies, we show that even up to distances of $10-15$ nm between the dislocations, these additional configurational forces are non-trivial in relation to the elastic Peach-Koehler force. Furthermore, the core force model is incorporated into a DDD implementation, where significant influence of core force on elementary dislocation mechanisms in Aluminum such as critical stress of a Frank-Read source and structure of a dislocation binary junction are demonstrated. To enable the above electronic-structure studies of dislocations in generic material systems, calculations using the more accurate and transferable Kohn-Sham Density Functional Theory (KS-DFT) are required. The final part of this thesis extends previous work on real-space adaptive spectral finite-element discretization of KS-DFT to develop numerical strategies and implementation innovations, which significantly reduce the computational pre-factor, while increasing the arithmetic intensity and lowering the data movement costs on both many-core and heterogeneous architectures. This has enabled systematically convergent and massively parallel (demonstrated up to 192,000 MPI tasks) KS-DFT calculations on material systems up to $sim 100,000$ electrons. Using GPUs, an unprecedented sustained performance of 46 PFLOPS (27.8% peak FP64 performance) is demonstrated on a large-scale benchmark dislocation system in Magnesium containing 105,080 electrons.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/153417/1/dsambit_1.pd

Algorithms for Large Scale Problems in Eigenvalue and Svd Computations and in Big Data Applications

As ”big data” has increasing influence on our daily life and research activities, it poses significant challenges on various research areas. Some applications often demand a fast solution of large, sparse eigenvalue and singular value problems; In other applications, extracting knowledge from large-scale data requires many techniques such as statistical calculations, data mining, and high performance computing. In this dissertation, we develop efficient and robust iterative methods and software for the computation of eigenvalue and singular values. We also develop practical numerical and data mining techniques to estimate the trace of a function of a large, sparse matrix and to detect in real-time blob-filaments in fusion plasma on extremely large parallel computers. In the first work, we propose a hybrid two stage SVD method for efficiently and accurately computing a few extreme singular triplets, especially the ones corresponding to the smallest singular values. The first stage achieves fast convergence while the second achieves the final accuracy. Furthermore, we develop a high-performance preconditioned SVD software based on the proposed method on top of the state-of-the-art eigensolver PRIMME. The method can be used with or without preconditioning, on parallel computers, and is superior to other state-of-the-art SVD methods in both efficiency and robustness. In the second study, we provide insights and develop practical algorithms to accomplish efficient and accurate computation of interior eigenpairs using refined projection techniques in non-Krylov iterative methods. By analyzing different implementations of the refined projection, we propose a new hybrid method to efficiently find interior eigenpairs without compromising accuracy. Our numerical experiments illustrate the efficiency and robustness of the proposed method. In the third work, we present a novel method to estimate the trace of matrix inverse that exploits the pattern correlation between the diagonal of the inverse of the matrix and that of some approximate inverse. We leverage various sampling and fitting techniques to fit the diagonal of the approximation to that of the inverse. Our method may serve as a standalone kernel for providing a fast trace estimate or as a variance reduction method for Monte Carlo in some cases. An extensive set of experiments demonstrate the potential of our method. In the fourth study, we provide first results on applying outlier detection techniques to effectively tackle the fusion blob detection problem on extremely large parallel machines. We present a real-time region outlier detection algorithm to efficiently find and track blobs in fusion experiments and simulations. Our experiments demonstrated we can achieve linear time speedup up to 1024 MPI processes and complete blob detection in two or three milliseconds

Discrétisation Espace-Temps d'Équations d'Ondes Élasto-Acoustiques dans des Bases Trefftz-DG Polynomiales

Discontinuous Finite Element Methods (DG FEM) have proven flexibility and accuracy for solving wave problems in complex media. However, they require a large number of degrees of freedom, which increases the corresponding computational cost compared with that of continuous finite element methods. Among the different variational approaches to solve boundary value problems, there exists a particular family of methods, based on the use of trial functions in the form of exact local solutions of the governing equations. The idea was first proposed by Trefftz in 1926, and since then it has been further developed and generalized. A Trefftz-DG variational formulation applied to wave problems reduces to surface integrals that should contribute to decreasing the computational costs.Trefftz-type approaches have been widely used for time-harmonic problems, while their implementation for time-dependent simulations is still limited. The feature of Trefftz-DG methods applied to time-dependent problems is in the use of space-time meshes. Indeed, standard DG methods lead to the construction of a semi-discrete system of ordinary differential equations in time which are integrated by using an appropriate scheme. But Trefftz-DG methods applied to wave problems lead to a global matrix including time and space discretizations which is huge and sparse. This significantly hampers the deployment of this technology for solving industrial problems.In this work, we develop a Trefftz-DG framework for solving mechanical wave problems including elasto-acoustic equations. We prove that the corresponding formulations are well-posed and we address the issue of solving the global matrix by constructing an approximate inverse obtained from the decomposition of the global matrix into a block-diagonal one. The inversion is then justified under a CFL-type condition. This idea allows for reducing the computational costs but its accuracy is limited to small computational domains. According to the limitations of the method, we have investigated the potential of Tent Pitcher algorithms following the recent works of Gopalakrishnan et al. It consists in constructing a space-time mesh made of patches that can be solved independently under a causality constraint. We have obtained very promising numerical results illustrating the potential of Tent Pitcher in particular when coupled with a Trefftz-DG method involving only surface terms. In this way, the space-time mesh is composed of elements which are 3D objects at most. It is also worth noting that this framework naturally allows for local time-stepping which is a plus to increase the accuracy while decreasing the computational burden.Les méthodes d'éléments finis de type Galerkine discontinu (DG FEM) ont démontré précision et efficacité pour résoudre des problèmes d'ondes dans des milieux complexes. Cependant, elles nécessitent un très grand nombre de degrés de liberté, ce qui augmente leur coût de calcul en comparaison du coût des méthodes d'éléments finis continus. Parmi les différentes approches variationnelles pour résoudre les problèmes aux limites, se distingue une famille particulière, basée sur l'utilisation de fonctions tests qui sont des solutions locales exactes des équations à résoudre. L'idée vient de E.Trefftz en 1926 et a depuis été largement développée et généralisée. Les méthodes variationnelles de type Trefftz-DG appliquées aux problèmes d'ondes se réduisent à des intégrales de surface, ce qui devrait contribuer à réduire les coûts de calcul.Les approches de type Trefftz ont été largement développées pour les problèmes harmoniques, mais leur utilisation pour des simulations en domaine transitoire est encore limitée. Quand elles sont appliquées dans le domaine temporel, les méthodes de Trefftz utilisent des maillages qui recouvrent le domaine espace-temps. C'est une des paraticularités de ces méthodes. En effet, les méthodes DG standards conduisent à la construction d'un système semi-discret d'équations différentielles ordinaires en temps qu'on intègre avec un schéma en temps explicite. Mais les méthodes de Trefftz-DG appliquées aux problèmes d'ondes conduisent à résoudre une matrice globale, contenant la discrétisation en espace et en temps, qui est de grande taille et creuse. Cette particularité gêne considérablement le déploiement de cette technologie pour résoudre des problèmes industriels.Dans ce travail, nous développons un environnement Trefftz-DG pour résoudre des problèmes d'ondes mécaniques, y compris les équations couplées de l'élasto-acoustique. Nous prouvons que les formulations obtenues sont bien posées et nous considérons la difficulté d'inverser la matrice globale en construisant un inverse approché obtenu à partir de la décomposition de la matrice globale en une matrice diagonale par blocs. Cette idée permet de réduire les coûts de calcul mais sa précision est limitée à de petits domaines de calcul. Etant données les limitations de la méthode, nous nous sommes intéressés au potentiel du "Tent Pitcher", en suivant les travaux récents de Gopalakrishnan et al. Il s'agit de construire un maillage espace-temps composé de macro-éléments qui peuvent être traités indépendamment en faisant une hypothèse de causalité. Nous avons obtenu des résultats préliminaires très encourageants qui illustrent bien l'intérêt du Tent Pitcher, en particulier quand il est couplé à une méthode de Trefftz-DG formulée à partir d'intégrales de surface seulement. Dans ce cas, le maillage espace-temps est composé d'éléments qui sont au plus de dimension 3. Il est aussi important de noter que ce cadre se prête à l'utilisation de pas de temps locaux ce qui est un plus pour gagner en précision avec des coûts de calcul réduits

Approximation of phase-field models with meshfree methods: exploring biomembrane dynamics

Las biomembranas constituyen la estructura de separación fundamental en las celulas animales, y son importantes en el diseño de sistemas bioinspirados. Su simulación presenta desafíos, especialmente cuando ésta implica dinámica y grandes cambios de forma o se estudian sistemas micrométricos, impidiendo el uso de modelos atomísticos y de grano grueso. El objetivo principal de esta tesis es el desarrollo de un marco computacional para entender la dinámica de biomembranas inmersas en fluido viscoso usando modelos de campo de fase. Los modelos de campo de fase introducen un campo escalar contínuo que define una interfase difusa, cuya física está codificada en las ecuaciones en derivadas parciales que la gobiernan. Estos modelos son capaces de soportar cambios dramáticos de forma y topología, y facilitan el acoplamiento de distintos fenómenos físicos. No obstante, presentan desafíos numéricos significativos, como el alto orden de las ecuaciones, la resolución de frentes móviles y abruptos, o una eficiente integración en el tiempo. En esta disertación abordamos estos puntos mediante la combinación de una discretización espacial con métodos sin malla usando las funciones base locales de máxima entropía, y una formulación variacional Lagrangiana para acoplamiento elástico-hidrodinámico. La suavidad del método sin malla genera una aproximación precisa del campo de fase y puede lidiar fácilmente con adaptatividad local, la aproximación Lagrangiana extiende de manera natural esta adaptividad a la dinámica, y la formulación variacional permite una integración variacional temporal no linealmente estable y robusta. La implementación numérica de estos métodos en un entorno de computación de alto rendimiento ha motivado el desarrollo de un nuevo código computacional. Este código integra el estado del arte de las librerías en paralelo e incorpora importantes contribuciones técnicas para solventar cuellos de botella que aparecen con el uso de métodos sin malla en computación a gran escala. El código resultante es flexible y ha sido aplicado a otros problemas científicos en varias colaboraciones que incluyen flexoelectricidad, conformado metálico, fluidos viscosos o fractura en materiales con energía de superficie altamente anisotrópica.Biomembranes are the fundamental separation structure in animal cells, and are also used in engineered bioinspired systems. Their simulation is challenging, particularly when large shape changes and dynamics are involved, or micrometer systems are considered, ruling out atomistic or coarse-grained molecular modeling. The main goal of this thesis is to develop a computational framework to understand the dynamics of biomembranes embedded in a viscous fluid using phase-field models. Phase-field models introduce a scalar continuous field to define a diffuse moving interface, whose physics is encoded in partial differential equations governing it. These models can deal with dramatic shape and topological transformations and are amenable to multiphysics coupling. However, they present significant numerical challenges, such as the high-order character of the equations, the resolution of sharp and moving fronts, or the efficient time-integration. We address all these issues through a combination of meshfree spacial discretization using local maximum-entropy basis functions, and a Lagrangian variational formulation of the coupled elasticity-hydrodynamics. The smooth meshfree approach provides accurate approximations of the phase-field and can easily deal with local adaptivity, the Lagrangian approach naturally extend adaptivity to dynamics, and the variational formulation enables nonlinearly-stable robust variational time integration. The numerical implementation of these methods in a high-performance computing framework has motivated the development of a new computer code, which integrates state-of-the-art parallel libraries and incorporates important technical contributions to overcome bottlenecks that arise in meshfree methods for large-scale problems. The resulting code is flexible and has been applied to other scientific problems in a number of collaborations dealing with flexoelectricity, metal forming, creeping flows, or fracture in materials with strongly anisotropic surface energy

Hp-spectral Methods for Structural Mechanics and Fluid Dynamics Problems

We consider the usage of higher order spectral element methods for the solution of problems in structures and fluid mechanics areas. In structures applications we study different beam theories, with mixed and displacement based formulations, consider the analysis of plates subject to external loadings, and large deformation analysis of beams with continuum based formulations. Higher order methods alleviate the problems of locking that have plagued finite element method applications to structures, and also provide for spectral accuracy of the solutions. For applications in computational fluid dynamics areas we consider the driven cavity problem with least squares based finite element methods. In the context of higher order methods, efficient techniques need to be devised for the solution of the resulting algebraic systems of equations and we explore the usage of element by element bi-orthogonal conjugate gradient solvers for solving problems effectively along with domain decomposition algorithms for fluid problems. In the context of least squares finite element methods we also explore the usage of Multigrid techniques to obtain faster convergence of the the solutions for the problems of interest. Applications of the traditional Lagrange based finite element methods with the Penalty finite element method are presented for modelling porous media flow problems. Finally, we explore applications to some CFD problems namely, the flow past a cylinder and forward facing step