DisPar Methods and Their Implementation on a Heterogeneous PC Cluster

Esta dissertação avalia duas áreas cruciais da simulação de advecção- difusão. A primeira parte é dedicada a estudos numéricos. Foi comprovado que existe uma relação directa entre os momentos de deslocamento de uma partícula de poluente e os erros de truncatura. Esta relação criou os fundamentos teóricos para criar uma nova família de métodos numéricos, DisPar. Foram introduzidos e avaliados três métodos. O primeiro é um método semi-Lagrangeano 2D baseado nos momentos de deslocamento de uma partícula para malhas regulares, DisPar-k. Com este método é possível controlar explicitamente o erro de truncatura desejado. O segundo método também se baseia nos momentos de deslocamento de uma partícula, sendo, contudo, desenvolvido para malhas uniformes não regulares, DisParV. Este método também apresentou uma forte robustez numérica. Ao contrário dos métodos DisPar-K e DisParV, o terceiro segue uma aproximação Eulereana com três regiões de destino da partícula. O método foi desenvolvido de forma a manter um perfil de concentração inicial homogéneo independentemente dos parâmetros usados. A comparação com o método DisPar-k em situações não lineares realçou as fortes limitações associadas aos métodos de advecção-difusão em cenários reais. A segunda parte da tese é dedicada à implementação destes métodos num Cluster de PCs heterogéneo. Para o fazer, foi desenvolvido um novo esquema de partição, AORDA. A aplicação, Scalable DisPar, foi implementada com a plataforma da Microsoft .Net, tendo sido totalmente escrita em C#. A aplicação foi testada no estuário do Tejo que se localiza perto de Lisboa, Portugal. Para superar os problemas de balanceamento de cargas provocados pelas marés, foram implementados diversos esquemas de partição: “Scatter Partitioning”, balanceamento dinâmico de cargas e uma mistura de ambos. Pelos testes elaborados, foi possível verificar que o número de máquinas vizinhas se apresentou como o mais limitativo em termos de escalabilidade, mesmo utilizando comunicações assíncronas. As ferramentas utilizadas para as comunicações foram a principal causa deste fenómeno. Aparentemente, o Microsoft .Net remoting 1.0 não funciona de forma apropriada nos ambientes de concorrência criados pelas comunicações assíncronas. Este facto não permitiu a obtenção de conclusões acerca dos níveis relativos de escalabilidade das diferentes estratégias de partição utilizadas. No entanto, é fortemente sugerido que a melhor estratégia irá ser “Scatter Partitioning” associada a balanceamento dinâmico de cargas e a comunicações assíncronas. A técnica de “Scatter Partitioning” mitiga os problemas de desbalanceamentos de cargas provocados pelas marés. Por outro lado, o balanceamento dinâmico será essencialmente activado no inicio da simulação para corrigir possíveis problemas nas previsões dos poderes de cada processador.This thesis assesses two main areas of the advection-diffusion simulation. The first part is dedicated to the numerical studies. It has been proved that there is a direct relation between pollutant particle displacement moments and truncation errors. This relation raised the theoretical foundations to create a new family of numerical methods, DisPar. Three methods have been introduced and appraised. The first is a 2D semi- Lagrangian method based on particle displacement moments for regular grids, DisPar-k. With this method one can explicitly control the desired truncation error. The second method is also based on particle displacement moments but it is targeted to regular/non-uniform grids, DisParV. The method has also shown a strong numerical capacity. Unlike DisPar-k and DisParV, the third method is a Eulerian approximation for three particle destination units. The method was developed so that an initial concentration profile will be kept homogeneous independently of the used parameters. The comparison with DisPar-k in non-linear situations has emphasized the strong shortcomings associated with numerical methods for advection-diffusion in real scenarios. The second part of the dissertation is dedicated to the implementation of these methods in a heterogeneous PC Cluster. To do so, a new partitioning method has been developed, AORDA. The application, Scalable DisPar, was implemented with the Microsoft .Net framework and was totally written in C#. The application was tested on the Tagus Estuary, near Lisbon (Portugal). To overcome the load imbalances caused by tides scatter partitioning was implemented, dynamic load balancing and a mix of both. By the tests made, it was possible to verify that the number of neighboring machines was the main factor affecting the application scalability, even with asynchronous communications. The tools used for communications mainly caused this. Microsoft .Net remoting 1.0 does not seem to properly work in environments with concurrency associated with the asynchronous communications. This did not allow taking conclusions about the relative efficiency between the partitioning strategies used. However, it is strongly suggested that the best approach will be to scatter partitioning with dynamic load balancing and with asynchronous communications. Scatter partitioning mitigates load imbalances caused by tides and dynamic load balancing is basically trigged at the begging of the simulation to correct possible problems in processor power predictions

A comparison of Eulerian and Lagrangian transport and non-linear reaction algorithms

When laboratory-measured chemical reaction rates are used in simulations at the field-scale, the models typically overpredict the apparent reaction rates. The discrepancy is primarily due to poorer mixing of chemically distinct waters at the larger scale. As a result, realistic field-scale predictions require accurate simulation of the degree of mixing between fluids. The Lagrangian particle-tracking (PT) method is a now-standard way to simulate the transport of conservative or sorbing solutes. The method’s main advantage is the absence of numerical dispersion (and its artificial mixing) when simulating advection. New algorithms allow particles of different species to interact in nonlinear (e.g., bimolecular) reactions. Therefore, the PT methods hold a promise of more accurate field-scale simulation of reactive transport because they eliminate the masking effects of spurious mixing due to advection errors inherent in grid-based methods. A hypothetical field-scale reaction scenario is constructed and run in PT and Eulerian (finite-volume/finite-difference) simulators. Grid-based advection schemes considered here include 1st- to 3rd-order spatially accurate total-variation-diminishing flux-limiting schemes, both of which are widely used in current transport/reaction codes. A homogeneous velocity field in which the Courant number is everywhere unity, so that the chosen Eulerian methods incur no error when simulating advection, shows that both the Eulerian and PT methods can achieve convergence in the L1 (integrated concentration) norm, but neither shows stricter pointwise convergence. In this specific case with a constant dispersion coefficient and bimolecular reaction A+B¿P, the correct total amount of product is 0.221MA0, where MA0 is the original mass of reactant A. When the Courant number drops, the grid-based simulations can show remarkable errors due to spurious over- and under-mixing. In a heterogeneous velocity field (keeping the same constant and isotropic dispersion), the PT simulations show an increased reaction total from 0.221MA0 to 0.372MA0 due to fluid deformation, while the 1st-order Eulerian simulations using ˜ 106 cells (with a classical grid Peclet number ¿x/aL of 10) have total product of 0.53MA0, or approximately twice as much additional reaction due to advection error. The 3rd-order TVD algorithm fares better, with total product of 0.394MA0, or about 1.14 times the increased reaction total. A very strict requirement on grid Peclet numbers for Eulerian simulations will be required for realistic reactions because of their nonlinear nature. We analytically estimate the magnitude of the effect for the end-member cases of very fast and very slow reactions and show that in either case, the mass produced is proportional to View the MathML source where Pe is the Peclet number. Therefore, extra mass is produced according to View the MathML source where the dispersion includes any numerical dispersion error. We test two PT methods, one that kills particles upon reaction and another that decrements a particle’s mass. For the bimolecular reaction studied here, the computational demands of the particle-killing methods are much smaller than, and the particle-number-preserving algorithm are on par with, the fastest Eulerian methods.Peer ReviewedPostprint (author's final draft

Parallel Local Approximation MCMC for Expensive Models

Performing Bayesian inference via Markov chain Monte Carlo (MCMC) can be exceedingly expensive when posterior evaluations invoke the evaluation of a computationally expensive model, such as a system of PDEs. In recent work [J. Amer. Statist. Assoc., 111 (2016), pp. 1591-1607] we described a framework for constructing and refining local approximations of such models during an MCMC simulation. These posterior-adapted approximations harness regularity of the model to reduce the computational cost of inference while preserving asymptotic exactness of the Markov chain. Here we describe two extensions of that work. First, we prove that samplers running in parallel can collaboratively construct a shared posterior approximation while ensuring ergodicity of each associated chain, providing a novel opportunity for exploiting parallel computation in MCMC. Second, focusing on the Metropolis-adjusted Langevin algorithm, we describe how a proposal distribution can successfully employ gradients and other relevant information extracted from the approximation. We investigate the practical performance of our approach using two challenging inference problems, the first in subsurface hydrology and the second in glaciology. Using local approximations constructed via parallel chains, we successfully reduce the run time needed to characterize the posterior distributions in these problems from days to hours and from months to days, respectively, dramatically improving the tractability of Bayesian inference.United States. Department of Energy. Office of Science. Scientific Discovery through Advanced Computing (SciDAC) Program (award DE-SC0007099)Natural Sciences and Engineering Research Council of CanadaUnited States. Office of Naval Researc

Surrogate-assisted Bayesian inversion for landscape and basin evolution models

The complex and computationally expensive features of the forward landscape and sedimentary basin evolution models pose a major challenge in the development of efficient inference and optimization methods. Bayesian inference provides a methodology for estimation and uncertainty quantification of free model parameters. In our previous work, parallel tempering Bayeslands was developed as a framework for parameter estimation and uncertainty quantification for the landscape and basin evolution modelling software Badlands. Parallel tempering Bayeslands features high-performance computing with dozens of processing cores running in parallel to enhance computational efficiency. Although parallel computing is used, the procedure remains computationally challenging since thousands of samples need to be drawn and evaluated. In large-scale landscape and basin evolution problems, a single model evaluation can take from several minutes to hours, and in certain cases, even days. Surrogate-assisted optimization has been with successfully applied to a number of engineering problems. This motivates its use in optimisation and inference methods suited for complex models in geology and geophysics. Surrogates can speed up parallel tempering Bayeslands by developing computationally inexpensive surrogates to mimic expensive models. In this paper, we present an application of surrogate-assisted parallel tempering where that surrogate mimics a landscape evolution model including erosion, sediment transport and deposition, by estimating the likelihood function that is given by the model. We employ a machine learning model as a surrogate that learns from the samples generated by the parallel tempering algorithm. The results show that the methodology is effective in lowering the overall computational cost significantly while retaining the quality of solutions.Comment: Under review. arXiv admin note: text overlap with arXiv:1811.0868