A Distributed Algorithm For Large-Scale Graph Partitioning

Detta kandidatarbete har sin placering på Skeppsbron/Skeppsbrokajen i centrala Stockholm. Inriktningen jag valde var att rita ett förslag för en fiskmarknad som skulle placeras på denna plats. Mitt arbete har fått inspireras av Sveriges största och mest kända fiskmarknad, Feskekörkan, i Göteborg. Analyser av Feskekörkan som organisation och dess planlösning har i mitt arbete lett till en tektoniskt uppbyggd struktur där material byggnadskonstruktion var viktiga element. Med bland annat en fiskfjällsfasad i mässing och en bärande skelett av storskaliga limträbalkar. Platsen som byggnaden ligger på är ett välbesökt promenadstråk med en bred och lång kajkant som används flitigt av såväl, turister som besöker gamla stan och det kungliga slottet, och Stockholmare som tar sig mellan Södermalm och Norrmalm. Jag har valt att bebygga platsen på ett sätt som både tar vara på det vackra promenadstråket men också ger möjlighet för besökande att stanna upp och ta del av fiskmarknaden.This candidate's work has its placement on Skeppsbron/Skeppsbrokajen in the central area of Stockholm. The focus I chose was to draw a proposal for a fishmarket that would be placed at this location. My work has been inspired by the largest and most famous fish market in Sweden, Feskekörkan, in Gothenburg. Analyses of Feskekörkan’s organization and its plan has, in my work, led to a tectonically constructed structure where building materials were important elements. Including a fish scale facade made of brass and a bearing skeleton of large glulam beams. The place which the building is situated on a popular promenade with a broad and long quay which is widely used by both, tourists visiting the Old Town and the Royal Palace, and the Stockholm citizens who ride between Södermalm and Norrmalm. I have chosen to build on the site in a way that both takes advantage of the beautiful promenade but also provides the opportunity for visitors to stop and take some of the fish market

Asynchronous Teams and Tasks in a Message Passing Environment

As the discipline of scientific computing grows, so too does the "skills gap" between the increasingly complex scientific applications and the efficient algorithms required. Increasing demand for computational power on the march towards exascale requires innovative approaches. Closing the skills gap avoids the many pitfalls that lead to poor utilisation of resources and wasted investment. This thesis tackles two challenges: asynchronous algorithms for parallel computing and fault tolerance. First I present a novel asynchronous task invocation methodology for Discontinuous Galerkin codes called enclave tasking. The approach modifies the parallel ordering of tasks that allows for efficient scaling on dynamic meshes up to 756 cores. It ensures high levels of concurrency and intermixes tasks of different computational properties. Critical tasks along domain boundaries are prioritised for an overlap of computation and communication. The second contribution is the teaMPI library, forming teams of MPI processes exchanging consistency data through an asynchronous "heartbeat". In contrast to previous approaches, teaMPI operates fully asynchronously with reduced overhead. It is also capable of detecting individually slow or failing ranks and inconsistent data among replicas. Finally I provide an outlook into how asynchronous teams using enclave tasking can be combined into an advanced team-based diffusive load balancing scheme. Both concepts are integrated into and contribute towards the ExaHyPE project, a next generation code that solves hyperbolic equation systems on dynamically adaptive cartesian grids

A statistical mechanics approach for an effective, scalable, and reliable distributed load balancing scheme for grid networks

The advances in computer and networking technologies over the past decades produced new type of collaborative computing environment called Grid Networks. Grid network is a parallel and distributed computing network system that possesses the ability to achieve a higher computing throughput by taking advantage of many computing resources available in the network. To achieve a scalable and reliable Grid network system, the workload needs to be efficiently distributed among the resources accessible on the network. A novel distributed algorithm based on statistical mechanics that provides an efficient load-balancing paradigm without any centralised monitoring is proposed here. The resulting load-balancer would be integrated into Grid network to increase its efficiency and resources utilisation. This distributed and scalable load-balancing framework is conducted using the biased random sampling (BRS) algorithm. In this thesis, a novel statistical mechanics approach that gives a distributed loadbalancing scheme by generating almost regular networks is proposed. The generated network system is self-organised and depends only on local information for load distribution and resource discovery. The in-degree of each node refers to its free resources, and job assignment and resource updating processes required for load balancing are accomplished by using random sampling (RS). An analytical solution for the stationary degree distributions has been derived that confirms that the edge distribution of the proposed network system is compatible with ER random networks. Therefore, the generated network system can provide an effective loadbalancing paradigm for the distributed resources accessible on large-scale network 1 systems. Furthermore, it has been demonstrated that introducing a geographic awareness factor in the random walk sampling can reduce the effects of communication latency in the Grid network environment. Theoretical and simulation results prove that the proposed BRS load-balancing scheme provides an effective, scalable, and reliable distributed load-balancing scheme for the distributed resources available on Grid networks

Adaptive load balancing for HPC applications

One of the critical factors that affect the performance of many applications is load imbalance. Applications are increasingly becoming sophisticated and are using irregular structures and adaptive refinement techniques, resulting in load imbalance. Moreover, systems are becoming more complex. The number of cores per node is increasing substantially and nodes are becoming heterogeneous. High variability in the performance of the hardware components introduces further imbalance. Load imbalance leads to drop in system utilization and degrades the performance. To address the load imbalance problem, many HPC applications employ dynamic load balancing algorithms to redistribute the work and balance the load. Therefore, performing load balancing is necessary to achieve high performance. Different application characteristics warrant different load balancing strategies. We need a variety of high-quality, scalable load balancing algorithms to cater to different applications. However, using an appropriate load balancer is insufficient to achieve good performance because performing load balancing incurs a cost. Moreover, due to the dynamic nature of the application, it is hard to decide when to perform load balancing. Therefore, deciding when to load balance and which strategy to use for load balancing may not be possible a priori. With the ever increasing core counts on a node, there will be a vast amount of on-node parallelism. Due to the massive on-node parallelism, load imbalance occurring at the node level can be mitigated within the node instead of performing a global load balancing. However, having the application developer manage resources and handle dynamic imbalances is inefficient as well as is a burden on the programmer. The focus of this dissertation is on developing scalable and adaptive techniques for handling load imbalance. The dissertation presents different load balancing algorithms for handling inter and intra-node load imbalance. It also presents an introspective run-time system, which will monitor the application and system characteristics and make load balancing decisions automatically

Numerical modelling of local scour with computational methods

Evaluating bed morphological evolution (specifically the scoured bed level) accurately using computational modelling is critical for analyses of the stability of many marine and coastal structures, such as piers, groynes, breakwaters, submarine pipelines and even telecommunication cables. This thesis considers the coupled hydrodynamic and morphodynamic modelling of the local scour around hydraulic structures, such as near a vertical pile or near a horizontal pipe. The focus in this study is on applying a fluid-structure interaction (FSI) approach to simulate the morphodynamical behaviour of the bed deformation, replacing the structural (i.e. solid mechanics) equation by the sediment continuity equation or Exner equation. Specifically, this works presents a novel method of mesh movement with anisotropic mesh adaptivity based on optimization for simulating local scour near structures with discontinuous Garlerkin (DG) discretisation methods for solving the flow field. Amongst the other goals of this work is the validation of the proposed procedure with previously performed laboratory as well as two- and three-dimensional numerical experiments. Additionally, performance is considered using an implementation of the methodology within Fluidity (http://fluidityproject.github.io/), an open-source, multi-physics, computational fluid dynamics (CFD) code, capable of handling arbitrary multi-scale unstructured tetrahedral meshes and including algorithms to perform dynamic anisotropic mesh adaptivity and mesh movement. The flexibility over mesh structure and resolution that these optimisation capabilities provide makes it potentially highly suitable for accounting the extreme bed morphological evolution close to a fixed solid structure under the action of hydrodynamics. Galerkin-based finite element methods have been used for the hydrodynamics (including discontinuous Galerkin discretisations) and morphological calculations, and automatic mesh deformation has been utilised to account for bed evolution changes while preserving the validity and quality of the mesh. Finally, the work extends the scope in regards of computational methods and considers scour modelling with pure Lagrangian and meshless methods such as smoothed particle hydrodynamics (SPH), which have also become of interest in the analysis and modelling of coastal sediment transport, particularly in scour-related processes. The SPH modelling is considered in a two-phase, flow-sediment fully Lagrangian scour simulation where the discrete-particle interaction forces between phases are resolved at the interface and continuous changes in the bed profile are obtained naturally.Open Acces

HPCCP/CAS Workshop Proceedings 1998

This publication is a collection of extended abstracts of presentations given at the HPCCP/CAS (High Performance Computing and Communications Program/Computational Aerosciences Project) Workshop held on August 24-26, 1998, at NASA Ames Research Center, Moffett Field, California. The objective of the Workshop was to bring together the aerospace high performance computing community, consisting of airframe and propulsion companies, independent software vendors, university researchers, and government scientists and engineers. The Workshop was sponsored by the HPCCP Office at NASA Ames Research Center. The Workshop consisted of over 40 presentations, including an overview of NASA's High Performance Computing and Communications Program and the Computational Aerosciences Project; ten sessions of papers representative of the high performance computing research conducted within the Program by the aerospace industry, academia, NASA, and other government laboratories; two panel sessions; and a special presentation by Mr. James Bailey

Advances in Grid Computing

This book approaches the grid computing with a perspective on the latest achievements in the field, providing an insight into the current research trends and advances, and presenting a large range of innovative research papers. The topics covered in this book include resource and data management, grid architectures and development, and grid-enabled applications. New ideas employing heuristic methods from swarm intelligence or genetic algorithm and quantum encryption are considered in order to explain two main aspects of grid computing: resource management and data management. The book addresses also some aspects of grid computing that regard architecture and development, and includes a diverse range of applications for grid computing, including possible human grid computing system, simulation of the fusion reaction, ubiquitous healthcare service provisioning and complex water systems

HIGH PERFORMANCE DECENTRALISED COMMUNITY DETECTION ALGORITHMS FOR BIG DATA FROM SMART COMMUNICATION APPLICATIONS

Many systems in the world can be represented as models of complex networks and subsequently be analysed fruitfully. One fundamental property of the real-world networks is that they usually exhibit inhomogeneity in which the network tends to organise according to an underlying modular structure, commonly referred to as community structure or clustering. Analysing such communities in large networks can help people better understand the structural makeup of the networks. For example, it can be used in mobile ad-hoc and sensor networks to improve the energy consumption and communication tasks. Thus, community detection in networks has become an important research area within many application fields such as computer science, physical sciences, mathematics and biology. Driven by the recent emergence of big data, clustering of real-world networks using traditional methods and algorithms is almost impossible to be processed in a single machine. The existing methods are limited by their computational requirements and most of them cannot be directly parallelised. Furthermore, in many cases the data set is very big and does not fit into the main memory of a single machine, therefore needs to be distributed among several machines. The main topic of this thesis is about network community detection within these big data networks. More specifically, in this thesis, a novel approach, namely Decentralized Iterative Community Clustering Approach (DICCA) for clustering large and undirected networks is introduced. An important property of this approach is its ability to cluster the entire network without the global knowledge of the network topology. Moreover, an extension of the DICCA called Parallel Decentralized Iterative Community Clustering approach (PDICCA) is proposed for efficiently processing data distributed across several machines. PDICCA is based on MapReduce computing platform to work efficiently in distributed and parallel fashion. In addition, the real-world networks are usually noisy and imperfect with missing and false edges. These imperfections are often difficult to eliminate and highly affect the quality and accuracy of conventional methods used to find the community structure in the network. However, in real-world networks, node attribute information is also available in addition to topology information. Considering more than one source of information for community detection could produce meaningful clusters and improve the robustness of the network. Therefore, a pre-processing approach that considers attribute information, shared neighbours and connectivity information aspects of the network for community detection is presented in this thesis as part of my research. Finally, a set of real-world mobile phone usage data obtained from Cambridge Laboratories (Device Analyzer) has been analysed as an exploratory step for viability to apply the algorithms developed in this thesis. All the proposed approaches have been evaluated and verified for feasibility using real-world large data set. The evaluation results of these experimentations prove very promising for the type of large data networks considered