Evaluating technologies and techniques for transitioning hydrodynamics applications to future generations of supercomputers

Current supercomputer development trends present severe challenges for scientific codebases. Moore’s law continues to hold, however, power constraints have brought an end to Dennard scaling, forcing significant increases in overall concurrency. The performance imbalance between the processor and memory sub-systems is also increasing and architectures are becoming significantly more complex. Scientific computing centres need to harness more computational resources in order to facilitate new scientific insights and maintaining their codebases requires significant investments. Centres therefore have to decide how best to develop their applications to take advantage of future architectures. To prevent vendor "lock-in" and maximise investments, achieving portableperformance across multiple architectures is also a significant concern. Efficiently scaling applications will be essential for achieving improvements in science and the MPI (Message Passing Interface) only model is reaching its scalability limits. Hybrid approaches which utilise shared memory programming models are a promising approach for improving scalability. Additionally PGAS (Partitioned Global Address Space) models have the potential to address productivity and scalability concerns. Furthermore, OpenCL has been developed with the aim of enabling applications to achieve portable-performance across a range of heterogeneous architectures. This research examines approaches for achieving greater levels of performance for hydrodynamics applications on future supercomputer architectures. The development of a Lagrangian-Eulerian hydrodynamics application is presented together with its utility for conducting such research. Strategies for improving application performance, including PGAS- and hybrid-based approaches are evaluated at large node-counts on several state-of-the-art architectures. Techniques to maximise the performance and scalability of OpenMP-based hybrid implementations are presented together with an assessment of how these constructs should be combined with existing approaches. OpenCL is evaluated as an additional technology for implementing a hybrid programming model and improving performance-portability. To enhance productivity several tools for automatically hybridising applications and improving process-to-topology mappings are evaluated. Power constraints are starting to limit supercomputer deployments, potentially necessitating the use of more energy efficient technologies. Advanced processor architectures are therefore evaluated as future candidate technologies, together with several application optimisations which will likely be necessary. An FPGA-based solution is examined, including an analysis of how effectively it can be utilised via a high-level programming model, as an alternative to the specialist approaches which currently limit the applicability of this technology

Observing the epoch of reionization and dark ages with redshifted 21-cm hydrogen line

The billion years subsequent to the Big Bang pose the next challenging frontier for precision cosmology. The concordant cosmological model, ΔCDM, propounds that during this period, the dark matter gravitationally shepherds the baryonic matter to form the primordial large-scale structures. This era is termed the Dark Ages (DA). The following era, the Epoch of Reionization (EoR), leads to the formation of the first stars and galaxies that reionize the permeating neutral hydrogen. The linear polarization of the cosmic background radiation and the Gunn-Peterson troughs in quasar absorption spectra provide indirect evidence for the EoR. Currently, there is no observational evidence for the DA. While state-of-the-art radio telescope arrays, Low Frequency Array (LOFAR) and Square Kilometre Array (SKA), propose various strategies to observe the early phases of the Universe, the advanced simulations employing high-performance computing (HPC) methodologies continue to play significant role in constraining various models based upon limited observational data. Despite a wide range of research, there is no end-to-end simulation solution available to quantifiably address the observational challenges due to statistical and systematic errors including foregrounds, ionosphere, polarization, RFI, instrument stability, and directional dependent gains. This research consolidates the cutting-edge simulation solutions, Cube-P3M, C2-Ray, and MeqTrees, to build an HPC prototype pipeline entitled, Simulating Interferometry Measurements (SIM). To establish and validate the efficacy of the SIM pipeline, the research builds a theoretical framework of two science drivers, viz., the presence of Lymanlimit absorbers and measuring non-Gaussianity from the 21-cm data. Thereafter, using the LOFAR and SKA telescope configurations, the SIM generates data visibility cubes with direction dependent and independent propagation effects. Finally, SIM extracts the original signal through standard techniques exploring the parametric phase-space. Results are presented herein

The readying of applications for heterogeneous computing

High performance computing is approaching a potentially significant change in architectural design. With pressures on the cost and sheer amount of power, additional architectural features are emerging which require a re-think to the programming models deployed over the last two decades. Today's emerging high performance computing (HPC) systems are maximising performance per unit of power consumed resulting in the constituent parts of the system to be made up of a range of different specialised building blocks, each with their own purpose. This heterogeneity is not just limited to the hardware components but also in the mechanisms that exploit the hardware components. These multiple levels of parallelism, instruction sets and memory hierarchies, result in truly heterogeneous computing in all aspects of the global system. These emerging architectural solutions will require the software to exploit tremendous amounts of on-node parallelism and indeed programming models to address this are emerging. In theory, the application developer can design new software using these models to exploit emerging low power architectures. However, in practice, real industrial scale applications last the lifetimes of many architectural generations and therefore require a migration path to these next generation supercomputing platforms. Identifying that migration path is non-trivial: With applications spanning many decades, consisting of many millions of lines of code and multiple scientific algorithms, any changes to the programming model will be extensive and invasive and may turn out to be the incorrect model for the application in question. This makes exploration of these emerging architectures and programming models using the applications themselves problematic. Additionally, the source code of many industrial applications is not available either due to commercial or security sensitivity constraints. This thesis highlights this problem by assessing current and emerging hard- ware with an industrial strength code, and demonstrating those issues described. In turn it looks at the methodology of using proxy applications in place of real industry applications, to assess their suitability on the next generation of low power HPC offerings. It shows there are significant benefits to be realised in using proxy applications, in that fundamental issues inhibiting exploration of a particular architecture are easier to identify and hence address. Evaluations of the maturity and performance portability are explored for a number of alternative programming methodologies, on a number of architectures and highlighting the broader adoption of these proxy applications, both within the authors own organisation, and across the industry as a whole

DOOM: SCARYDARKFAST

A close examination about what is considered the most important first-person video game ever made and its influence on how we play games toda

Designing for mod development: user creativity as product development strategy on the firm-hosted 3D software platform

The thesis is designed to improve our understanding of user participation in Web-based development practices in the commercial setting of the 3D software industry. It aims to investigate whether the creative capacities of users and their contributions to the online firm-hosted 3D platform are indicative of a novel configuration of production that influences the processes of product development across firm boundaries. The thesis mobilizes the user participation literature developing in media research as its main theoretical framework. It builds on insights derived from work on user participation in media sites as seen through a cultural lens, in particular, as developed in Henry Jenkins' notions of 'participatory' and 'convergence culture'. The user participation literature is supported by a combination of insights drawn from work on communities of practice and user-centred innovation so as to offer a more robust approach to examine and appreciate the firm-hosted 3D platform as a site of user participation. More specifically, the conceptual framework for the study provides a basis for an examination of the ways a software developer finn encourages user participation in a market and of how this enables and facilitâtes particular modes of user creativity. These are shown to shape and maintain a firm-hosted platform that aids product development efforts that are expected to benefit the developer fimi. An empirical study of the platform, Second Life, provides the basis for the analysis of finn-user interactions which are shown to underpin a distinctive finn leaming process in the context of product development that occurs across permeable fimi boundaries. The thesis yields insight into the way a developer firm invites its user base to partner with it in product development, indicating how aspects of user participation associated with non-market dynamics are embedded in commercial activity and professionalism. The pivotal role of users is revealed in the design, development and sustainability of a firm-hosted 3D product. The findings point to interesting relationships between the distinctive creative capacities of users and the range of capabilities afforded by the firm-provided design space. Variations in user participation and contributions to product development suggest that particular patterns of learning opportunities occur. The analysis yields several new concepts including a 'modification effect market' which are used to extend existing conceptualizations of user participation in digitai development practices in the commercial setting of the 3D software industry

Ray-traced radiative transfer on massively threaded architectures

In this thesis, I apply techniques from the field of computer graphics to ray tracing in astrophysical simulations, and introduce the grace software library. This is combined with an extant radiative transfer solver to produce a new package, taranis. It allows for fully-parallel particle updates via per-particle accumulation of rates, followed by a forward Euler integration step, and is manifestly photon-conserving. To my knowledge, taranis is the first ray-traced radiative transfer code to run on graphics processing units and target cosmological-scale smooth particle hydrodynamics (SPH) datasets. A significant optimization effort is undertaken in developing grace. Contrary to typical results in computer graphics, it is found that the bounding volume hierarchies (BVHs) used to accelerate the ray tracing procedure need not be of high quality; as a result, extremely fast BVH construction times are possible (< 0.02 microseconds per particle in an SPH dataset). I show that this exceeds the performance researchers might expect from CPU codes by at least an order of magnitude, and compares favourably to a state-of-the-art ray tracing solution. Similar results are found for the ray-tracing itself, where again techniques from computer graphics are examined for effectiveness with SPH datasets, and new optimizations proposed. For high per-source ray counts (≳ 104), grace can reduce ray tracing run times by up to two orders of magnitude compared to extant CPU solutions developed within the astrophysics community, and by a factor of a few compared to a state-of-the-art solution. taranis is shown to produce expected results in a suite of de facto cosmological radiative transfer tests cases. For some cases, it currently out-performs a serial, CPU-based alternative by a factor of a few. Unfortunately, for the most realistic test its performance is extremely poor, making the current taranis code unsuitable for cosmological radiative transfer. The primary reason for this failing is found to be a small minority of particles which always dominate the timestep criteria. Several plausible routes to mitigate this problem, while retaining parallelism, are put forward

Games as Services - Final Report

The ultra competitive nature of global game industry, characterized by spiraling production times and development costs, has in the recent times forced the game developers to search for alternative approaches. As a consequence, digital distribution systems, subscription-based models and micro-transactions have challenged the traditional circuits of game development, play and distribution. A common theme across the transformations ranging from persistent game worlds and casual games to automatic content updates and player-created content is that they make games, more or less, available "as services". The need for a particular research project examining the rise of the service paradigm among game industry was identified already in relation the Neogames centre surveys conducted in 2007. The Finnish game industry representatives were widely aware of the change towards service-driven models but the individual game studios had no resources for a larger analysis. The practical objectives of the Games as Services (GaS) project are motivated from this background. The project, conducted in collaboration between the University of Tampere Department of Information Studies and Interactive Media (INFIM) and The Helsinki Institute for Information Technology (HIIT), has aimed at producing an overview on the nature of the service paradigm and its consequences to games, game culture and business. In addition, individual case studies have been conducted in order to shed light on particularly interesting subthemes. During the project's timeframe (2008-10), the core themes of the GaS have only increased in relevance and visibility. The reasons for this are twofold. Firstly, the online games market of games, including subscription, digital game download, DLC, virtual commodities and value-added services is steadily expanding (PwC 2009). Secondly, with the advent of casual and social games, entirely new audiences have been introduced to digital games. Players are not so much asked to structure their lives to fit the demands of a game (Juul 2009). Instead, the games are increasingly designed to serve the players and to fit into their lives. In the age of "contextual gaming", play is increasingly tied to the practices and rhythms of everyday life and playful behaviors are often rooted in social relations and exchanges of information that are used to maintain and expand the networks of relationships (Mäyrä 2008). Many signs indicate that the days of digital games packaged as "fire and forget" commodities are numbered. The global game industry is actively moving from providing discrete offerings towards establishing ongoing relationships with players (Chang 2010, 24). Not all games will be based on a constant update model, but even the more traditional releases will be transformed after the launch by patches, upgrades, expansions and modifications. The development budgets are forecast to reflect this change, as significant investments are moved from the launch to operating the ongoing service. The transitions described are not entirely unique to the games market but similar developments can be identified in other media markets. It is, however, noticeable how easily and successfully many sectors of gaming have already shifted towards a service-based economy. In this respect, all the traditional media industries have a lesson or two to learn from games as services (ibid.)

Profiling and optimizing K-means algorithms in a beowulf cluster environment

The K-means algorithm is a well known statistical agglomeration algorithm used to sort a database of unlabeled items into K groups. As part of the fitness function of an Evolutionary Algorithm (EA), the optimization of the K-means algorithm has become a point of great interest. Although many approaches have been proposed for its parallelization and optimization, very few address the question of scalability and efficiency. In most cases, the description of the execution environment remains opaque and precise profiles of the program are mostly absent. Performance and efficiency issues are quickly relegated to communicafion issues. We address these deficiencies by presenting a detailed description of two parallel environments, the Beowulf style clusters and the Symmetric Multi-Processors (SMP) parallel machines. A mixture of theoretical and empirical models were used to characterize these environments and set baseline expectations pertaining to the K-means algorithm. Due to the necessity of a multidisciplinary expertise, a detailed use of Tuning and Analysis Utilities (TAU) is provided to ease the parallel performance profiling task. Coupled with the high precision counter interface provided by Performance Application Programming Interface (PAPI), we present a grey box method by which a parallel master-slave implementation of the K-means is evolved into a highly efficient island version of itself. Communications and computational optimization were guided by prior theoretical and empirical models of the parallel execution environment. Our work has revealed that there is much more to parallel processing than the simple balance between computation and communications. We have brought forth the negative impact of using mathematical libraries for specific problems and identified performance issues specific to some versions of the same series of Message Passing Inerface (MPI) libraries. High precision profiling has shown that data representation and processing can be a more significant source of scalability bottleneck than computation and communications put together

Project knole: an autocosmic approach to authoring resonant computational characters

Project knole, consisting of this thesis and a mixed reality installation artwork centred around a computational simulation, is a practice-based response to the question of how a character in a work of computational narrative art might maintain their defining quality of dynamic agency within a system (arguably one of the key potentials of the form), while achieving the ‘resonant’ qualities of characters in more materially-static artforms. In all aspects of this project, I explore a new design philosophy for achieving this balance; between the authorship of a procedural computational system, and the ability of that system to ‘resonate’ with the imagination of an audience. This philosophy, which I term the ‘autocosmic’, seeks inspiration for the curation of audience response outside the obvious boundaries of artistic discipline, across the wider spectrum of human imaginative engagement; examples often drawn from mostly non-aesthetic domains. As well as defining the terms ‘resonance’ and ‘autocosmic’, and delineating my methodology more generally, this thesis demonstrates how the ‘autocosmic’ was employed within my creative work. In particular, it shows how some of the perennial problems of computational character development might be mediated by exploring other non-aesthetic examples of imaginative, narrative engagement with personified systems. In the context of this project, such examples come from the historio-cultural relationship between human beings and the environments they inhabit, outside of formal artistic practice. From this ‘autocosmic’ launchpad, I have developed an artwork that starts to explore how this rich cultural and biological lineage of human social engagement with systemic place can be applied fruitfully to the development of a ‘resonant’ computational character

Warez

When most people think of piracy, they think of Bittorrent and The Pirate Bay. These public manifestations of piracy, though, conceal an elite worldwide, underground, organized network of pirate groups who specialize in obtaining media – music, videos, games, and software – before their official sale date and then racing against one another to release the material for free. Warez: The Infrastructure and Aesthetics of Piracy is the first scholarly research book about this underground subculture, which began life in the pre-internet era Bulletin Board Systems and moved to internet File Transfer Protocol servers (“topsites”) in the mid- to late-1990s. The “Scene,” as it is known, is highly illegal in almost every aspect of its operations. The term “Warez” itself refers to pirated media, a derivative of “software.” Taking a deep dive in the documentary evidence produced by the Scene itself, Warez describes the operations and infrastructures an underground culture with its own norms and rules of participation, its own forms of sociality, and its own artistic forms. Even though forms of digital piracy are often framed within ideological terms of equal access to knowledge and culture, Eve uncovers in the Warez Scene a culture of competitive ranking and one-upmanship that is at odds with the often communalist interpretations of piracy. Broad in scope and novel in its approach, Warez is indispensible reading for anyone interested in recent developments in digital culture, access to knowledge and culture, and the infrastructures that support our digital age