Garbage collection auto-tuning for Java MapReduce on Multi-Cores

MapReduce has been widely accepted as a simple programming pattern that can form the basis for efficient, large-scale, distributed data processing. The success of the MapReduce pattern has led to a variety of implementations for different computational scenarios. In this paper we present MRJ, a MapReduce Java framework for multi-core architectures. We evaluate its scalability on a four-core, hyperthreaded Intel Core i7 processor, using a set of standard MapReduce benchmarks. We investigate the significant impact that Java runtime garbage collection has on the performance and scalability of MRJ. We propose the use of memory management auto-tuning techniques based on machine learning. With our auto-tuning approach, we are able to achieve MRJ performance within 10% of optimal on 75% of our benchmark tests

LEGaTO: first steps towards energy-efficient toolset for heterogeneous computing

LEGaTO is a three-year EU H2020 project which started in December 2017. The LEGaTO project will leverage task-based programming models to provide a software ecosystem for Made-in-Europe heterogeneous hardware composed of CPUs, GPUs, FPGAs and dataflow engines. The aim is to attain one order of magnitude energy savings from the edge to the converged cloud/HPC.Peer ReviewedPostprint (author's final draft

Reusability in software engineering

This paper surveys recent work concerning reusability in software engineering. The current directions in software reusability are discussed, and the two major approaches of reusable building blocks and reusable patterns studied. An extensive bibliography, parts of which are annotated, is included

A Taxonomy of Workflow Management Systems for Grid Computing

With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. We also survey several representative Grid workflow systems developed by various projects world-wide to demonstrate the comprehensiveness of the taxonomy. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.Comment: 29 pages, 15 figure

Julia Programming Language Benchmark Using a Flight Simulation

Julia is a relatively new computer language that aims to reduce the challenge for mathmodelers to develop fast computer tools and simulations. It potentially combines the ease-of-coding feature of scripting languages (like Python) with the performance of compiled languages (like C++). A key question for Julia application to the simulation domain is, Can Julia, with its obvious coding simplicity, provide runtime speeds comparable to conventional compiled languages for flight simulation? A unique combination of existing elements can be employed to address the previous question: Extensively documented object-oriented simulation architecture, Industry standard rocket flight simulation, Separate versions (C++, Java, and Python) already benchmarked

Computationally-intensive Econometrics using a Distributed Matrix-programming Language

This paper reviews the need for powerful facilities in econometrics, focusing on concrete problems which arise in financial economics and in macroeconomics. We argue that the profession is being held back by the lack of easy to use generic software which is able to exploit the availability of cheap clusters of distributed computers. Our response is to extend, in a number of directions, the well known matrix-programming interpreted language Ox developed by the first author. We note three possible levels of extensions: (i) Ox with parallelization explicit in the Ox code; (ii) Ox with a parallelized run-time library; (iii) Ox with a parallelized interpreter. This paper studies and implements the first case, emphasizing the need for deterministic computing in science. We give examples in the context of financial economics and time-series modelling.Distributed computing; Econometrics; High-performance computing; Matrix-programming language