124,606 research outputs found

    Accelerating Large-Scale Data Analysis by Offloading to High-Performance Computing Libraries using Alchemist

    Full text link
    Apache Spark is a popular system aimed at the analysis of large data sets, but recent studies have shown that certain computations---in particular, many linear algebra computations that are the basis for solving common machine learning problems---are significantly slower in Spark than when done using libraries written in a high-performance computing framework such as the Message-Passing Interface (MPI). To remedy this, we introduce Alchemist, a system designed to call MPI-based libraries from Apache Spark. Using Alchemist with Spark helps accelerate linear algebra, machine learning, and related computations, while still retaining the benefits of working within the Spark environment. We discuss the motivation behind the development of Alchemist, and we provide a brief overview of its design and implementation. We also compare the performances of pure Spark implementations with those of Spark implementations that leverage MPI-based codes via Alchemist. To do so, we use data science case studies: a large-scale application of the conjugate gradient method to solve very large linear systems arising in a speech classification problem, where we see an improvement of an order of magnitude; and the truncated singular value decomposition (SVD) of a 400GB three-dimensional ocean temperature data set, where we see a speedup of up to 7.9x. We also illustrate that the truncated SVD computation is easily scalable to terabyte-sized data by applying it to data sets of sizes up to 17.6TB.Comment: Accepted for publication in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, London, UK, 201

    The long road from Ljubljana to Kyoto: Implementing emissions trading mechanism and CO2 tax

    Get PDF
    According to the Kyoto Protocol, Slovenia is required to reduce GHG emissions to an average of 8% below base year 1986 emissions in the period 2008-2012. Slovenia established different measures for reducing GHG emissions long before its ratification. It was first transition country who implemented CO2 tax in the 1997. Several changes in CO2 tax have not brought the desired results. CO2 emissions have actually increased. At the beginning of 2005, Slovenia joined other EU member states by implementing the emissions trading instrument, defined by new EU Directive. At the same time, Slovenia has adopted a new CO2 tax system, which is compatible with the new circumstances. The main purpose of this paper is to present the characteristics of Slovenian approach to national allocation plan for emissions trading and analyze the problems of the CO2 tax in Slovenia. Paper also describes the compliance cost of achieving the Kyoto target and expected movements on the Slovenian allowances market.CO2 tax, Kyoto Protocol, emissions trading, national allocation plan, emissions allowances
    • …
    corecore