SQPR: Stream Query Planning with Reuse

When users submit new queries to a distributed stream processing system (DSPS), a query planner must allocate physical resources, such as CPU cores, memory and network bandwidth, from a set of hosts to queries. Allocation decisions must provide the correct mix of resources required by queries, while achieving an efficient overall allocation to scale in the number of admitted queries. By exploiting overlap between queries and reusing partial results, a query planner can conserve resources but has to carry out more complex planning decisions. In this paper, we describe SQPR, a query planner that targets DSPSs in data centre environments with heterogeneous resources. SQPR models query admission, allocation and reuse as a single constrained optimisation problem and solves an approximate version to achieve scalability. It prevents individual resources from becoming bottlenecks by re-planning past allocation decisions and supports different allocation objectives. As our experimental evaluation in comparison with a state-of-the-art planner shows SQPR makes efficient resource allocation decisions, even with a high utilisation of resources, with acceptable overheads

CloudScope: diagnosing and managing performance interference in multi-tenant clouds

© 2015 IEEE.Virtual machine consolidation is attractive in cloud computing platforms for several reasons including reduced infrastructure costs, lower energy consumption and ease of management. However, the interference between co-resident workloads caused by virtualization can violate the service level objectives (SLOs) that the cloud platform guarantees. Existing solutions to minimize interference between virtual machines (VMs) are mostly based on comprehensive micro-benchmarks or online training which makes them computationally intensive. In this paper, we present CloudScope, a system for diagnosing interference for multi-tenant cloud systems in a lightweight way. CloudScope employs a discrete-time Markov Chain model for the online prediction of performance interference of co-resident VMs. It uses the results to optimally (re)assign VMs to physical machines and to optimize the hypervisor configuration, e.g. the CPU share it can use, for different workloads. We have implemented CloudScope on top of the Xen hypervisor and conducted experiments using a set of CPU, disk, and network intensive workloads and a real system (MapReduce). Our results show that CloudScope interference prediction achieves an average error of 9%. The interference-aware scheduler improves VM performance by up to 10% compared to the default scheduler. In addition, the hypervisor reconfiguration can improve network throughput by up to 30%

SwiftAnalytics: optimizing object stores for big data analytics

Due to their scalability and low cost, object-based storage systems are an attractive storage solution and widely deployed. To gain valuable insight from the data residing in object storage but avoid expensive copying to a distributed filesystem (e.g. HDFS), it would be natural to directly use them as a storage backend for data-parallel analytics frameworks such as Spark or MapReduce. Unfortunately, executing data-parallel frameworks on object storage exhibits severe performance prob- lems, reducing average job completion times by up to 6.5 × . We identify the two most severe performance problems when running data-parallel frameworks on the OpenStack Swift object storage system in comparison to the HDFS distributed filesystem: (i) the fixed mapping of object names to storage nodes prevents local writes and adds delay when objects are renamed; (ii) the coarser granularity of objects compared to blocks reduces data locality during reads. We propose the SwiftAnalytics object storage system to address them: (i) it uses locality-aware writes to control an object’s location and eliminate unnecessary I/O related to renames during job completion, speeding up analytics jobs by up to 5.1 × ; (ii) it transparently chunks objects into smaller sized parts to improve data-locality, leading to up to 3.4 × faster reads

Accelerating Publish/Subscribe Matching on Reconfigurable Supercomputing Platform

Abstract—A modular design is proposed and analyzed for accelerating the publish/subscribe matching algorithm in reconfigurable hardware. With help from a performance model, we demonstrate an optimized FPGA implementation which is scalable and efficient enough for many of today’s most demanding web and financial applications. Our design achieves 5.9 times speedup over software while consuming around 0.5 % of power. I

Meta-dataflows: efficient exploratory dataflow jobs

Distributed dataflow systems such as Apache Spark and Apache Flink are used to derive new insights from large datasets. While they efficiently execute concrete data processing workflows, expressed as dataflow graphs, they lack generic support for exploratory work- flows : if a user is uncertain about the correct processing pipeline, e.g. in terms of data cleaning strategy or choice of model parame- ters, they must repeatedly submit modified jobs to the system. This, however, misses out on optimisation opportunities for exploratory workflows, both in terms of scheduling and memory allocation. We describe meta-dataflows (MDFs), a new model to effectively express exploratory workflows and efficiently execute them on compute clusters. With MDFs, users specify a family of dataflows using two primitives: (a) an explore operator automatically con- siders choices in a dataflow; and (b) a choose operator assesses the result quality of explored dataflow branches and selects a subset of the results. We propose optimisations to execute MDFs: a system can (i) avoid redundant computation when exploring branches by reusing intermediate results and discarding results from underper- forming branches; and (ii) consider future data access patterns in the MDF when allocating cluster memory. Our evaluation shows that MDFs improve the runtime of exploratory workflows by up to 90% compared to sequential execution

Magnetism and Structural Chemistry of the n = 1 Ruddlesden-Popper Phases La₄LiMnO₈ and La₃SrLiMnO₈

Polycrystalline samples of La4LiMnO8 and La3SrLiMnO8 have been studied by a combination of X-ray diffraction (XRD), neutron diffraction (ND), 6Li MAS NMR, electron microscopy (EM), and magnetometry. Room-temperature XRD and ND measurements suggest that both compounds have the K2NiF4 structure, with a disordered arrangement of Li and Mn over the six-coordinate sites. However, MAS NMR and EM demonstrate the presence of local 1:1 Li:Mn order on these sites, and EM shows that although cation order is well-developed in each xy sheet of corner-sharing octahedra, the sheets are stacked randomly along z. The structures are best described as paracrystalline, and many of the concepts of conventional crystallography are inapplicable. Magnetometry and low-temperature ND experiments show that, despite their paracrystallinity, the two compounds are ordered antiferromagnetically with susceptibility maxima at 26 and 18 K, respectively, and with ordered magnetic moments of 3.61(6) and 2.3(1) muB per Mn cation at 2 K. Anisotropic peak broadening reveals a 2D character in the magnetic behavior of both compounds, and La3SrLiMnO8 is well-modeled as a quadratic layer S = 3/2 Heisenberg antiferromagnet