4,790 research outputs found
Tupleware: Redefining Modern Analytics
There is a fundamental discrepancy between the targeted and actual users of
current analytics frameworks. Most systems are designed for the data and
infrastructure of the Googles and Facebooks of the world---petabytes of data
distributed across large cloud deployments consisting of thousands of cheap
commodity machines. Yet, the vast majority of users operate clusters ranging
from a few to a few dozen nodes, analyze relatively small datasets of up to a
few terabytes, and perform primarily compute-intensive operations. Targeting
these users fundamentally changes the way we should build analytics systems.
This paper describes the design of Tupleware, a new system specifically aimed
at the challenges faced by the typical user. Tupleware's architecture brings
together ideas from the database, compiler, and programming languages
communities to create a powerful end-to-end solution for data analysis. We
propose novel techniques that consider the data, computations, and hardware
together to achieve maximum performance on a case-by-case basis. Our
experimental evaluation quantifies the impact of our novel techniques and shows
orders of magnitude performance improvement over alternative systems
On the Fly Orchestration of Unikernels: Tuning and Performance Evaluation of Virtual Infrastructure Managers
Network operators are facing significant challenges meeting the demand for
more bandwidth, agile infrastructures, innovative services, while keeping costs
low. Network Functions Virtualization (NFV) and Cloud Computing are emerging as
key trends of 5G network architectures, providing flexibility, fast
instantiation times, support of Commercial Off The Shelf hardware and
significant cost savings. NFV leverages Cloud Computing principles to move the
data-plane network functions from expensive, closed and proprietary hardware to
the so-called Virtual Network Functions (VNFs). In this paper we deal with the
management of virtual computing resources (Unikernels) for the execution of
VNFs. This functionality is performed by the Virtual Infrastructure Manager
(VIM) in the NFV MANagement and Orchestration (MANO) reference architecture. We
discuss the instantiation process of virtual resources and propose a generic
reference model, starting from the analysis of three open source VIMs, namely
OpenStack, Nomad and OpenVIM. We improve the aforementioned VIMs introducing
the support for special-purpose Unikernels and aiming at reducing the duration
of the instantiation process. We evaluate some performance aspects of the VIMs,
considering both stock and tuned versions. The VIM extensions and performance
evaluation tools are available under a liberal open source licence
Recommended from our members
Automation of Determination of Optimal Intra-Compute Node Parallelism
Maximizing the productivity of modern multicore and manycore chips requires optimizing parallelism at the compute node level. This is, however, a complex multi-step process. It is an iterative method requiring determining optimal degrees of parallel scalability and optimizing memory access behavior. Further, there are multiple cases to be considered, programs which use only MPI or OpenMP and hybrid (MPI +OpenMP) programs. This paper presents a set of three coordinated workflows for determining the optimal parallelism at the program level for MPI programs and at the loop level for hybrid (MPI+OpenMP) cases. The paper also details mostly automated implementations of these workflows using the PerfExpert infrastructure. Finally the paper presents case studies demonstrating both the applicability and the effectiveness of optimizing parallelism at the compute node level. The results shown in the paper will provide valuable information to further advance in the full automation of the workflows. The software implementing the parallelism scalability optimization is open source and available for download.Texas Advanced Computing Center (TACC)Computer Science
- …