13,638 research outputs found
A Generic Storage API
We present a generic API suitable for provision of highly generic storage
facilities that can be tailored to produce various individually customised
storage infrastructures. The paper identifies a candidate set of minimal
storage system building blocks, which are sufficiently simple to avoid
encapsulating policy where it cannot be customised by applications, and
composable to build highly flexible storage architectures. Four main generic
components are defined: the store, the namer, the caster and the interpreter.
It is hypothesised that these are sufficiently general that they could act as
building blocks for any information storage and retrieval system. The essential
characteristics of each are defined by an interface, which may be implemented
by multiple implementing classes.Comment: Submitted to ACSC 200
CloudJet4BigData: Streamlining Big Data via an Accelerated Socket Interface
Big data needs to feed users with fresh processing results and cloud platforms can be used to speed up big data applications. This paper describes a new data communication protocol (CloudJet) for long distance and large volume big data accessing operations to alleviate the large latencies encountered in sharing big data resources in the clouds. It encapsulates a dynamic multi-stream/multi-path engine at the socket level, which conforms to Portable Operating System Interface (POSIX) and thereby can accelerate any POSIX-compatible applications across IP based networks. It was demonstrated that CloudJet accelerates typical big data applications such as very large database (VLDB), data mining, media streaming and office applications by up to tenfold in real-world tests
The Parallel Persistent Memory Model
We consider a parallel computational model that consists of processors,
each with a fast local ephemeral memory of limited size, and sharing a large
persistent memory. The model allows for each processor to fault with bounded
probability, and possibly restart. On faulting all processor state and local
ephemeral memory are lost, but the persistent memory remains. This model is
motivated by upcoming non-volatile memories that are as fast as existing random
access memory, are accessible at the granularity of cache lines, and have the
capability of surviving power outages. It is further motivated by the
observation that in large parallel systems, failure of processors and their
caches is not unusual.
Within the model we develop a framework for developing locality efficient
parallel algorithms that are resilient to failures. There are several
challenges, including the need to recover from failures, the desire to do this
in an asynchronous setting (i.e., not blocking other processors when one
fails), and the need for synchronization primitives that are robust to
failures. We describe approaches to solve these challenges based on breaking
computations into what we call capsules, which have certain properties, and
developing a work-stealing scheduler that functions properly within the context
of failures. The scheduler guarantees a time bound of in expectation, where and are the work and
depth of the computation (in the absence of failures), is the average
number of processors available during the computation, and is the
probability that a capsule fails. Within the model and using the proposed
methods, we develop efficient algorithms for parallel sorting and other
primitives.Comment: This paper is the full version of a paper at SPAA 2018 with the same
nam
Decision-focussed resource modelling for design decision support
Resource management including resource allocation, levelling, configuration and monitoring has been recognised as critical to design decision making. It has received increasing research interests in recent years. Different definitions, models and systems have been developed and published in literature. One common issue with existing research is that the resource modelling has focussed on the information view of resources. A few acknowledged the importance of resource capability to design management, but none has addressed the evaluation analysis of resource fitness to effectively support design decisions. This paper proposes a decision-focused resource model framework that addresses the combination of resource evaluation with resource information from multiple perspectives. A resource management system constructed on the resource model framework can provide functions for design engineers to efficiently search and retrieve the best fit resources (based on the evaluation results) to meet decision requirements. Thus, the system has the potential to provide improved decision making performance compared with existing resource management systems
- …