17,190 research outputs found
Improving the Performance and Endurance of Persistent Memory with Loose-Ordering Consistency
Persistent memory provides high-performance data persistence at main memory.
Memory writes need to be performed in strict order to satisfy storage
consistency requirements and enable correct recovery from system crashes.
Unfortunately, adhering to such a strict order significantly degrades system
performance and persistent memory endurance. This paper introduces a new
mechanism, Loose-Ordering Consistency (LOC), that satisfies the ordering
requirements at significantly lower performance and endurance loss. LOC
consists of two key techniques. First, Eager Commit eliminates the need to
perform a persistent commit record write within a transaction. We do so by
ensuring that we can determine the status of all committed transactions during
recovery by storing necessary metadata information statically with blocks of
data written to memory. Second, Speculative Persistence relaxes the write
ordering between transactions by allowing writes to be speculatively written to
persistent memory. A speculative write is made visible to software only after
its associated transaction commits. To enable this, our mechanism supports the
tracking of committed transaction ID and multi-versioning in the CPU cache. Our
evaluations show that LOC reduces the average performance overhead of memory
persistence from 66.9% to 34.9% and the memory write traffic overhead from
17.1% to 3.4% on a variety of workloads.Comment: This paper has been accepted by IEEE Transactions on Parallel and
Distributed System
Efficient Logging in Non-Volatile Memory by Exploiting Coherency Protocols
Non-volatile memory (NVM) technologies such as PCM, ReRAM and STT-RAM allow
processors to directly write values to persistent storage at speeds that are
significantly faster than previous durable media such as hard drives or SSDs.
Many applications of NVM are constructed on a logging subsystem, which enables
operations to appear to execute atomically and facilitates recovery from
failures. Writes to NVM, however, pass through a processor's memory system,
which can delay and reorder them and can impair the correctness and cost of
logging algorithms.
Reordering arises because of out-of-order execution in a CPU and the
inter-processor cache coherence protocol. By carefully considering the
properties of these reorderings, this paper develops a logging protocol that
requires only one round trip to non-volatile memory while avoiding expensive
computations. We show how to extend the logging protocol to building a
persistent set (hash map) that also requires only a single round trip to
non-volatile memory for insertion, updating, or deletion
Scalable Reliable SD Erlang Design
This technical report presents the design of Scalable Distributed (SD) Erlang: a set of language-level changes that aims to enable Distributed Erlang to scale for server applications on commodity hardware with at most 100,000 cores. We cover a number of aspects, specifically anticipated architecture, anticipated failures, scalable data structures, and scalable computation. Other two components that guided us in the design of SD Erlang are design principles and typical Erlang applications. The design principles summarise the type of modifications we aim to allow Erlang scalability. Erlang exemplars help us to identify the main Erlang scalability issues and hypothetically validate the SD Erlang design
Real-Time Data Processing With Lambda Architecture
Data has evolved immensely in recent years, in type, volume and velocity. There are several frameworks to handle the big data applications. The project focuses on the Lambda Architecture proposed by Marz and its application to obtain real-time data processing. The architecture is a solution that unites the benefits of the batch and stream processing techniques. Data can be historically processed with high precision and involved algorithms without loss of short-term information, alerts and insights. Lambda Architecture has an ability to serve a wide range of use cases and workloads that withstands hardware and human mistakes. The layered architecture enhances loose coupling and flexibility in the system. This a huge benefit that allows understanding the trade-offs and application of various tools and technologies across the layers. There has been an advancement in the approach of building the LA due to improvements in the underlying tools. The project demonstrates a simplified architecture for the LA that is maintainable
Defining and Verifying Durable Opacity: Correctness for Persistent Software Transactional Memory
Non-volatile memory (NVM), aka persistent memory, is a new paradigm for
memory that preserves its contents even after power loss. The expected ubiquity
of NVM has stimulated interest in the design of novel concepts ensuring
correctness of concurrent programming abstractions in the face of persistency.
So far, this has lead to the design of a number of persistent concurrent data
structures, built to satisfy an associated notion of correctness: durable
linearizability.
In this paper, we transfer the principle of durable concurrent correctness to
the area of software transactional memory (STM). Software transactional memory
algorithms allow for concurrent access to shared state. Like linearizability
for concurrent data structures, opacity is the established notion of
correctness for STMs. First, we provide a novel definition of durable opacity
extending opacity to handle crashes and recovery in the context of NVM. Second,
we develop a durably opaque version of an existing STM algorithm, namely the
Transactional Mutex Lock (TML). Third, we design a proof technique for durable
opacity based on refinement between TML and an operational characterisation of
durable opacity by adapting the TMS2 specification. Finally, we apply this
proof technique to show that the durable version of TML is indeed durably
opaque. The correctness proof is mechanized within Isabelle.Comment: This is the full version of the paper that is to appear in FORTE 2020
(https://www.discotec.org/2020/forte
- …