632 research outputs found
Replicating Persistent Memory Key-Value Stores with Efficient RDMA Abstraction
Combining persistent memory (PM) with RDMA is a promising approach to
performant replicated distributed key-value stores (KVSs). However, existing
replication approaches do not work well when applied to PM KVSs: 1) Using RPC
induces software queueing and execution at backups, increasing request latency;
2) Using one-sided RDMA WRITE causes many streams of small PM writes, leading
to severe device-level write amplification (DLWA) on PM. In this paper, we
propose Rowan, an efficient RDMA abstraction to handle replication writes in PM
KVSs; it aggregates concurrent remote writes from different servers, and lands
these writes to PM in a sequential (thus low DLWA) and one-sided (thus low
latency) manner. We realize Rowan with off-the-shelf RDMA NICs. Further, we
build Rowan-KV, a log-structured PM KVS using Rowan for replication. Evaluation
shows that under write-intensive workloads, compared with PM KVSs using RPC and
RDMA WRITE for replication, Rowan-KV boosts throughput by 1.22X and 1.39X as
well as lowers median PUT latency by 1.77X and 2.11X, respectively, while
largely eliminating DLWA.Comment: Accepted to OSDI 202
Recommended from our members
Building Reliable Software for Persistent Memory
Persistent memory (PMEM) technologies preserve data across power cycles and provide performance comparable to DRAM. In emerging computer systems, PMEM will operate on the main memory bus, becoming byte-addressable and cache-coherent. One key feature enabled by persistent memory is to allow software directly accessing durable data using the CPU’s load/store instructions, even from the user-space.However, building reliable software for persistent memory faces new challenges from two aspects: crash consistency and fault tolerance. Maintaining crash consistency requires the ability to recover data integrity in the event of system crashes. Using load/store instructions to access durable data introduces a new programming paradigm, that is prone to new types of programming errors. Fault tolerance involves detecting and recovering from persistent memory errors, including memory media errors and scribbles from software bugs. With direct access, file systems and user-space applications have to explicitly manage these errors, instead of relying on convenient functions from lower I/O stacks.We identify unique challenges in improving reliability for PMEM-based software and propose solutions. The thesis first introduces NOVA-Fortis, a fault-tolerant PMEM file system incorporating replication, checksums, and parity for protecting the file system’s metadata and the user’s file data. NOVA-Fortis is both fast and resilient in the face of corruption due to media errors and software bugs.NOVA-Fortis only protects file data via the read() and write() system calls. When an application memory-maps a PMEM file, NOVA-Fortis has to disable file data protection because mmap() leaves the file system unaware of updates made to the file. For protecting memory-mapped PMEM data, we present Pangolin, a fault-tolerant persistent object library to protect an application’s objects from persistent memory errors.Writing programs to ensure crash consistency in PMEM remains challenging. Recovery bugs arise as a new type of programming error, preventing a post-crash PMEM file from recovering to a consistent state. Thus, we design two debugging tools for persistent memory programming: PmemConjurer and PmemSanitizer. PmemConjurer is a static analyzer using symbolic execution to find recovery bugs without running a compiled program. PmemSanitizer contains compiler instrumentation and run-time recovery bug analysis, compensating PmemConjurer with multi-threading support and store reordering tests
IPCFA: A Methodology for Acquiring Forensically-Sound Digital Evidence in the Realm of IAAS Public Cloud Deployments
Cybercrimes and digital security breaches are on the rise: savvy businesses and organizations of all sizes must ready themselves for the worst. Cloud computing has become the new normal, opening even more doors for cybercriminals to commit crimes that are not easily traceable. The fast pace of technology adoption exceeds the speed by which the cybersecurity community and law enforcement agencies (LEAs) can invent countermeasures to investigate and prosecute such criminals. While presenting defensible digital evidence in courts of law is already complex, it gets more complicated if the crime is tied to public cloud computing, where storage, network, and computing resources are shared and dispersed over multiple geographical areas. Investigating such crimes involves collecting evidence data from the public cloud that is court-sound. Digital evidence court admissibility in the U.S. is governed predominantly by the Federal Rules of Evidence and Federal Rules of Civil Procedures. Evidence authenticity can be challenged by the Daubert test, which evaluates the forensic process that took place to generate the presented evidence.
Existing digital forensics models, methodologies, and processes have not adequately addressed crimes that take place in the public cloud. It was only in late 2020 that the Scientific Working Group on Digital Evidence (SWGDE) published a document that shed light on best practices for collecting evidence from cloud providers. Yet SWGDE’s publication does not address the gap between the technology and the legal system when it comes to evidence admissibility. The document is high level with more focus on law enforcement processes such as issuing a subpoena and preservation orders to the cloud provider.
This research proposes IaaS Public Cloud Forensic Acquisition (IPCFA), a methodology to acquire forensic-sound evidence from public cloud IaaS deployments. IPCFA focuses on bridging the gap between the legal and technical sides of evidence authenticity to help produce admissible evidence that can withstand scrutiny in U.S. courts. Grounded in design research science (DSR), the research is rigorously evaluated using two hypothetical scenarios for crimes that take place in the public cloud. The first scenario takes place in AWS and is hypothetically walked-thru. The second scenario is a demonstration of IPCFA’s applicability and effectiveness on Azure Cloud. Both cases are evaluated using a rubric built from the federal and civil digital evidence requirements and the international best practices for iv digital evidence to show the effectiveness of IPCFA in generating cloud evidence sound enough to be considered admissible in court
Understanding and Optimizing Flash-based Key-value Systems in Data Centers
Flash-based key-value systems are widely deployed in today’s data centers for providing high-speed data processing services. These systems deploy flash-friendly data structures, such as slab and Log Structured Merge(LSM) tree, on flash-based Solid State Drives(SSDs) and provide efficient solutions in caching and storage scenarios. With the rapid evolution of data centers, there appear plenty of challenges and opportunities for future optimizations.
In this dissertation, we focus on understanding and optimizing flash-based key-value systems from the perspective of workloads, software, and hardware as data centers evolve. We first propose an on-line compression scheme, called SlimCache, considering the unique characteristics of key-value workloads, to virtually enlarge the cache space, increase the hit ratio, and improve the cache performance. Furthermore, to appropriately configure increasingly complex modern key-value data systems, which can have more than 50 parameters with additional hardware and system settings, we quantitatively study and compare five multi-objective optimization methods for auto-tuning the performance of an LSM-tree based key-value store in terms of throughput, the 99th percentile tail latency, convergence time, real-time system throughput, and the iteration process, etc. Last but not least, we conduct an in-depth, comprehensive measurement work on flash-optimized key-value stores with recently emerging 3D XPoint SSDs. We reveal several unexpected bottlenecks in the current key-value store design and present three exemplary case studies to showcase the efficacy of removing these bottlenecks with simple methods on 3D XPoint SSDs. Our experimental results show that our proposed solutions significantly outperform traditional methods. Our study also contributes to providing system implications for auto-tuning the key-value system on flash-based SSDs and optimizing it on revolutionary 3D XPoint based SSDs
An Insider Misuse Threat Detection and Prediction Language
Numerous studies indicate that amongst the various types of security threats, the
problem of insider misuse of IT systems can have serious consequences for the health
of computing infrastructures. Although incidents of external origin are also dangerous,
the insider IT misuse problem is difficult to address for a number of reasons. A
fundamental reason that makes the problem mitigation difficult relates to the level of
trust legitimate users possess inside the organization. The trust factor makes it difficult
to detect threats originating from the actions and credentials of individual users. An
equally important difficulty in the process of mitigating insider IT threats is based on
the variability of the problem. The nature of Insider IT misuse varies amongst
organizations. Hence, the problem of expressing what constitutes a threat, as well as
the process of detecting and predicting it are non trivial tasks that add up to the multi-
factorial nature of insider IT misuse.
This thesis is concerned with the process of systematizing the specification of insider
threats, focusing on their system-level detection and prediction. The design of suitable
user audit mechanisms and semantics form a Domain Specific Language to detect and
predict insider misuse incidents. As a result, the thesis proposes in detail ways to
construct standardized descriptions (signatures) of insider threat incidents, as means
of aiding researchers and IT system experts mitigate the problem of insider IT misuse.
The produced audit engine (LUARM – Logging User Actions in Relational Mode) and
the Insider Threat Prediction and Specification Language (ITPSL) are two utilities that
can be added to the IT insider misuse mitigation arsenal. LUARM is a novel audit
engine designed specifically to address the needs of monitoring insider actions. These
needs cannot be met by traditional open source audit utilities. ITPSL is an XML based
markup that can standardize the description of incidents and threats and thus make use
of the LUARM audit data. Its novelty lies on the fact that it can be used to detect as
well as predict instances of threats, a task that has not been achieved to this date by a
domain specific language to address threats.
The research project evaluated the produced language using a cyber-misuse
experiment approach derived from real world misuse incident data. The results of the
experiment showed that the ITPSL and its associated audit engine LUARM
provide a good foundation for insider threat specification and prediction. Some
language deficiencies relate to the fact that the insider threat specification process
requires a good knowledge of the software applications used in a computer system. As
the language is easily expandable, future developments to improve the language
towards this direction are suggested
Verifying correctness of persistent concurrent data structures: a sound and complete method
Non-volatile memory (NVM), aka persistent memory, is a new memory paradigm that preserves its contents even after power loss. The expected ubiquity of NVM has stimulated interest in the design of persistent concurrent data structures, together with associated notions of correctness. In this paper, we present a formal proof technique for durable linearizability, which is a correctness criterion that extends linearizability to handle crashes and recovery in the context ofNVM.Our proofs are based on refinement of Input/Output automata (IOA) representations of concurrent data structures. To this end, we develop a generic procedure for transforming any standard sequential data structure into a durable specification and prove that this transformation is both sound and complete. Since the durable specification only exhibits durably linearizable behaviours, it serves as the abstract specification in our refinement proof. We exemplify our technique on a recently proposed persistentmemory queue that builds on Michael and Scott’s lock-free queue. To support the proofs, we describe an automated translation procedure from code to IOA and a thread-local proof technique for verifying correctness of invariants
VISOR: virtual machine images management service for cloud infarestructures
Cloud Computing is a relatively novel paradigm that aims to fulfill the computing as utility dream. It has appeared to bring the possibility of providing computing resources (such as servers, storage and networks) as a service and on demand, making them accessible through common Internet protocols. Through cloud offers, users only need to pay for the amount of resources they
need and for the time they use them. Virtualization is the clouds key technology, acting upon virtual machine images to deliver fully functional virtual machine instances. Therefore, virtual machine images play an important role in Cloud Computing and their efficient management becomes a key concern that should be carefully addressed. To tackle this requirement, most cloud offers provide their own image repository, where images are stored and retrieved from, in order to instantiate new virtual machines. However, the rise of Cloud Computing has brought
new problems in managing large collections of images.
Existing image repositories are not able to efficiently manage, store and catalogue virtual machine images from other clouds through the same centralized service repository. This becomes especially important when considering the management of multiple heterogeneous cloud offers. In fact, despite the hype around Cloud Computing, there are still existing barriers to its widespread adoption. Among them, clouds interoperability is one of the most notable issues.
Interoperability limitations arise from the fact that current cloud offers provide proprietary interfaces, and their services are tied to their own requirements. Therefore, when dealing with multiple heterogeneous clouds, users face hard to manage integration and compatibility issues.
The management and delivery of virtual machine images across different clouds is an example of such interoperability constraints.
This dissertation presents VISOR, a cloud agnostic virtual machine images management service and repository. Our work towards VISOR aims to provide a service not designed to fit in a specific cloud offer but rather to overreach sharing and interoperability limitations among different clouds. With VISOR, the management of clouds interoperability can be seamlessly abstracted
from the underlying procedures details. In this way, it aims to provide users with the
ability to manage and expose virtual machine images across heterogeneous clouds, throughout the same generic and centralized repository and management service. VISOR is an open source software with a community-driven development process, thus it can be freely customized and further improved by everyone. The conducted tests to evaluate its performance and resources
usage rate have shown VISOR as a stable and high performance service, even when compared
with other services already in production. Lastly, placing clouds as the main target audience is not a limitation for other use cases. In fact, virtualization and virtual machine images are not exclusively linked to cloud environments. Therefore and given the service agnostic design concerns, it is possible to adapt it to other usage scenarios as well.A Computação em Nuvem (”Cloud Computing”) é um paradigma relativamente novo que visa
cumprir o sonho de fornecer a computação como um serviço. O mesmo surgiu para possibilitar o
fornecimento de recursos de computação (servidores, armazenamento e redes) como um serviço
de acordo com as necessidades dos utilizadores, tornando-os acessĂveis atravĂ©s de protocolos de
Internet comuns. Através das ofertas de ”cloud”, os utilizadores apenas pagam pela quantidade
de recursos que precisam e pelo tempo que os usam. A virtualização é a tecnologia chave
das ”clouds”, atuando sobre imagens de máquinas virtuais de forma a gerar máquinas virtuais
totalmente funcionais. Sendo assim, as imagens de máquinas virtuais desempenham um papel
fundamental no ”Cloud Computing” e a sua gestão eficiente torna-se um requisito que deve ser
cuidadosamente analisado. Para fazer face a tal necessidade, a maioria das ofertas de ”cloud”
fornece o seu prĂłprio repositĂłrio de imagens, onde as mesmas sĂŁo armazenadas e de onde
são copiadas a fim de criar novas máquinas virtuais. Contudo, com o crescimento do ”Cloud
Computing” surgiram novos problemas na gestão de grandes conjuntos de imagens.
Os repositórios existentes não são capazes de gerir, armazenar e catalogar images de máquinas
virtuais de forma eficiente a partir de outras ”clouds”, mantendo um único repositório e serviço
centralizado. Esta necessidade torna-se especialmente importante quando se considera a gestĂŁo
de múltiplas ”clouds” heterogéneas. Na verdade, apesar da promoção extrema do ”Cloud Computing”, ainda existem barreiras à sua adoção generalizada. Entre elas, a interoperabilidade
entre ”clouds” é um dos constrangimentos mais notáveis. As limitações de interoperabilidade
surgem do fato de as ofertas de ”cloud” atuais possuĂrem interfaces proprietárias, e de os seus
serviços estarem vinculados às suas próprias necessidades. Os utilizadores enfrentam assim
problemas de compatibilidade e integração difĂceis de gerir, ao lidar com ”clouds” de diferentes fornecedores. A gestĂŁo e disponibilização de imagens de máquinas virtuais entre diferentes
”clouds” é um exemplo de tais restrições de interoperabilidade.
Esta dissertação apresenta o VISOR, o qual é um repositório e serviço de gestão de imagens de máquinas virtuais genérico. O nosso trabalho em torno do VISOR visa proporcionar um
serviço que nĂŁo foi concebido para lidar com uma ”cloud” especĂfica, mas sim para superar as
limitações de interoperabilidade entre ”clouds”. Com o VISOR, a gestão da interoperabilidade
entre ”clouds” Ă© abstraĂda dos detalhes subjacentes. Desta forma pretende-se proporcionar
aos utilizadores a capacidade de gerir e expor imagens entre ”clouds” heterogéneas, mantendo
um repositório e serviço de gestão centralizados. O VISOR é um software de código livre com
um processo de desenvolvimento aberto. O mesmo pode ser livremente personalizado e melhorado por qualquer pessoa. Os testes realizados para avaliar o seu desempenho e a taxa de
utilização de recursos mostraram o VISOR como sendo um serviço estável e de alto desempenho,
mesmo quando comparado com outros serviços já em utilização. Por fim, colocar as ”clouds”
como principal público-alvo não representa uma limitação para outros tipos de utilização. Na
verdade, as imagens de máquinas virtuais e a virtualização não estão exclusivamente ligadas a
ambientes de ”cloud”. Assim sendo, e tendo em conta as preocupações tidas no desenho de um
serviço genĂ©rico, tambĂ©m Ă© possĂvel adaptar o nosso serviço a outros cenários de utilização
- …