Understanding Persistent-Memory Related Issues in the Linux Kernel

Persistent memory (PM) technologies have inspired a wide range of PM-based system optimizations. However, building correct PM-based systems is difficult due to the unique characteristics of PM hardware. To better understand the challenges as well as the opportunities to address them, this paper presents a comprehensive study of PM-related issues in the Linux kernel. By analyzing 1,553 PM-related kernel patches in-depth and conducting experiments on reproducibility and tool extension, we derive multiple insights in terms of PM patch categories, PM bug patterns, consequences, fix strategies, triggering conditions, and remedy solutions. We hope our results could contribute to the development of robust PM-based storage systemsComment: ACM TRANSACTIONS ON STORAGE(TOS'23

Programming Persistent Memory

Beginning and experienced programmers will use this comprehensive guide to persistent memory programming. You will understand how persistent memory brings together several new software/hardware requirements, and offers great promise for better performance and faster application startup times—a huge leap forward in byte-addressable capacity compared with current DRAM offerings. This revolutionary new technology gives applications significant performance and capacity improvements over existing technologies. It requires a new way of thinking and developing, which makes this highly disruptive to the IT/computing industry. The full spectrum of industry sectors that will benefit from this technology include, but are not limited to, in-memory and traditional databases, AI, analytics, HPC, virtualization, and big data. Programming Persistent Memory describes the technology and why it is exciting the industry. It covers the operating system and hardware requirements as well as how to create development environments using emulated or real persistent memory hardware. The book explains fundamental concepts; provides an introduction to persistent memory programming APIs for C, C++, JavaScript, and other languages; discusses RMDA with persistent memory; reviews security features; and presents many examples. Source code and examples that you can run on your own systems are included. What You’ll Learn Understand what persistent memory is, what it does, and the value it brings to the industry Become familiar with the operating system and hardware requirements to use persistent memory Know the fundamentals of persistent memory programming: why it is different from current programming methods, and what developers need to keep in mind when programming for persistence Look at persistent memory application development by example using the Persistent Memory Development Kit (PMDK) Design and optimize data structures for persistent memory Study how real-world applications are modified to leverage persistent memory Utilize the tools available for persistent memory programming, application performance profiling, and debugging Who This Book Is For C, C++, Java, and Python developers, but will also be useful to software, cloud, and hardware architects across a broad spectrum of sectors, including cloud service providers, independent software vendors, high performance compute, artificial intelligence, data analytics, big data, etc

Speculative Execution as an Operating System Service

Software faults and vulnerabilities continue to present significant obstacles to achieving reliable and secure software. In an effort to overcome these obstacles, systems often incorporate self-monitoring and self-healing functionality. Our hypothesis is that internal monitoring is not an effective long-term strategy. However, monitoring mechanisms that are completely external lose the advantage of application-specific knowledge available to an inline monitor. To balance these tradeoffs, we present the design of VxF, an environment where both supervision and automatic remediation can take place by speculatively executing 'slices' of an application. VxF introduces the concept of an endolithic kernel by providing execution as an operating system service: execution of a process slice takes place inside a kernel thread rather than directly on the system microprocessor

Building Reliable Software for Persistent Memory

Persistent memory (PMEM) technologies preserve data across power cycles and provide performance comparable to DRAM. In emerging computer systems, PMEM will operate on the main memory bus, becoming byte-addressable and cache-coherent. One key feature enabled by persistent memory is to allow software directly accessing durable data using the CPU’s load/store instructions, even from the user-space.However, building reliable software for persistent memory faces new challenges from two aspects: crash consistency and fault tolerance. Maintaining crash consistency requires the ability to recover data integrity in the event of system crashes. Using load/store instructions to access durable data introduces a new programming paradigm, that is prone to new types of programming errors. Fault tolerance involves detecting and recovering from persistent memory errors, including memory media errors and scribbles from software bugs. With direct access, file systems and user-space applications have to explicitly manage these errors, instead of relying on convenient functions from lower I/O stacks.We identify unique challenges in improving reliability for PMEM-based software and propose solutions. The thesis first introduces NOVA-Fortis, a fault-tolerant PMEM file system incorporating replication, checksums, and parity for protecting the file system’s metadata and the user’s file data. NOVA-Fortis is both fast and resilient in the face of corruption due to media errors and software bugs.NOVA-Fortis only protects file data via the read() and write() system calls. When an application memory-maps a PMEM file, NOVA-Fortis has to disable file data protection because mmap() leaves the file system unaware of updates made to the file. For protecting memory-mapped PMEM data, we present Pangolin, a fault-tolerant persistent object library to protect an application’s objects from persistent memory errors.Writing programs to ensure crash consistency in PMEM remains challenging. Recovery bugs arise as a new type of programming error, preventing a post-crash PMEM file from recovering to a consistent state. Thus, we design two debugging tools for persistent memory programming: PmemConjurer and PmemSanitizer. PmemConjurer is a static analyzer using symbolic execution to find recovery bugs without running a compiled program. PmemSanitizer contains compiler instrumentation and run-time recovery bug analysis, compensating PmemConjurer with multi-threading support and store reordering tests

Core Technologies for Native XML Database Management Systems

This work investigates the core technologies required to build Database Management Systems (DBMSs) for large collections of XML documents. We call such systems XML Base Management Systems (XBMSs). We identify requirements, and analyze how they can be met using a conventional DBMS. Our conclusion is that an XML support layer on top of an existing conventional DBMS does not address the requirements for XBMSs. Hence, we built a Native XBMS, called Natix. Natix has been developed completely from scratch, incorporating optimizations for high-performance XML processing in those places where they are most effective

Studying a Virtual Testbed for Unverified Data

It is difficult to fully know the effects a piece of software will have on your computer, particularly when the software is distributed by an unknown source. The research in this paper focuses on malware detection, virtualization, and sandbox/honeypot techniques with the goal of improving the security of installing useful, but unverifiable, software. With a combination of these techniques, it should be possible to install software in an environment where it cannot harm a machine, but can be tested to determine its safety. Testing for malware, performance, network connectivity, memory usage, and interoperability can be accomplished without allowing the program to access the base operating system of a machine. After the full effects of the software are understood and it is determined to be safe, it could then be run from, and given access to, the base operating system. This thesis investigates the feasibility of creating a system to verify the security of unknown software while ensuring it will have no negative impact on the host machine

ARF : an Automated Real-Time Fuzzy Logic Threat Evaluation System.

Intrusion Detection has emerged as a powerful component of network security systems. A wide range of hardware and software components exist to meet most basic security needs on all platforms. These systems log system usage that could be considered as a breach of security in many networks. However, signature based intrusion detection systems have one catastrophic downfall, in that the number of alerts being logged can quickly outgrow the amount of resources necessary to investigate this anomalous behavior. This thesis explores the use of a fuzzy logic based analysis engine that gives an overall threat level of an intrusion detection sensor, prioritizing alerts that are the most threatening. This application gives security personnel a launching point to determine where security holes exist and a snapshot of the threats that exist in a system. The fuzzy logic system is based on a set of membership functions that define certain metrics from an alert dataset and a set of rules that determine a threat level based on the defined metrics. This application functions as a proof of concept prototype for an administrative tool that can analyze multiple sensors across multiple networks and give a reasonable output of the threat level across a series of intrusion detection sensors on a network. Initial testing indicates promising performance results for testing the threat level of a remote sensor using this methodology

Implementation of Distributed Transactions in BPEL

Cílem této bakalářské práce je implementovat podporu distribuovaných transakcí do projektu RiftSaw tak, aby webové služby mohly být volány v rámci distribuovaných transakcí podnikovými procesy. A to pouze v tom případě, že operace webové služby vyžaduje být provedena v rámci distribuované transakce. Oproti již funkčním implementacím přináší podporu specifikace WS-BusinessActivity a jiný způsob kontroly, zda má podnikový proces použít distribuované transakce u volaných webových služeb.The goal of this work is to implement a support of distributed transactions into the project RiftSaw so that web services can be invoked within distributed transactions by business processes. And only if a web service operation requires to be performed within a distributed transaction. Comparing to already working implementations, the presented sulution brings support of WS-BusinessActivity specification and a different way of checking that a business process use distributed transactions for invoked web services.

Smart Contracts using Blockchain

The contract is the sovereign tool employed to manage agreements between entities in today’s society. It plays a crucial role in a variety of different fields, ranging from politics to finance. This fact implies the efficiency of these applications is determined in part by the efficiency of the contracts they rely on. Despite their important role, contracts have changed relatively little in the last few centuries and remain based on an outdated technology of bureaucracy and procedures done by hand. Such systems are full of unnecessary complications, are incredibly wasteful in terms of time, money and resources, and are susceptible to human failure. In the last few years, a type of contract represented by a computer program has appeared. This concept, known as a smart contract, is based on the emerging blockchain technology. Blockchain is a type of distributed system which assures the immutability of data via the use of mathematically secure cryptographic techniques and that, as will be discussed, is well-suited for the implementation of smart contract systems. Transitioning contracts into the digital era would not only allow them to catch up to the technological pace of society but also would be advantageous from a safety and efficiency standpoint. This body of work will test the feasibility of using blockchain-based smart contracts to facilitate the first steps of this evolution. This thesis assembles a proof of concept platform that supports the specification and execution of smart contracts on a blockchain network. This proof of concept will in particular target the use case of opening a bank account, aiming to create an efficient, permanent, reliable and safe process. To achieve this, we constructed a Hyperledger Fabric network. We present herein the system developed and discuss the nuances pertaining to deploying a codebase on a blockchain, the evaluation of our system, and finally some visions for further development of this and related use cases