Logging versus Soft Updates: Asynchronous Meta-data Protection in File Systems

The UNIX Fast File System (FFS) is probably the most widely-used file system for performance comparisons. However, such comparisons frequently overlook many of the performance enhancements that have been added over the past decade. In this paper, we explore the two most commonly used approaches for improving the performance of meta-data operations and recovery: logging and Soft Updates. The commercial sector has moved en masse to logging file systems, as evidenced by their presence on nearly every server platform available today: Solaris, AIX, Digital UNIX, HP-UX, Irix, and Windows NT. On all but Solaris, the default file system uses logging. In the meantime, Soft Updates holds the promise of providing stronger reliability guarantees than logging, with faster recovery and superior performance in certain boundary cases. In this paper, we explore the benefits of both Soft Updates and logging, comparing their behavior on both microbenchmarks and workload-based macrobenchmarks. We find that logging alone is not sufficient to “solve” the meta-data update problem. If synchronous semantics are required (i.e., meta-data operations are durable once the system call returns), then the logging systems cannot realize their full potential. Only when this synchronicity requirement is relaxed can logging systems approach the performance of systems like Soft Updates. Our asynchronous logging and Soft Updates systems perform comparably in most cases. While Soft Updates excels in some meta-data intensive microbenchmarks, it outperforms logging on only two of the four workloads we examined and performs less well on one.Engineering and Applied Science

Effective testing for concurrency bugs

In the current multi-core era, concurrency bugs are a serious threat to software reliability. As hardware becomes more parallel, concurrent programming will become increasingly pervasive. However, correct concurrent programming is known to be extremely challenging for developers and can easily lead to the introduction of concurrency bugs. This dissertation addresses this challenge by proposing novel techniques to help developers expose and detect concurrency bugs. We conducted a bug study to better understand the external and internal effects of real-world concurrency bugs. Our study revealed that a significant fraction of concurrency bugs qualify as semantic or latent bugs, which are two particularly challenging classes of concurrency bugs. Based on the insights from the study, we propose a concurrency bug detector, PIKE that analyzes the behavior of program executions to infer whether concurrency bugs have been triggered during a concurrent execution. In addition, we present the design of a testing tool, SKI, that allows developers to test operating system kernels for concurrency bugs in a practical manner. SKI bridges the gap between user-mode testing and kernel-mode testing by enabling the systematic exploration of the kernel thread interleaving space. Our evaluation shows that both PIKE and SKI are effective at finding concurrency bugs.Im gegenwärtigen Multicore-Zeitalter sind Fehler aufgrund von Nebenläufigkeit eine ernsthafte Bedrohung der Zuverlässigkeit von Software. Mit der wachsenden Parallelisierung von Hardware wird nebenläufiges Programmieren nach und nach allgegenwärtig. Diese Art von Programmieren ist jedoch als äußerst schwierig bekannt und kann leicht zu Programmierfehlern führen. Die vorliegende Dissertation nimmt sich dieser Herausforderung an indem sie neuartige Techniken vorschlägt, die Entwicklern beim Aufdecken von Nebenläufigkeitsfehlern helfen. Wir führen eine Studie von Fehlern durch, um die externen und internen Effekte von in der Praxis vorkommenden Nebenläufigkeitsfehlern besser zu verstehen. Diese ergibt, dass ein bedeutender Anteil von solchen Fehlern als semantisch bzw. latent zu charakterisieren ist -- zwei besonders herausfordernde Klassen von Nebenläufigkeitsfehlern. Basierend auf den Erkenntnissen der Studie entwickeln wir einen Detektor (PIKE), der Programmausführungen daraufhin analysiert, ob Nebenläufigkeitsfehler aufgetreten sind. Weiterhin präsentieren wir das Design eines Testtools (SKI), das es Entwicklern ermöglicht, Betriebssystemkerne praktikabel auf Nebenläufigkeitsfehler zu überprüfen. SKI füllt die Lücke zwischen Testen im Benutzermodus und Testen im Kernelmodus, indem es die systematische Erkundung der Kernel-Thread-Verschachtelungen erlaubt. Unsere Auswertung zeigt, dass sowohl PIKE als auch SKI effektiv Nebenläufigkeitsfehler finden

The Design of a High-Integrity Disk Management Subsystem

This dissertation describes and experimentally evaluates the design of the Logical Disk, a disk management subsystem that guarantees the integrity of data stored on disk even after system failures, while still providing performance competitive to other storage systems. Current storage systems that use the hard disk as storage medium, such as file systems, often do not provide sufficient protection against loss of data after a system failure. The designers of such systems are afraid that the amount of effort necessary for data protection would also result in too much loss of performance. The Logical Disk uses many different techniques to guarantee data integrity, including the support to execute multiple commands as one atomic action and avoiding `in-place updates' at all times. The techniques used to provide competitive performance include the technique of combining many, small write commands into one large, sequential, and thus efficient, write to disk, and clustering the data on disk continuously and automatically.Tanenbaum, A.S. [Promotor]Jonge, W. de [Copromotor

Analyzing Metadata Performance in Distributed File Systems

Distributed file systems are important building blocks in modern computing environments. The challenge of increasing I/O bandwidth to files has been largely resolved by the use of parallel file systems and sufficient hardware. However, determining the best means by which to manage large amounts of metadata, which contains information about files and directories stored in a distributed file system, has proved a more difficult challenge. The objective of this thesis is to analyze the role of metadata and present past and current implementations and access semantics. Understanding the development of the current file system interfaces and functionality is a key to understanding their performance limitations. Based on this analysis, a distributed metadata benchmark termed DMetabench is presented. DMetabench significantly improves on existing benchmarks and allows stress on metadata operations in a distributed file system in a parallelized manner. Both intranode and inter-node parallelity, current trends in computer architecture, can be explicitly tested with DMetabench. This is due to the fact that a distributed file system can have different semantics inside a client node rather than semantics between multiple nodes. As measurements in larger distributed environments may exhibit performance artifacts difficult to explain by reference to average numbers, DMetabench uses a time-logging technique to record time-related changes in the performance of metadata operations and also protocols additional details of the runtime environment for post-benchmark analysis. Using the large production file systems at the Leibniz Supercomputing Center (LRZ) in Munich, the functionality of DMetabench is evaluated by means of measurements on different distributed file systems. The results not only demonstrate the effectiveness of the methods proposed but also provide unique insight into the current state of metadata performance in modern file systems

Counting People in ITS Systems Based on WiFi Traffic Analysis

Tato práce se zaměřuje na zachytávání a analýzu Wi-Fi provozu s cílem odhadnout počet osob pomocí filtrování MAC adres. K tomu jsou využívány různé programy k~zachytávání Wi-Fi paketů. Zachycený provoz je pak následně analyzován pro stanovení počtu přítomných osob. Dále se výzkum zabývá výzvou filtrování falešných MAC adres generovaných zařízeními, které mohou zkreslit přesnost odhadu. Tato práce přispívá do~oblasti analýzy Wi-Fi provozu a poskytuje cenné poznatky, které budou později využity pro zlepšení dohledu davu v integrovaných veřejných dopravních systémech.This thesis focuses on capturing and analyzing Wi-Fi traffic to estimate the number of individuals using MAC address filtering. Various programs are utilized to capture Wi-Fi packets around a monitoring station, and the collected traffic is later analyzed to determine the number of people present. Additionally, the research addresses the challenge of filtering out fake MAC addresses generated by devices, which can distort the accuracy of the estimation. This research contributes to the field of Wi-Fi traffic analysis and provides valuable knowledge and will be later used for improving crowd management in public integrated transport systems.460 - Katedra informatikyvýborn

Multi-purpose embedded communication gateway : system design and testbed implementation

Masteroppgave i Informasjons- og kommunikasjonsteknologi IKT590 Universitetet i Agder 2014This dissertation revolves around developing a multi-purpose embedded communication gateway. The gateway is equipped with multiple communication interfaces including Ethernet, Bluetooth, WiFi, Zigbee, LTE, and it can be configured and utilized for many purposes, such as a failover of an Ethernet cable via 4G in order to maintain the network connectivity. Raspberry Pi circuit board and the operating system Raspbian are selected as the hardware and the software platforms respectively. Different communication interfaces are coordinated by the Raspberry Pi and are configured via Linux scripts according to various use cases. Furthermore, a hardware watchdog is adopted to enhance the availability of system. In addition, the system is encapsulated into a box to increase its portability. The system is validated and evaluated through rigorous test-bed experiments. Experiment results indicate that the developed router works smoothly and reliably in environments with little electrical disturbances

2OS

In this book I approach the problem of understanding an OS from the point of view of a C programmer who needs to understand enough of how an OS works to program efficiently and avoid traps and pitfalls arising from not understanding what is happening underneath you. If you have a deep understanding of the memory system, you will not program in a style that loses significant performance by breaking the assumptions of the OS designer. If you have an understanding of how IO works, you can make good use of OS services. As you work through this book you will see other examples