3,533 research outputs found

    TANDEM: taming failures in next-generation datacenters with emerging memory

    Get PDF
    The explosive growth of online services, leading to unforeseen scales, has made modern datacenters highly prone to failures. Taming these failures hinges on fast and correct recovery, minimizing service interruptions. Applications, owing to recovery, entail additional measures to maintain a recoverable state of data and computation logic during their failure-free execution. However, these precautionary measures have severe implications on performance, correctness, and programmability, making recovery incredibly challenging to realize in practice. Emerging memory, particularly non-volatile memory (NVM) and disaggregated memory (DM), offers a promising opportunity to achieve fast recovery with maximum performance. However, incorporating these technologies into datacenter architecture presents significant challenges; Their distinct architectural attributes, differing significantly from traditional memory devices, introduce new semantic challenges for implementing recovery, complicating correctness and programmability. Can emerging memory enable fast, performant, and correct recovery in the datacenter? This thesis aims to answer this question while addressing the associated challenges. When architecting datacenters with emerging memory, system architects face four key challenges: (1) how to guarantee correct semantics; (2) how to efficiently enforce correctness with optimal performance; (3) how to validate end-to-end correctness including recovery; and (4) how to preserve programmer productivity (Programmability). This thesis aims to address these challenges through the following approaches: (a) defining precise consistency models that formally specify correct end-to-end semantics in the presence of failures (consistency models also play a crucial role in programmability); (b) developing new low-level mechanisms to efficiently enforce the prescribed models given the capabilities of emerging memory; and (c) creating robust testing frameworks to validate end-to-end correctness and recovery. We start our exploration with non-volatile memory (NVM), which offers fast persistence capabilities directly accessible through the processor’s load-store (memory) interface. Notably, these capabilities can be leveraged to enable fast recovery for Log-Free Data Structures (LFDs) while maximizing performance. However, due to the complexity of modern cache hierarchies, data hardly persist in any specific order, jeop- ardizing recovery and correctness. Therefore, recovery needs primitives that explicitly control the order of updates to NVM (known as persistency models). We outline the precise specification of a novel persistency model – Release Persistency (RP) – that provides a consistency guarantee for LFDs on what remains in non-volatile memory upon failure. To efficiently enforce RP, we propose a novel microarchitecture mechanism, lazy release persistence (LRP). Using standard LFDs benchmarks, we show that LRP achieves fast recovery while incurring minimal overhead on performance. We continue our discussion with memory disaggregation which decouples memory from traditional monolithic servers, offering a promising pathway for achieving very high availability in replicated in-memory data stores. Achieving such availability hinges on transaction protocols that can efficiently handle recovery in this setting, where compute and memory are independent. However, there is a challenge: disaggregated memory (DM) fails to work with RPC-style protocols, mandating one-sided transaction protocols. Exacerbating the problem, one-sided transactions expose critical low-level ordering to architects, posing a threat to correctness. We present a highly available transaction protocol, Pandora, that is specifically designed to achieve fast recovery in disaggregated key-value stores (DKVSes). Pandora is the first one-sided transactional protocol that ensures correct, non-blocking, and fast recovery in DKVS. Our experimental implementation artifacts demonstrate that Pandora achieves fast recovery and high availability while causing minimal disruption to services. Finally, we introduce a novel target litmus-testing framework – DART – to validate the end-to-end correctness of transactional protocols with recovery. Using DART’s target testing capabilities, we have found several critical bugs in Pandora, highlighting the need for robust end-to-end testing methods in the design loop to iteratively fix correctness bugs. Crucially, DART is lightweight and black-box, thereby eliminating any intervention from the programmers

    ENHANCING CLOUD SYSTEM RUNTIME TO ADDRESS COMPLEX FAILURES

    Get PDF
    As the reliance on cloud systems intensifies in our progressively digital world, understanding and reinforcing their reliability becomes more crucial than ever. Despite impressive advancements in augmenting the resilience of cloud systems, the growing incidence of complex failures now poses a substantial challenge to the availability of these systems. With cloud systems continuing to scale and increase in complexity, failures not only become more elusive to detect but can also lead to more catastrophic consequences. Such failures question the foundational premises of conventional fault-tolerance designs, necessitating the creation of novel system designs to counteract them. This dissertation aims to enhance distributed systems’ capabilities to detect, localize, and react to complex failures at runtime. To this end, this dissertation makes contributions to address three emerging categories of failures in cloud systems. The first part delves into the investigation of partial failures, introducing OmegaGen, a tool adept at generating tailored checkers for detecting and localizing such failures. The second part grapples with silent semantic failures prevalent in cloud systems, showcasing our study findings, and introducing Oathkeeper, a tool that leverages past failures to infer rules and expose these silent issues. The third part explores solutions to slow failures via RESIN, a framework specifically designed to detect, diagnose, and mitigate memory leaks in cloud-scale infrastructures, developed in collaboration with Microsoft Azure. The dissertation concludes by offering insights into future directions for the construction of reliable cloud systems

    Distributed Ledger Technology (DLT) Applications in Payment, Clearing, and Settlement Systems:A Study of Blockchain-Based Payment Barriers and Potential Solutions, and DLT Application in Central Bank Payment System Functions

    Get PDF
    Payment, clearing, and settlement systems are essential components of the financial markets and exert considerable influence on the overall economy. While there have been considerable technological advancements in payment systems, the conventional systems still depend on centralized architecture, with inherent limitations and risks. The emergence of Distributed ledger technology (DLT) is being regarded as a potential solution to transform payment and settlement processes and address certain challenges posed by the centralized architecture of traditional payment systems (Bank for International Settlements, 2017). While proof-of-concept projects have demonstrated the technical feasibility of DLT, significant barriers still hinder its adoption and implementation. The overarching objective of this thesis is to contribute to the developing area of DLT application in payment, clearing and settlement systems, which is still in its initial stages of applications development and lacks a substantial body of scholarly literature and empirical research. This is achieved by identifying the socio-technical barriers to adoption and diffusion of blockchain-based payment systems and the solutions proposed to address them. Furthermore, the thesis examines and classifies various applications of DLT in central bank payment system functions, offering valuable insights into the motivations, DLT platforms used, and consensus algorithms for applicable use cases. To achieve these objectives, the methodology employed involved a systematic literature review (SLR) of academic literature on blockchain-based payment systems. Furthermore, we utilized a thematic analysis approach to examine data collected from various sources regarding the use of DLT applications in central bank payment system functions, such as central bank white papers, industry reports, and policy documents. The study's findings on blockchain-based payment systems barriers and proposed solutions; challenge the prevailing emphasis on technological and regulatory barriers in the literature and industry discourse regarding the adoption and implementation of blockchain-based payment systems. It highlights the importance of considering the broader socio-technical context and identifying barriers across all five dimensions of the social technical framework, including technological, infrastructural, user practices/market, regulatory, and cultural dimensions. Furthermore, the research identified seven DLT applications in central bank payment system functions. These are grouped into three overarching themes: central banks' operational responsibilities in payment and settlement systems, issuance of central bank digital money, and regulatory oversight/supervisory functions, along with other ancillary functions. Each of these applications has unique motivations or value proposition, which is the underlying reason for utilizing in that particular use case

    An information system for a multiplayer geolocation game

    Get PDF
    Mestrado de dupla diplomação com a National Polytechnic University of ArméniaA great deal of travel can educate tourists. As tourism becomes more accessible, many new technologies are being used to provide interesting, appealing, and efficient trips so that travelers can find what they want, avoid getting lost, and get the most out of their trip. Science, education, and culture all benefit from technological advancement. As many good ideas can now be turned into reality, interactive education has found a place in everyday life. Research and review of geolocation-based applications (such as hiking path providers) and guide applications were conducted to better understand the technologies and tools related to the topic. The best features were designed to solve the problem after summarizing the results of applications, their advantages and disadvantages. The purpose of this paper is to create a geolocation-based game that will allow travelers to explore nearby attractions, learn about local culture, and broaden their knowledge while traveling. To summarize, a dynamic platform that can be implemented in a variety of ways was created.Uma grande quantidade de viagens pode educar os turistas. À medida que o turismo se torna mais acessível, muitas novas tecnologias estão sendo usadas para fornecer viagens interessantes, atraentes e eficientes para que os viajantes possam encontrar o que desejam, evitar se perder e aproveitar ao máximo sua viagem. Ciência, educação e cultura se beneficiam do avanço tecnológico. No muitas boas ideias podem agora se tornar realidade, a educação interativa encontrou um lugar na vida cotidiana. Pesquisa e revisão de aplicativos baseados em geolocalização (como provedores de trilhas para caminhadas) e aplicativos de guia foram realizados para entender melhor as tecnologias e ferramentas relacionadas ao tema. As melhores funcionalidades foram projetadas para solucionar o problema após resumir os resultados das aplicações, suas vantagens e desvantagens. O objetivo deste artigo é criar um jogo baseado em geolocalização que permita aos viajantes explorar atrações próximas, aprender sobre a cultura local e ampliar seus conhecimentos durante a viagem. Para resumir, foi criada uma plataforma dinâmica que pode ser implementada de várias maneiras

    A BIM - GIS Integrated Information Model Using Semantic Web and RDF Graph Databases

    Get PDF
    In recent years, 3D virtual indoor and outdoor urban modelling has become an essential geospatial information framework for civil and engineering applications such as emergency response, evacuation planning, and facility management. Building multi-sourced and multi-scale 3D urban models are in high demand among architects, engineers, and construction professionals to achieve these tasks and provide relevant information to decision support systems. Spatial modelling technologies such as Building Information Modelling (BIM) and Geographical Information Systems (GIS) are frequently used to meet such high demands. However, sharing data and information between these two domains is still challenging. At the same time, the semantic or syntactic strategies for inter-communication between BIM and GIS do not fully provide rich semantic and geometric information exchange of BIM into GIS or vice-versa. This research study proposes a novel approach for integrating BIM and GIS using semantic web technologies and Resources Description Framework (RDF) graph databases. The suggested solution's originality and novelty come from combining the advantages of integrating BIM and GIS models into a semantically unified data model using a semantic framework and ontology engineering approaches. The new model will be named Integrated Geospatial Information Model (IGIM). It is constructed through three stages. The first stage requires BIMRDF and GISRDF graphs generation from BIM and GIS datasets. Then graph integration from BIM and GIS semantic models creates IGIMRDF. Lastly, the information from IGIMRDF unified graph is filtered using a graph query language and graph data analytics tools. The linkage between BIMRDF and GISRDF is completed through SPARQL endpoints defined by queries using elements and entity classes with similar or complementary information from properties, relationships, and geometries from an ontology-matching process during model construction. The resulting model (or sub-model) can be managed in a graph database system and used in the backend as a data-tier serving web services feeding a front-tier domain-oriented application. A case study was designed, developed, and tested using the semantic integrated information model for validating the newly proposed solution, architecture, and performance

    Improving Static Analysis for Software Security at Compile-time and Runtime

    No full text
    Software security is a crucial factor in software development and maintenance. Static analysis approaches can help secure software in different ways. First, it can help identify vulnerabilities ahead-of run. For example, we can search vulnerable code in wild that are similar to buggy code in existing CVE databases, in which the program properties computed by static analysis are desired. For more complicated bugs, such as concurrency bugs, static analyses can infer more complex program properties, including the relation of pointers (i.e., alias analysis) in the program and the partial order between statements (e.g., happens-before relation), thus detect potential vulnerabilities. Second, static analyses can compute program properties (e.g., data-flow, control-flow) that we can check at runtime to achieve specific security goals (e.g., no control-flow hijack). In this dissertation, we present three approaches of computing static program properties, combined with other methods, that improves the state-of-the-art for securing real-world software at compile-time and runtime. First, for the core of searching vulnerable code, measuring code similar-ity, we present a new approach that combines static program properties, data-flow and control-flow, with deep learning method. This is to address two limitations of existing techniques: scalability and imprecision. With deep neural networks, the classification is efficient on modern GPUs, while data-flow/control-flow only needs to be computed once for each code. And by leveraging the in-formation in the encoded data-flow/control-flow and the training datasets, the deep neural network model can learn a good metric for measuring similarity between codes. Second, we present a new approach for detecting atomicity violations in Rust programs, which is a kind of semantic bugs and one of the main source of Rust concurrency issues. We use alias analysis to compute if two atomic operations could access the same variable, and happens-before relation to compute if two such atomic operations could be executed concurrently. With observed heuristics from existing research study, we then statically check if a set of atomic operations from different threads could potentially cause an atomicity violation. This approach is evaluated to be effective in a benchmark of real world Rust programs with known atomicity violations. We also find a potential new atomicity violation in a Rust project from Github. Both the above two approaches detect bugs ahead-of run. However, real-world programs are almost impossible to be bug-free. Therefore, we present a new approach that effectively defends target program against non-control data attacks by checking statically computed program proper-ties at runtime. Specifically, we introduce a new concept, origin, to abstract a piece of program path and all memory objects owned by it. We then compute all intended cross-origin data flow at compile-time, and instrument the program to check any violations to it at runtime. With an origin-based heap allocator, this approach incurs very low runtime overhead but is still effective on a benchmark of real-world programs with known CVEs

    Efficient concurrent data structure access parallelism techniques for increasing scalability

    Get PDF
    Multi-core processors have revolutionised the way data structures are designed by bringing parallelism to mainstream computing. Key to exploiting hardware parallelism available in multi-core processors are concurrent data structures. However, some concurrent data structure abstractions are inherently sequential and incapable of harnessing the parallelism performance of multi-core processors. Designing and implementing concurrent data structures to harness hardware parallelism is challenging due to the requirement of correctness, efficiency and practicability under various application constraints. In this thesis, our research contribution is towards improving concurrent data structure access parallelism to increase data structure performance. We propose new design frameworks that improve access parallelism of already existing concurrent data structure designs. Also, we propose new concurrent data structure designs with significant performance improvements. To give an insight into the interplay between hardware and concurrent data structure access parallelism, we give a detailed analysis and model the performance scalability with varying parallelism.In the first part of the thesis, we focus on data structure semantic relaxation. By relaxing the semantics of a data structure, a bigger design space, that allows weaker synchronization and more useful parallelism, is unveiled. Investigating new data structure designs, capable of trading semantics for achieving better performance in a monotonic way, is a major challenge in the area. We algorithmically address this challenge in this part of the thesis. We present an efficient, lock-free, concurrent data structure design framework for out-of-order semantic relaxation. We introduce a new two-dimensional algorithmic design, that uses multiple instances of a given data structure to improve access parallelism. In the second part of the thesis, we propose an efficient priority queue that improves access parallelism by reducing the number of synchronization points for each operation. Priority queues are fundamental abstract data types, often used to manage limited resources in parallel systems. Typical proposed parallel priority queue implementations are based on heaps or skip lists. In recent literature, skip lists have been shown to be the most efficient design choice for implementing priority queues. Though numerous intricate implementations of skip list based queues have been proposed in the literature, their performance is constrained by the high number of global atomic updates per operation and the high memory consumption, which are proportional to the number of sub-lists in the queue. In this part of the thesis, we propose an alternative approach for designing lock-free linearizable priority queues, that significantly improve memory efficiency and throughput performance, by reducing the number of global atomic updates and memory consumption as compared to skip-list based queues. To achieve this, our new design combines two structures; a search tree and a linked list, forming what we call a Tree Search List Queue (TSLQueue). Subsequently, we analyse and introduce a model for lock-free concurrent data structure access parallelism. The major impediment to scaling concurrent data structures is memory contention when accessing shared data structure access points, leading to thread serialisation, and hindering parallelism. Aiming to address this challenge, a significant amount of work in the literature has proposed multi-access techniques that improve concurrent data structure parallelism. However, there is little work on analysing and modelling the execution behaviour of concurrent multi-access data structures especially in a shared memory setting. In this part of the thesis, we analyse and model the general execution behaviour of concurrent multi-access data structures in the shared memory setting. We study and analyse the behaviour of the two popular random access patterns: shared (Remote) and exclusive (Local) access, and the behaviour of the two most commonly used atomic primitives for designing lock-free data structures: Compare and Swap, and, Fetch and Add

    Shufflecake: Plausible Deniability for Multiple Hidden Filesystems on Linux

    Get PDF
    We present Shufflecake, a new plausible deniability design to hide the existence of encrypted data on a storage medium making it very difficult for an adversary to prove the existence of such data. Shufflecake can be considered a ``spiritual successor\u27\u27 of tools such as TrueCrypt and VeraCrypt, but vastly improved: it works natively on Linux, it supports any filesystem of choice, and can manage multiple volumes per device, so to make deniability of the existence of hidden partitions really plausible. Compared to ORAM-based solutions, Shufflecake is extremely fast and simpler but does not offer native protection against multi-snapshot adversaries. However, we discuss security extensions that are made possible by its architecture, and we show evidence why these extensions might be enough to thwart more powerful adversaries. We implemented Shufflecake as an in-kernel tool for Linux, adding useful features, and we benchmarked its performance showing only a minor slowdown compared to a base encrypted system. We believe Shufflecake represents a useful tool for people whose freedom of expression is threatened by repressive authorities or dangerous criminal organizations, in particular: whistleblowers, investigative journalists, and activists for human rights in oppressive regimes

    The count/mass distinction in native and non-native grammar

    No full text
    This dissertation investigates whether the interpretation of different types of nouns is affected by count/mass morphosyntax and whether second language (L2) learners are affected by first language (L1) transfer from their native language or are instead affected by semantic universals in L2 acquisition of the count/mass distinction. Furthermore, this dissertation also examines how similarly/differently L2-learners behave in online and offline tasks regarding the acquisition of the count/mass morphosyntax in English. As current literature does not provide a conclusive answer to these questions, three experiments were conducted to address this gap. Experiment 1 investigates whether L2-learners transfer the properties of plural marking from their native language to their L2 regarding object-mass nouns. Native speakers (NSs) of English, L1-Korean L2-English learners, and L1-Mandarin L2-English learners were compared. The results show no transfer effects, but L2-learners’ performances were affected by semantic universals (atomicity). Furthermore, it was also found that L2-learners show the exact same patterns in online and offline tasks, suggesting that L2-learners’ knowledge about how atomicity relates to the count/mass distinction (specifically, the mapping between plural marking and atomicity) is present at both explicit and implicit levels. Experiments 2a, 2b, and 2c examine whether the interpretation of different nouns is affected by the count/mass morphosyntax or universality in native and non-native grammar. In this set of experiments, NSs of English, Korean, and Mandarin Chinese and L1-Korean L2-English learners and L1-Mandarin L2-English learners were tested and compared. The results indicate that the interpretation of different types of nouns were affected by semantic universals (atomicity) without L1-transfer effects from the count/mass morphosyntax of their L1s. Experiments 3a and 3b investigate whether L2-learners transfer the properties of plural marking from their native language to their L2 with respect to flexible nouns. In these two experiments, English NSs, L1-Korean L2-English learners, and L1-Mandarin L2-English learners were compared. The results show that L2-learners, though not affected by L1-transfer, were affected by semantic universals (atomicity) in their use of flexible nouns. Furthermore, unlike in Experiment 1, it was found that L2-learners performed better in the offline task than in the online task, suggesting that L2-learners’ knowledge about how atomicity relates to the count/mass distinction is better at the explicit level than the implicit level. A possible reason for such differences in the results between Experiment 1 and Experiments 3a and 3b is that interpretation trumps morphosyntax with object-mass nouns while the judgments of flexible nouns are flexible. Thus, it is possible that flexible judgments on flexible nouns are what led learners to be successful on the GJT to some extent. Taken together, this dissertation shows that universality (atomicity), and not L1-transfer, plays a role in the interpretation of different types of nouns and in the L2-acquisition of the count/mass distinction in English. This dissertation also demonstrates that L2-learners’ knowledge at explicit and implicit levels are separate with L2-learners having different explicit and implicit knowledge depending on noun type.LimitedAuthor requested closed access (OA after 2yrs) in Vireo ETD syste
    • …
    corecore