Canonical queries as a query answering device (Information Science)

Issued as Annual reports [nos. 1-2], and Final report, Project no. G-36-60

Fault tolerant software technology for distributed computing system

Issued as Monthly reports [nos. 1-23], Interim technical report, Technical guide books [nos. 1-2], and Final report, Project no. G-36-64

ADDING PERSISTENCE TO MAIN MEMORY PROGRAMMING

Unlocking the true potential of the new persistent memories (PMEMs) requires eliminating traditional persistent I/O abstractions altogether, by introducing persistent semantics directly into main memory programming. Such a programming model elevates failure atomicity to a first-class application property in addition to in-memory data layout, concurrency-control, and fault tolerance, and therefore requires redesign of programming abstractions for both program correctness and maximum performance gains. To address these challenges, this thesis proposes a set of system software designs that integrate persistence with main memory programming, and makes the following contributions. First, this thesis proposes a PMEM-aware I/O runtime, NVStream, that supports fast durable streaming I/O. NVStream uses a memory-based I/O interface that integrates with existing I/O data movement operations of an application to accelerate persistent data writes. NVStream carefully designs its persistent data storage layout and crash-consistent semantics to match both application and PMEM characteristics. Specifically, we leverage the streaming nature of I/O in HPC workflows, to benefit from using a log-structured PMEM storage engine design, that uses relaxed write orderings and append-only failure-atomic semantics to form strongly consistent application checkpoints. Furthermore, we identify that optimizing the I/O software stack exposes the PMEM bandwidth limitations as a bottleneck during parallel HPC I/O writes, and propose a novel data movement design – PHX. PHX uses alternative network data movement paths available in datacenters to ease up the bandwidth pressure on the PMEM memory interconnects, all while maintaining the correctness of the persistent data. Next, the thesis explores the challenges and opportunities of using PMEM for true main memory persistent programming – a single data domain for both runtime and persistent applicationstate. Such a programming model includes maintaining ACID properties during each and every update to applications persistent structures. ACID-qualified persistent programming for multi-threaded applications is hard, as the programmer has to reason about both crash-consistency and synchronization – crash-sync – semantics for programming correctness. The thesis contributes new understanding of the correctness requirements for mixing different crash-consistent and synchronization protocols, characterizes the performance of different crash-sync realizations for different applications and hardware architectures, and draws actionable insights for future designs of PMEM systems. Finally, the application state stored on node-local persistent memory is still vulnerable to catastrophic node failures. The thesis proposes a replicated persistent memory runtime, Blizzard, that supports truly fault tolerant, concurrent and persistent data-structure programming. Blizzard carefully integrates userspace networking with byte addressable PMEM for a fast, persistent memory replication runtime. The design also incorporates a replication-aware crash-sync protocol that supports consistent and concurrent updates on persistent data-structures. Blizzard offers applications the flexibility to use the data structures that best match their functional requirements, while offering better performance, and providing crucial reliability guarantees lacking from existing persistent memory runtimes.Ph.D

Atomic Transfer for Distributed Systems

Building applications and information systems increasingly means dealing with concurrency and faults stemming from distribution of system components. Atomic transactions are a well-known method for transferring the responsibility for handling concurrency and faults from developers to the software\u27s execution environment, but incur considerable execution overhead. This dissertation investigates methods that shift some of the burden of concurrency control into the network layer, to reduce response times and increase throughput. It anticipates future programmable network devices, enabling customized high-performance network protocols. We propose Atomic Transfer (AT), a distributed algorithm to prevent race conditions due to messages crossing on a path of network switches. Switches check request messages for conflicts with response messages traveling in the opposite direction. Conflicting requests are dropped, obviating the request\u27s receiving host from detecting and handling the conflict. AT is designed to perform well under high data contention, as concurrency control effort is balanced across a network instead of being handled by the contended endpoint hosts themselves. We use AT as the basis for a new optimistic transactional cache consistency algorithm, supporting execution of atomic applications caching shared data. We then present a scalable refinement, allowing hierarchical consistent caches with predictable performance despite high data update rates. We give detailed I/O Automata models of our algorithms along with correctness proofs. We begin with a simplified model, assuming static network paths and no message loss, and then refine it to support dynamic network paths and safe handling of message loss. We present a trie-based data structure for accelerating conflict-checking on switches, with benchmarks suggesting the feasibility of our approach from a performance stand-point

Decentralized Online Voting

Elections are crucial in any democratic nation. However, since election turnout levels in Europe have decreased in the last 25 years, studies have been conducted to understand the root causes of the problem and assess how voters could be convinced to vote. This document focused on the voters’ convenience factor by first performing an empirical review on how election systems are implemented in Europe, which methods exist, and analyzing how remote voting is deployed across European nations. The scope of this work has then been narrowed down to online voting. This document has studied how ballot data decentralization could contribute to an acceptable solution for online voting. As a result, the Solid project was studied to understand how it could fit into such a solution so that it could be used to store ballots under the responsibility of their respective voters. Such a study involved the design and implementation of a proof of concept to understand the feasibility of Solid for decentralizing ballot data. A survey was then conducted to evaluate the proof of concept from a voter perspective, having presented results that indicate that while online voting is deemed more convenient, respondents did not find significant value in storing ballots in a decentralized way using Solid.Eleições são um conceito crucial em qualquer nação democrática. No entanto, derivado do constante aumento da abstenção nos países europeus nos últimos 25 anos, estudos têm sido feitos no sentido de perceber quais os factores que causam este problema, e entender como é que os eleitores podem ser convencidos a votar. Este documento é focado na conveniência do eleitor. Numa primeira fase, é feita uma revisão empírica de como os sistemas eleitorais estão implementados na Europa, nomeadamente que métodos existem e como se comparam entre si. De seguida, é feita uma análise em como a votação remota está implantada em alguns países europeus, sendo posteriormente direccionada de forma mais concreta para a votação online. Este projeto procura estudar como é que os dados dos boletins de voto podem ser persistidos de forma descentralizada e se essa característica é viável no contexto de uma possível solução de votação online. Desta forma, o projecto Solid foi estudado para perceber se este poderia ser enquadrado nessa solução, de forma a que os boletins de voto pudessem ser guardados sob a responsabilidade dos seus respectivos eleitores. Esse estudo envolveu o desenho e implementação de uma prova de conceito que representasse essa solução. Após o seu desenvolvimento, a prova de conceito foi alvo de uma experiência. Os inquiridos experimentaram a solução e posteriormente responderam a um questionário sobre o funcionamento da prova de conceito e sobre a votação online em geral. Os resultados indicam que embora a votação online seja encarada como uma forma mais conveniente de votar do que presencialmente, os inquiridos não valorizam significativamente a persistência de boletins de voto de forma descentralizada

Designing key-value stores for emerging memory and disaggregation technologies

With the increasing convergence of applications to the cloud, cloud-based key-value stores (KVSs) should offer high performance, scalability, elasticity, utilization, and crash resilience. However, conventional storage technologies and monolithic server models make it challenging to achieve these goals. The transition to the new emerging memory and disaggregation technologies, such as PM (Persistent Memory), RDMA (Remote Direct Memory Access), and CXL (Compute Express Link), can readily offer opportunities to achieve these goals. However, these new technologies have distinct characteristics from the conventional technologies. Thus, to efficiently and reliably utilize them, KVSs must be carefully designed to avoid sub-optimal design choices without compromising their inherent hardware-guaranteed benefits. In this dissertation, we seek to answer the following question: how can we achieve a high-performance, scalable, elastic, and crash-recoverable KVS for disaggregated persistent memory (DPM)? In particular, we explore solutions to achieve these goals by introducing new indexing, caching, and partitioning techniques. We design new indexing data structures for a high-performance, scalable, and crash-recoverable data storage at PM, employ caching strategies for high performance by reducing expensive accesses to DPM, and tailor partitioning techniques to achieve elastic, scalable resource deployment. This dissertation first presents Recipe, a principled approach for converting concurrent DRAM indexes to crash-consistent indexes for PM. The main insight behind Recipe is that isolation provided by a certain class of concurrent DRAM indexes can be translated to crash consistency when the same index is used in PM. We present a set of conditions that enable the identification of this class of DRAM indexes, and the actions to be taken to convert each index to be persistent. Next, we presents Dinomo, the first key-value store for DPM based on RDMA interconnects that simultaneously achieves high common-case performance, scalability, and elasticity. Dinomo uses a novel combination of techniques such as ownership partitioning, disaggregated adaptive caching, selective replication, and lock-free and log-free PM indexing to achieve these goals. Finally, we present Shift, a cache-conscious KVS designs for CXL disaggregated memory. Shift sheds new light on the existing PM indexes and partitioning schemes originally proposed for the different system domains to achieve a high-performance, scalable, elastic, crash-recoverable KVS for CXL disaggregated memory. Furthermore, Shift employs lock intention log to improve the PM indexes to be partial-failure-resilient and non-hierarchical processing to take both advantages of KN cache and direct accesses to CXL disaggregated memory.Computer Science

Proposed AICPA Audit and Accounting Guide : Airlines;Airlines; Exposure draft (American Institute of Certified Public Accountants), 2007, Sept. 12

This guide does the following: (1) Identifies certain requirements set forth in pronouncements in levels a, b, and c of the hierarchy of sources of GAAP. In these instances, entities are required to apply those requirements based on their standing in levels a, b, or c of the hierarchy rather than because of their inclusion in this guide. (2) Describes AcSEC’s understanding of prevalent or sole industry practice concerning certain issues. In addition, this guide may indicate that AcSEC expresses a preference for the prevalent or sole industry practice, or it may indicate that AcSEC expresses a preference for another practice that is not the prevalent or sole industry practice; alternatively, AcSEC may express no view on the matter. (3) Identifies certain other, but not necessarily all, industry practices concerning certain accounting issues without expressing AcSEC’s views on them Provides guidance that has been supported by AcSEC on the accounting, reporting, or disclosure treatment of transactions or events that is not set forth in pronouncements in levels a, b, and c of the hierarchy of sources of GAAP.https://egrove.olemiss.edu/aicpa_sop/1335/thumbnail.jp

Wide-Area Situation Awareness based on a Secure Interconnection between Cyber-Physical Control Systems

Posteriormente, examinamos e identificamos los requisitos especiales que limitan el diseño y la operación de una arquitectura de interoperabilidad segura para los SSC (particularmente los SCCF) del smart grid. Nos enfocamos en modelar requisitos no funcionales que dan forma a esta infraestructura, siguiendo la metodología NFR para extraer requisitos esenciales, técnicas para la satisfacción de los requisitos y métricas para nuestro modelo arquitectural. Estudiamos los servicios necesarios para la interoperabilidad segura de los SSC del SG revisando en profundidad los mecanismos de seguridad, desde los servicios básicos hasta los procedimientos avanzados capaces de hacer frente a las amenazas sofisticadas contra los sistemas de control, como son los sistemas de detección, protección y respuesta ante intrusiones. Nuestro análisis se divide en diferentes áreas: prevención, consciencia y reacción, y restauración; las cuales general un modelo de seguridad robusto para la protección de los sistemas críticos. Proporcionamos el diseño para un modelo arquitectural para la interoperabilidad segura y la interconexión de los SCCF del smart grid. Este escenario contempla la interconectividad de una federación de proveedores de energía del SG, que interactúan a través de la plataforma de interoperabilidad segura para gestionar y controlar sus infraestructuras de forma cooperativa. La plataforma tiene en cuenta las características inherentes y los nuevos servicios y tecnologías que acompañan al movimiento de la Industria 4.0. Por último, presentamos una prueba de concepto de nuestro modelo arquitectural, el cual ayuda a validar el diseño propuesto a través de experimentaciones. Creamos un conjunto de casos de validación que prueban algunas de las funcionalidades principales ofrecidas por la arquitectura diseñada para la interoperabilidad segura, proporcionando información sobre su rendimiento y capacidades.Las infraestructuras críticas (IICC) modernas son vastos sistemas altamente complejos, que precisan del uso de las tecnologías de la información para gestionar, controlar y monitorizar el funcionamiento de estas infraestructuras. Debido a sus funciones esenciales, la protección y seguridad de las infraestructuras críticas y, por tanto, de sus sistemas de control, se ha convertido en una tarea prioritaria para las diversas instituciones gubernamentales y académicas a nivel mundial. La interoperabilidad de las IICC, en especial de sus sistemas de control (SSC), se convierte en una característica clave para que estos sistemas sean capaces de coordinarse y realizar tareas de control y seguridad de forma cooperativa. El objetivo de esta tesis se centra, por tanto, en proporcionar herramientas para la interoperabilidad segura de los diferentes SSC, especialmente los sistemas de control ciber-físicos (SCCF), de forma que se potencie la intercomunicación y coordinación entre ellos para crear un entorno en el que las diversas infraestructuras puedan realizar tareas de control y seguridad cooperativas, creando una plataforma de interoperabilidad segura capaz de dar servicio a diversas IICC, en un entorno de consciencia situacional (del inglés situational awareness) de alto espectro o área (wide-area). Para ello, en primer lugar, revisamos las amenazas de carácter más sofisticado que amenazan la operación de los sistemas críticos, particularmente enfocándonos en los ciberataques camuflados (del inglés stealth) que amenazan los sistemas de control de infraestructuras críticas como el smart grid. Enfocamos nuestra investigación al análisis y comprensión de este nuevo tipo de ataques que aparece contra los sistemas críticos, y a las posibles contramedidas y herramientas para mitigar los efectos de estos ataques