LeCo: Lightweight Compression via Learning Serial Correlations

Lightweight data compression is a key technique that allows column stores to exhibit superior performance for analytical queries. Despite a comprehensive study on dictionary-based encodings to approach Shannon's entropy, few prior works have systematically exploited the serial correlation in a column for compression. In this paper, we propose LeCo (i.e., Learned Compression), a framework that uses machine learning to remove the serial redundancy in a value sequence automatically to achieve an outstanding compression ratio and decompression performance simultaneously. LeCo presents a general approach to this end, making existing (ad-hoc) algorithms such as Frame-of-Reference (FOR), Delta Encoding, and Run-Length Encoding (RLE) special cases under our framework. Our microbenchmark with three synthetic and six real-world data sets shows that a prototype of LeCo achieves a Pareto improvement on both compression ratio and random access speed over the existing solutions. When integrating LeCo into widely-used applications, we observe up to 3.9x speed up in filter-scanning a Parquet file and a 16% increase in Rocksdb's throughput

Global Title X Series \u2713: Game Report

The Chief of Naval Operations’ (CNO’s) annual Title 10 War Game (also known as Global) conducted at the Naval War College (NWC) has become a primary venue for exploring emerging concepts. This year’s effort is a continuation of the NWC War Gaming Department’s examination of the Air-Sea Battle (ASB) concept. The 2012 Global War Game concluded that current command and control (C2) structures at the operational level of war may be inadequate to effectively execute cross-domain operations as envisioned by the concept. While the ASB concept outlines the need to command and control ‘cross-domain operations’ which are joint, networked and integrated, no organizational structure is proposed. The concept only suggests that any suitable structure must be capable of tight, real-time coordination

Extracting and utilizing hidden structures in large datasets

The hidden structure within datasets --- capturing the inherent structure within the data not explicitly captured or encoded in the data format --- can often be automatically extracted and used to improve various data processing applications. Utilizing such hidden structure enables us to potentially surpass traditional algorithms that do not take this structure into account. In this thesis, we propose a general framework for algorithms that automatically extract and employ hidden structures to improve data processing performance, and discuss a set of design principles for developing such algorithms. We provide three examples to demonstrate the power of this framework in practice, showcasing how we can use hidden structures to either outperform state-of-the-art methods, or enable new applications that are previously impossible. We believe that this framework can offer new opportunities for the design of algorithms that surpass the current limit, and empower new applications in database research and many other data-centric disciplines

Towards (R)evolving Cities Urban fragilities and prospects in the 21st century

Towards (R)evolving Cities: Urban Fragilities and Prospects in the 21st century first questions how we perceive the ‘intelligence’ of a city. The New Frontier of development for urban civilisations certainly includes digital and technological evolution, but it does not consider technology to be the final answer to all contemporary cities’ problems. The formidable challenges of the COVID-19 pandemic have thrown existing urban fragilities into stark relief. At the same time however they have highlighted the potential of digital solutions for reaching a new level of interconnected civility. (R)evolving cities evolve by adopting the principles of the circular economy in the higher interest of their citizens’ well-being: they consume therefore without devouring, recycle as much as possible what they metabolize, limit the effects of their ecological footprint and ultimately lead their inhabitants, with maternal guidance and care, to a new idea of citizenship. As protagonists of this evolutionary leap, the citizens of (R)evolving cities will abandon their predatory approach, reaching a higher stage of integration in the ecosystem and becoming more respectful of reciprocal relationships. (R)evolving cities are above all ‘polite’ cities, or rather cities whose citizens are consciously educated in the principles of sustainable development, the essential basis for contemporary civil coexistence

Knowledge management solutions and selection tool for engineering organisations

It is widely accepted that engineering research, design, development and manufacturing processes are highly reliant upon the valuable knowledge, experiences and skills stored within the company's systems, processes, documents and employees. If these key knowledge resources can be identified, maintained and efficiently controlled, prior successes and failures can be capitalised upon, best practices can be captured and transferred and new solutions can be developed with minimal duplication of efforts and without unnecessary replication of prior work. Away from manufacturing and engineering organisations, in the broader business world, exists an array of solutions, tools and techniques developed specifically to facilitate the management of knowledge and experience these are collectively labelled as Knowledge Management (KM) tools and solutions. Such solutions, tools and techniques have achieved widespread recognition for their capabilities and consequent importance in enhancing processes across a variety of business applications and contexts. However their relevancy, applicability and relative merits in particular manufacturing and mechanical engineering (MME) contexts have generally not been identified or investigated. This thesis reviews and presents a large number of diverse KM solutions and implementations across industries and organisations and creates a new and unique single KM solutions space in which these solutions are characterised. The KM solution space is subsequently utilised by a new KM methodology and support tool that facilitates and demonstrates the enhancement of mechanical and manufacturing engineering processes through analysis followed by selection and implementation of the most appropriate existing KM solutions. The KM Tool is demonstrated via three industrial case studies detailing the process concerns and associated improvements identified and implemented. The KM Solution Space developed during this research has shown that there is significant opportunity to improve mechanical and manufacturing engineering processes through the adoption of appropriate KM solutions from the broader business world. The KM Tool developed via this research facilitates this identification and adoption of the most appropriate KM solution. In addition to the MME processes covered by the scope of this research there is additional scope to extend the use of the KM Tool and KM Solution Space to other business areas that have not yet had extensive exposure to KM

Explicit or Symbolic Translation of Linear Temporal Logic to Automata

Formal verification techniques are growing increasingly vital for the development of safety-critical software and hardware in practice. Techniques such as requirements-based design and model checking for system verification have been successfully used to verify systems for air traffic control, airplane separation assurance, autopilots, CPU logic designs, life-support, medical equipment, and other functions that ensure human safety. Formal behavioral specifications written early in the system-design process and communicated across all design phases increase the efficiency, consistency, and quality of the system under development. We argue that to prevent introducing design or verification errors, it is crucial to test specifications for satisfiability. We advocate for the adaptation of a new sanity check via satisfiability checking for property assurance. Our focus here is on specifications expressed in Linear Temporal Logic (LTL). We demonstrate that LTL satisfiability checking reduces to model checking and satisfiability checking for the specification, its complement, and a conjunction of all properties should be performed as a first step to LTL model checking. We report on an experimental investigation of LTL satisfiability checking. We introduce a large set of rigorous benchmarks to enable objective evaluation of LTL-to-automaton algorithms in terms of scalability, performance, correctness, and size of the automata produced. For explicit model checking, we use the Spin model checker; we tested all LTL-to-explicit automaton translation tools that were publicly available when we conducted our study. For symbolic model checking, we use CadenceSMV, NuSMV, and SAL-SMC for both LTL-to-symbolic automaton translation and to perform the satisfiability check. Our experiments result in two major findings. First, scalability, correctness, and other debilitating performance issues afflict most LTL translation tools. Second, for LTL satisfiability checking, the symbolic approach is clearly superior to the explicit approach. Ironically, the explicit approach to LTL-to-automata had been heavily studied while only one algorithm existed for LTL-to-symbolic automata. Since 1994, there had been essentially no new progress in encoding symbolic automata for BDD-based analysis. Therefore, we introduce a set of 30 symbolic automata encodings. The set consists of novel combinations of existing constructs, such as different LTL formula normal forms, with a novel transition-labeled symbolic automaton form, a new way to encode transitions, and new BDD variable orders based on algorithms for tree decomposition of graphs. An extensive set of experiments demonstrates that these encodings translate to significant, sometimes exponential, improvement over the current standard encoding for symbolic LTL satisfiability checking. Building upon these ideas, we return to the explicit automata domain and focus on the most common type of specifications used in industrial practice: safety properties. We show that we can exploit the inherent determinism of safety properties to create a set of 26 explicit automata encodings comprised of novel aspects including: state numbers versus state labels versus a state look-up table, finite versus infinite acceptance conditions, forward-looking versus backward-looking transition encodings, assignment-based versus BDD-based alphabet representation, state and transition minimization, edge abbreviation, trap-state elimination, and determinization either on-the-fly or up-front using the subset construction. We conduct an extensive experimental evaluation and identify an encoding that offers the best performance in explicit LTL model checking time and is constantly faster than the previous best explicit automaton encoding algorithm

Fifth NASA Goddard Conference on Mass Storage Systems and Technologies

This document contains copies of those technical papers received in time for publication prior to the Fifth Goddard Conference on Mass Storage Systems and Technologies held September 17 - 19, 1996, at the University of Maryland, University Conference Center in College Park, Maryland. As one of an ongoing series, this conference continues to serve as a unique medium for the exchange of information on topics relating to the ingestion and management of substantial amounts of data and the attendant problems involved. This year's discussion topics include storage architecture, database management, data distribution, file system performance and modeling, and optical recording technology. There will also be a paper on Application Programming Interfaces (API) for a Physical Volume Repository (PVR) defined in Version 5 of the Institute of Electrical and Electronics Engineers (IEEE) Reference Model (RM). In addition, there are papers on specific archives and storage products

Incomplete Innovation and the Premature Disruption of Legal Services

Article published in the Michigan State Law Review