This Time It's Personal: from PIM to the Perfect Digital Assistant

Interacting with digital PIM tools like calendars, to-do lists, address books, bookmarks and so on, is a highly manual, often repetitive and frequently tedious process. Despite increases in memory and processor power over the past two decades of personal computing, not much has changed in the way we engage with such applications. We must still manually decompose frequently performed tasks into multiple smaller, data specific processes if we want to be able to recall or reuse the information in some meaningful way. "Meeting with Yves at 5 in Stata about blah" breaks down into rigid, fixed semantics in separate applications: data to be recorded in calendar fields, address book fields and, as for the blah, something that does not necessarily exist as a PIM application data structure. We argue that a reason Personal Information Management tools may be so manual, and so effectively fragmented, is that they are not personal enough. If our information systems were more personal, that is, if they knew in a manner similar to the way a personal assistant would know us and support us, then our tools would be more helpful: an assistive PIM tool would gather together the necessary material in support of our meeting with Yves. We, therefore, have been investigating the possible paths towards PIM tools as tools that work for us, rather than tools that seemingly make us work for them. To that end, in the following sections we consider how we may develop a framework for PIM tools as "perfect digital assistants" (PDA). Our impetus has been to explore how, by considering the affordances of a Real World personal assistant, we can conceptualize a design framework, and from there a development program for a digital simulacrum of such an assistant that is not for some far off future, but for the much nearer term

Compression of Structured High-Throughput Sequencing Data

Large biological datasets are being produced at a rapid pace and create substantial storage challenges, particularly in the domain of high-throughput sequencing (HTS). Most approaches currently used to store HTS data are either unable to quickly adapt to the requirements of new sequencing or analysis methods (because they do not support schema evolution), or fail to provide state of the art compression of the datasets. We have devised new approaches to store HTS data that support seamless data schema evolution and compress datasets substantially better than existing approaches. Building on these new approaches, we discuss and demonstrate how a multi-tier data organization can dramatically reduce the storage, computational and network burden of collecting, analyzing, and archiving large sequencing datasets. For instance, we show that spliced RNA-Seq alignments can be stored in less than 4% the size of a BAM file with perfect data fidelity. Compared to the previous compression state of the art, these methods reduce dataset size more than 40% when storing exome, gene expression or DNA methylation datasets. The approaches have been integrated in a comprehensive suite of software tools (http://goby.campagnelab.org) that support common analyses for a range of high-throughput sequencing assays.National Center for Research Resources (U.S.) (Grant UL1 RR024996)Leukemia & Lymphoma Society of America (Translational Research Program Grant LLS 6304-11)National Institute of Mental Health (U.S.) (R01 MH086883

Combining Provenance Management and Schema Evolution

The combination of provenance management and schema evolution using the CHASE algorithm is the focus of our research in the area of research data management. The aim is to combine the construc- tion of a CHASE inverse mapping to calculate the minimal part of the original database — the minimal sub-database — with a CHASE-based schema mapping for schema evolution

Migrating relational data to an OODB: strategies and lessions from a molecular biology experience

Journal ArticleThe growing maturity of OODB technology is causing many enterprises to consider migrating relational databases to OODBs. While data remapping is relatively straightforward, greater challenges lie in economically and non-invasively adapting legacy application software. We report on a genetics laboratory database migration experiment, which was facilitated by both organization of the relational data in object-like form and a C++ framework designed to insulate application code from relational artifacts. To our surprise, the framework failed to encapsulate three subtle aspects of the relational implementation, thereby "contaminating" application code. We describe the underlying issues, and offer cautionary guidance to future migrators

A Serverless Distributed Ledger for Enterprises

Enterprises have been attracted by the capability of blockchains to provide a single source of truth for workloads that span companies, geographies, and clouds while retaining the independence of each party’s IT operations. However, so far production applications have remained rare, stymied by technical limitations of existing blockchain technologies and challenges with their integration into enterprises' IT systems. In this paper, we collect enterprises' requirements on distributed ledgers for data sharing and integration from a technical perspective, argue that they are not sufficiently addressed by available blockchain frameworks, and propose a novel distributed ledger design that is ``serverless'', i.e., built on cloud-native resources. We evaluate its qualitative and quantitative properties and give evidence that enterprises already heavily reliant on cloud service providers would consider such an approach acceptable, particularly if it offers ease of deployment, low transactional cost structure, and a combination of latency and scalability aligned with real-time IT application needs

Data and Query Adaptation Using DaemonX

The most common applications of the today's IT world are information systems. The problems related to their design and implementation have sufficiently been solved. However, the true problems occur when an IS is already deployed and user requirements change. In this paper we introduce DaemonX - an evolution management framework which enables to manage evolution of complex applications efficiently and correctly. Using the idea of plug-ins, it enables to model almost any kind of a data format (currently XML, UML, ER, and BPMN). Since it preserves also mapping among modeled constructs of modeled formats via a common platform-independent model, it naturally supports propagation of changes to all related and affected parts

Robust Contract Evolution in a TypeSafe MicroServices Architecture

Microservices architectures allow for short deployment cycles and immediate effects but offer no safety mechanisms when service contracts need to be changed. Maintaining the soundness of microservice architectures is an error-prone task that is only accessible to the most disciplined development teams. We present a microservice management system that statically verifies service interfaces and supports the seamless evolution of compatible interfaces. We define a compatibility relation that captures real evolution patterns and embodies known good practices on the evolution of interfaces. Namely, we allow for the addition, removal, and renaming of data fields of a producer module without breaking or needing to upgrade consumer services. The evolution of interfaces is supported by runtime generated proxy components that dynamically adapt data exchanged between services to match with the statically checked service code.The model was instantiated in a core language whose semantics is defined by a labeled transition system and a type system that prevents breaking changes from being deployed. Standard soundness results for the core language entail the existence of adapters, hence the absence of adaptation errors and the correctness of the management model. This adaptive approach allows for gradual deployment of modules, without halting the whole system and avoiding losing or misinterpreting data exchanged between system nodes. Experimental data shows that an average of 69% of deployments that would require adaptation and recompilation are safe under our approach