125 research outputs found
Database Queries that Explain their Work
Provenance for database queries or scientific workflows is often motivated as
providing explanation, increasing understanding of the underlying data sources
and processes used to compute the query, and reproducibility, the capability to
recompute the results on different inputs, possibly specialized to a part of
the output. Many provenance systems claim to provide such capabilities;
however, most lack formal definitions or guarantees of these properties, while
others provide formal guarantees only for relatively limited classes of
changes. Building on recent work on provenance traces and slicing for
functional programming languages, we introduce a detailed tracing model of
provenance for multiset-valued Nested Relational Calculus, define trace slicing
algorithms that extract subtraces needed to explain or recompute specific parts
of the output, and define query slicing and differencing techniques that
support explanation. We state and prove correctness properties for these
techniques and present a proof-of-concept implementation in Haskell.Comment: PPDP 201
The lifecycle of provenance metadata and its associated challenges and opportunities
This chapter outlines some of the challenges and opportunities associated
with adopting provenance principles and standards in a variety of disciplines,
including data publication and reuse, and information sciences
Distilling Structure in Scientific Workflows
International audienceIn this work, we have conducted a series of experiments to better understand the structure of scientific workflows. In particular, we have investigated techniques to understand why scientific workflows may or may not have a series-parallel structure
DataHub: Collaborative Data Science & Dataset Version Management at Scale
Relational databases have limited support for data collaboration, where teams
collaboratively curate and analyze large datasets. Inspired by software version
control systems like git, we propose (a) a dataset version control system,
giving users the ability to create, branch, merge, difference and search large,
divergent collections of datasets, and (b) a platform, DataHub, that gives
users the ability to perform collaborative data analysis building on this
version control system. We outline the challenges in providing dataset version
control at scale.Comment: 7 page
- …