Constructing provenance-aware distributed systems with data propagation

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2010.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student submitted PDF version of thesis.Includes bibliographical references (p. 93-96).Is it possible to construct a heterogeneous distributed computing architecture capable of solving interesting complex problems? Can we easily use this architecture to maintain a detailed history or provenance of the data processed by it? Most existing distributed architectures can perform only one operation at a time. While they are capable of tracing possession of data, these architectures do not always track the network of operations used to synthesize new data. This thesis presents a distributed implementation of data propagation, a computational model that provides for concurrent processing that is not constrained to a single distributed operation. This system is capable of distributing computation across a heterogeneous network. It allows for the division of multiple simultaneous operations in a single distributed system. I also identify four constraints that may be placed on general-purpose data propagation to allow for deterministic computation in such a distributed propagation network. This thesis also presents an application of distributed propagation by illustrating how a generic transformation may be applied to existing propagator networks to allow for the maintenance of data provenance. I show that the modular structure of data propagation permits the simple modification of a propagator network design to maintain the histories of data.by Ian Campbell Jacobi.S.M

Automated testing of Hypermedia REST applications

Testimine on oluline osa tarkvaraarenduse elutsüklis ja testidel põhinev arendamine on üks peamistest praktikatest Agile metoodikas. Tihti keskenduvad programmeerijad RESTful rakenduse loomise protsessis äriloogika testimisele ja unustavad kontrollida protokolli, mis teostab REST interaktsioone. Selles kontekstis pakutakse välja tööriist, mis automatiseerib testide genereerimist ja teostab interaktsioone RESTful rakendusega. Tööriist võtab sisendiks kasutuslood, mis on koostatud Gherkini kitsendatud versiooniga. See on domeenispetsiifiline keel käitumispõhiseks arenduseks. Kasutuslood, mis on kirjutatud selles Gherkini variandis, hõlmavad REST rakenduse poolt nõutud interaktsioone sellisel viisil, et neist on võimalik genereerida teste. Veelgi enam, tööriist genereerib samalt kasutusloolt täisfunktsionaalse pseudoteostuse.\n\rProgrammeerijad saavad kasutada neid pseudoteostusi kliendipoole arendamiseks, vajamata REST rakenduse tegelikku teostust. Käesolev töö tutvustab tööriista kasutust ja disainiprintsiipe ning esitab näite selle kasutamisest.Testing is one essential part of the software development lifecycle and Test Driven Development is one of the main practices in agile methodology. During the development of a RESTful web application, developers oftentimes focus only in testing the business logic and neglect testing the protocol implementing REST interactions. In this context, we propose a tool to automate the generation of test cases that exercise the interactions required by a RESTful application. The tool takes as input user stories written in restricted version of Gherkin, a widely use domain specific language for behaviour driven development. User stories written in this variant of Gherkin capture the essence of the interactions required by a REST application in a way that it is possible to derive test cases from them. Moreover, the tool derives fully functional mock implementations from the same input user story. Such mock implementations can be then used by programmers to develop the client side without requiring the actual implementation of the REST application. This document introduces the design principles and implementation of the tool and presents a study case showcasing its use

RECODE: Revision Control for Digital Images

Revision control is a vital component in the collaborative development of artifacts such as software code and multimedia. While revision control has been widely deployed for text files, very few attempts to control the versioning of binary files can be found in the literature. This can be inconvenient for multimedia applications that use a significant amount of binary data, such as images, videos, meshes, and animations. Existing strategies such as storing whole files for individual revisions or simple binary deltas, respectively consume significant storage and complex semantic information. To overcome these limitations, in this paper we present RECODE, a revision control system for digital images. It stores revisions in the form of a DAG (directed acyclic graph) where nodes represent editing operations, and edges describe the spatial and temporal relationships between operations. Being integrated with GitHub, the largest project hosting platform, RECODE also facilitates the artistic creation process of distributed teams with different workflows that include image editing and digital painting. A preliminary user study was performed to assess the perceived usability of the proposed system

Gitana: a SQL-based Git Repository Inspector

International audienceSoftware development projects are notoriously complex and difficult to deal with. Several support tools such as issue tracking, code review and Source Control Management (SCM) systems have been introduced in the past decades to ease development activities. While such tools efficiently track the evolution of a given aspect of the project (e.g., bug reports), they provide just a partial view of the project and often lack of advanced querying mechanisms limiting themselves to command line or simple GUI support. This is particularly true for projects that rely on Git, the most popular SCM system today. In this paper, we propose a conceptual schema for Git and an approach that, given a Git repository, exports its data to a relational database in order to (1) promote data integration with other existing SCM tools and (2) enable writing queries on Git data using standard SQL syntax. To ensure efficiency, our approach comes with an incremental propagation mechanism that refreshes the database content with the latest modifications. We have implemented our approach in Gitana, an open-source tool available on GitHub

Composing a Web of Audio Applications

International audienceThe Web offers a great opportunity to share, deploy and use programs without installation difficulties. In this article we explore the idea of freely combining/composing real-time audio applications deployed on the Web using Faust audio DSP language

Collaborative Open Data versioning: a pragmatic approach using Linked Data

Most Open Government Data initiatives are centralised and unidirectional (i.e., they release data dumps in CSV or PDF format). Hence for non trivial applications reusers make copies of the government datasets to curate their local data copy. This situation is not optimal as it leads to duplication of efforts and reduces the possibility of sharing improvements. To improve the usefulness of publishing open data, several authors recommeded to use standard formats and data versioning. Here we focus on publishing versioned open linked data (i.e., in RDF format) because they allow one party to annotate data released independently by another party thus reducing the need to duplicate entire datasets. After describing a pipeline to open up legacy-databases data in RDF format, we argue that RDF is suitable to implement a scalable feedback channel, and we investigate what steps are needed to implement a distributed RDFversioning system in production

Decentralized linked open data in constrained wireless sensor networks

Data generated by sensors in Internet of Things ecosystems contains lots of valuable information, which is often not used to its full potential. This is mainly due to the fact that data is stored in proprietary storages and formats. Manufacturers of sensor devices often offer closed platforms to view and manage the data, which limits their reusability. Moreover, questions start to raise about true data ownership over data generated from monitoring our everyday lives. In order to overcome these issues several initiatives have emerged in the past to hand over data to the rightful owner. One of these initiatives is Solid, currently focusing on socially linked data. However, never before did one apply the Solid principles to Internet of Things data. Therefore, in this paper, a novel approach is presented where sensor data is handled from sensor to storage using open data formats and standards to ensure interoperability and reusability. It is shown that combining existing concepts can be helpful in designing decentralized Internet of Things data storages, on top of which data can be incorporated into the Linked Open Data cloud. This has been done by comparing the overhead of a regular approach, using Linked Open Data concepts on top of a sensor device, to an approach that was optimized for device management in constrained Internet of Things networks

Efficiency of an E-Commerce Web Application with MERN Stack and Modern Tools

The majority of people in today's generation use technology to manage their life and take care of their basic necessities. Many of us in our generation shop for clothing, groceries, and electronics and even fancy items via e-commerce websites.  We created a single chocolatier e-commerce web application utilizing the MERN stack, which consists of the MongoDB database, the Express.JS framework, the React.JS library, and the Node.JS platform. This program has several views for users and administrators, is fully functioning with 8 main functions.   Depending on the interests of the consumer, we can purchase many chocolate types and brands through this website. In this project, we have the option to add and remove, edit, and save various products. For the website, we have created administrative features including login and logout options, admin dashboard, category and brand management for customers, payment method options for carts, review, and promotion management. Customers could search, sort, filter, and add goods to the cart. The customer can pay and purchase the things when the bills are created based on the cart contents. On the other hand, we have used the modern tools which are necessary to improve the functional and non-functional requirements of the application

Mapping the NGSI-LD Context Model on Top of a SPARQL Event Processing Architecture: Implementation Guidelines

NGSI-LD is an open specification released by ETSI which proposes an information model and an API for an easy to use and standard management of context information. The NGSI-LD information model is framed within an ontology and adopts JSON-LD as serialization format for context information. This paper presents an approach to the implementation of the NGSI-LD specification over a SPARQL Event Processing Architecture. This work is being developed within the European-Brasilian H2020 SWAMP project focused on implementing an Internet of Things platform providing services for smart water management in agriculture