268 research outputs found
PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development
This paper describes PlinyCompute, a system for development of
high-performance, data-intensive, distributed computing tools and libraries. In
the large, PlinyCompute presents the programmer with a very high-level,
declarative interface, relying on automatic, relational-database style
optimization to figure out how to stage distributed computations. However, in
the small, PlinyCompute presents the capable systems programmer with a
persistent object data model and API (the "PC object model") and associated
memory management system that has been designed from the ground-up for high
performance, distributed, data-intensive computing. This contrasts with most
other Big Data systems, which are constructed on top of the Java Virtual
Machine (JVM), and hence must at least partially cede performance-critical
concerns such as memory management (including layout and de/allocation) and
virtual method/function dispatch to the JVM. This hybrid approach---declarative
in the large, trusting the programmer's ability to utilize PC object model
efficiently in the small---results in a system that is ideal for the
development of reusable, data-intensive tools and libraries. Through extensive
benchmarking, we show that implementing complex objects manipulation and
non-trivial, library-style computations on top of PlinyCompute can result in a
speedup of 2x to more than 50x or more compared to equivalent implementations
on Spark.Comment: 48 pages, including references and Appendi
Improving variability of applications using adaptive object-models
Tese de mestrado integrado. Engenharia Informática e Computação. Universidade do Porto. Faculdade de Engenharia. 201
A Context-Oriented Extension of F#
Context-Oriented programming languages provide us with primitive constructs
to adapt program behaviour depending on the evolution of their operational
environment, namely the context. In previous work we proposed ML_CoDa, a
context-oriented language with two-components: a declarative constituent for
programming the context and a functional one for computing. This paper
describes the implementation of ML_CoDa as an extension of F#.Comment: In Proceedings FOCLASA 2015, arXiv:1512.0694
Towards multilingual programming environments
Software projects consist of different kinds of artifacts: build files, configuration files, markup files, source code in different software languages, and so on. At the same time, however, most integrated development environments (IDEs) are focused on a single (programming) language. Even if a programming environment supports multiple languages (e.g., Eclipse), IDE features such as cross-referencing, refactoring, or debugging, do not often cross language boundaries. What would it mean for programming environment to be truly multilingual? In this short paper we sketch a vision of a system that integrates IDE support across language boundaries. We propose to build this system on a foundation of unified source code models and metaprogramming. Nevertheless, a number of important and hard research questions still need to be addressed
Towards Intelligent Databases
This article is a presentation of the objectives and techniques
of deductive databases. The deductive approach to databases aims at extending
with intensional definitions other database paradigms that describe
applications extensionaUy. We first show how constructive specifications can
be expressed with deduction rules, and how normative conditions can be defined
using integrity constraints. We outline the principles of bottom-up and
top-down query answering procedures and present the techniques used for
integrity checking. We then argue that it is often desirable to manage with
a database system not only database applications, but also specifications of
system components. We present such meta-level specifications and discuss
their advantages over conventional approaches
Constrained Query Answering
Traditional answering methods evaluate queries only against positive
and definite knowledge expressed by means of facts and deduction rules. They do
not make use of negative, disjunctive or existential information. Negative or indefinite
knowledge is however often available in knowledge base systems, either as
design requirements, or as observed properties. Such knowledge can serve to rule out
unproductive subexpressions during query answering. In this article, we propose an
approach for constraining any conventional query answering procedure with general,
possibly negative or indefinite formulas, so as to discard impossible cases and to
avoid redundant evaluations. This approach does not impose additional conditions
on the positive and definite knowledge, nor does it assume any particular semantics
for negation. It adopts that of the conventional query answering procedure it
constrains. This is achieved by relying on meta-interpretation for specifying the
constraining process. The soundness, completeness, and termination of the underlying
query answering procedure are not compromised. Constrained query answering
can be applied for answering queries more efficiently as well as for generating more
informative, intensional answers
The load shedding advisor: An example of a crisis-response expert system
A Prolog-based prototype expert system is described that was implemented by the Network Operations Branch of the NASA Goddard Space Flight Center. The purpose of the prototype was to test whether a small, inexpensive computer system could be used to host a load shedding advisor, a system which would monitor major physical environment parameters in a computer facility, then recommend appropriate operator reponses whenever a serious condition was detected. The resulting prototype performed significantly to efficiency gains achieved by replacing a purely rule-based design methodology with a hybrid approach that combined procedural, entity-relationship, and rule-based methods
Language-integrated provenance in Haskell
Scientific progress increasingly depends on data management, particularly to
clean and curate data so that it can be systematically analyzed and reused. A
wealth of techniques for managing and curating data (and its provenance) have
been proposed, largely in the database community. In particular, a number of
influential papers have proposed collecting provenance information explaining
where a piece of data was copied from, or what other records were used to
derive it. Most of these techniques, however, exist only as research prototypes
and are not available in mainstream database systems. This means scientists
must either implement such techniques themselves or (all too often) go without.
This is essentially a code reuse problem: provenance techniques currently
cannot be implemented reusably, only as ad hoc, usually unmaintained extensions
to standard databases. An alternative, relatively unexplored approach is to
support such techniques at a higher abstraction level, using metaprogramming or
reflection techniques. Can advanced programming techniques make it easier to
transfer provenance research results into practice?
We build on a recent approach called language-integrated provenance, which
extends language-integrated query techniques with source-to-source query
translations that record provenance. In previous work, a proof of concept was
developed in a research programming language called Links, which supports
sophisticated Web and database programming. In this paper, we show how to adapt
this approach to work in Haskell building on top of the Database-Supported
Haskell (DSH) library.
Even though it seemed clear in principle that Haskell's rich programming
features ought to be sufficient, implementing language-integrated provenance in
Haskell required overcoming a number of technical challenges due to
interactions between these capabilities. Our implementation serves as a proof
of concept showing how this combination of metaprogramming features can, for
the first time, make data provenance facilities available to programmers as a
library in a widely-used, general-purpose language.
In our work we were successful in implementing forms of provenance known as
where-provenance and lineage. We have tested our implementation using a simple
database and query set and established that the resulting queries are executed
correctly on the database. Our implementation is publicly available on GitHub.
Our work makes provenance tracking available to users of DSH at little cost.
Although Haskell is not widely used for scientific database development, our
work suggests which languages features are necessary to support provenance as
library. We also highlight how combining Haskell's advanced type programming
features can lead to unexpected complications, which may motivate further
research into type system expressiveness
Language-integrated provenance by trace analysis
Language-integrated provenance builds on language-integrated query techniques
to make provenance information explaining query results readily available to
programmers. In previous work we have explored language-integrated approaches
to provenance in Links and Haskell. However, implementing a new form of
provenance in a language-integrated way is still a major challenge. We propose
a self-tracing transformation and trace analysis features that, together with
existing techniques for type-directed generic programming, make it possible to
define different forms of provenance as user code. We present our design as an
extension to a core language for Links called LinksT, give examples showing its
capabilities, and outline its metatheory and key correctness properties.Comment: DBPL 201
- …