2,493 research outputs found
Weaving Rules into [email protected] for Embedded Smart Systems
Smart systems are characterised by their ability to analyse measured data in
live and to react to changes according to expert rules. Therefore, such systems
exploit appropriate data models together with actions, triggered by
domain-related conditions. The challenge at hand is that smart systems usually
need to process thousands of updates to detect which rules need to be
triggered, often even on restricted hardware like a Raspberry Pi. Despite
various approaches have been investigated to efficiently check conditions on
data models, they either assume to fit into main memory or rely on high latency
persistence storage systems that severely damage the reactivity of smart
systems. To tackle this challenge, we propose a novel composition process,
which weaves executable rules into a data model with lazy loading abilities. We
quantitatively show, on a smart building case study, that our approach can
handle, at low latency, big sets of rules on top of large-scale data models on
restricted hardware.Comment: pre-print version, published in the proceedings of MOMO-17 Worksho
A Delta Debugger for ILP Query Execution
Because query execution is the most crucial part of Inductive Logic
Programming (ILP) algorithms, a lot of effort is invested in developing faster
execution mechanisms. These execution mechanisms typically have a low-level
implementation, making them hard to debug. Moreover, other factors such as the
complexity of the problems handled by ILP algorithms and size of the code base
of ILP data mining systems make debugging at this level a very difficult job.
In this work, we present the trace-based debugging approach currently used in
the development of new execution mechanisms in hipP, the engine underlying the
ACE Data Mining system. This debugger uses the delta debugging algorithm to
automatically reduce the total time needed to expose bugs in ILP execution,
thus making manual debugging step much lighter.Comment: Paper presented at the 16th Workshop on Logic-based Methods in
Programming Environments (WLPE2006
Proceedings of the 3rd Workshop on Domain-Specific Language Design and Implementation (DSLDI 2015)
The goal of the DSLDI workshop is to bring together researchers and
practitioners interested in sharing ideas on how DSLs should be designed,
implemented, supported by tools, and applied in realistic application contexts.
We are both interested in discovering how already known domains such as graph
processing or machine learning can be best supported by DSLs, but also in
exploring new domains that could be targeted by DSLs. More generally, we are
interested in building a community that can drive forward the development of
modern DSLs. These informal post-proceedings contain the submitted talk
abstracts to the 3rd DSLDI workshop (DSLDI'15), and a summary of the panel
discussion on Language Composition
The Family of MapReduce and Large Scale Data Processing Systems
In the last two decades, the continuous increase of computational power has
produced an overwhelming flow of data which has called for a paradigm shift in
the computing architecture and large scale data processing mechanisms.
MapReduce is a simple and powerful programming model that enables easy
development of scalable parallel applications to process vast amounts of data
on large clusters of commodity machines. It isolates the application from the
details of running a distributed program such as issues on data distribution,
scheduling and fault tolerance. However, the original implementation of the
MapReduce framework had some limitations that have been tackled by many
research efforts in several followup works after its introduction. This article
provides a comprehensive survey for a family of approaches and mechanisms of
large scale data processing mechanisms that have been implemented based on the
original idea of the MapReduce framework and are currently gaining a lot of
momentum in both research and industrial communities. We also cover a set of
introduced systems that have been implemented to provide declarative
programming interfaces on top of the MapReduce framework. In addition, we
review several large scale data processing systems that resemble some of the
ideas of the MapReduce framework for different purposes and application
scenarios. Finally, we discuss some of the future research directions for
implementing the next generation of MapReduce-like solutions.Comment: arXiv admin note: text overlap with arXiv:1105.4252 by other author
Write-limited sorts and joins for persistent memory
To mitigate the impact of the widening gap between the memory needs of CPUs and what standard memory technology can deliver, system architects have introduced a new class of memory technology termed persistent memory. Persistent memory is byteaddressable, but exhibits asymmetric I/O: writes are typically one order of magnitude more expensive than reads. Byte addressability combined with I/O asymmetry render the performance profile of persistent memory unique. Thus, it becomes imperative to find new ways to seamlessly incorporate it into database systems. We do so in the context of query processing. We focus on the fundamental operations of sort and join processing. We introduce the notion of write-limited algorithms that effectively minimize the I/O cost. We give a high-level API that enables the system to dynamically optimize the workflow of the algorithms; or, alternatively, allows the developer to tune the write profile of the algorithms. We present four different techniques to incorporate persistent memory into the database processing stack in light of this API. We have implemented and extensively evaluated all our proposals. Our results show that the algorithms deliver on their promise of I/O-minimality and tunable performance. We showcase the merits and deficiencies of each implementation technique, thus taking a solid first step towards incorporating persistent memory into query processing. 1
- …