12,534 research outputs found
Knowledge-infused and Consistent Complex Event Processing over Real-time and Persistent Streams
Emerging applications in Internet of Things (IoT) and Cyber-Physical Systems
(CPS) present novel challenges to Big Data platforms for performing online
analytics. Ubiquitous sensors from IoT deployments are able to generate data
streams at high velocity, that include information from a variety of domains,
and accumulate to large volumes on disk. Complex Event Processing (CEP) is
recognized as an important real-time computing paradigm for analyzing
continuous data streams. However, existing work on CEP is largely limited to
relational query processing, exposing two distinctive gaps for query
specification and execution: (1) infusing the relational query model with
higher level knowledge semantics, and (2) seamless query evaluation across
temporal spaces that span past, present and future events. These allow
accessible analytics over data streams having properties from different
disciplines, and help span the velocity (real-time) and volume (persistent)
dimensions. In this article, we introduce a Knowledge-infused CEP (X-CEP)
framework that provides domain-aware knowledge query constructs along with
temporal operators that allow end-to-end queries to span across real-time and
persistent streams. We translate this query model to efficient query execution
over online and offline data streams, proposing several optimizations to
mitigate the overheads introduced by evaluating semantic predicates and in
accessing high-volume historic data streams. The proposed X-CEP query model and
execution approaches are implemented in our prototype semantic CEP engine,
SCEPter. We validate our query model using domain-aware CEP queries from a
real-world Smart Power Grid application, and experimentally analyze the
benefits of our optimizations for executing these queries, using event streams
from a campus-microgrid IoT deployment.Comment: 34 pages, 16 figures, accepted in Future Generation Computer Systems,
October 27, 201
Towards Ideal Semantics for Analyzing Stream Reasoning
The rise of smart applications has drawn interest to logical reasoning over
data streams. Recently, different query languages and stream
processing/reasoning engines were proposed in different communities. However,
due to a lack of theoretical foundations, the expressivity and semantics of
these diverse approaches are given only informally. Towards clear
specifications and means for analytic study, a formal framework is needed to
define their semantics in precise terms. To this end, we present a first step
towards an ideal semantics that allows for exact descriptions and comparisons
of stream reasoning systems.Comment: International Workshop on Reactive Concepts in Knowledge
Representation (ReactKnow 2014), co-located with the 21st European Conference
on Artificial Intelligence (ECAI 2014). Proceedings of the International
Workshop on Reactive Concepts in Knowledge Representation (ReactKnow 2014),
pages 17-22, technical report, ISSN 1430-3701, Leipzig University, 2014.
http://nbn-resolving.de/urn:nbn:de:bsz:15-qucosa-150562 2014,
Weaving Rules into [email protected] for Embedded Smart Systems
Smart systems are characterised by their ability to analyse measured data in
live and to react to changes according to expert rules. Therefore, such systems
exploit appropriate data models together with actions, triggered by
domain-related conditions. The challenge at hand is that smart systems usually
need to process thousands of updates to detect which rules need to be
triggered, often even on restricted hardware like a Raspberry Pi. Despite
various approaches have been investigated to efficiently check conditions on
data models, they either assume to fit into main memory or rely on high latency
persistence storage systems that severely damage the reactivity of smart
systems. To tackle this challenge, we propose a novel composition process,
which weaves executable rules into a data model with lazy loading abilities. We
quantitatively show, on a smart building case study, that our approach can
handle, at low latency, big sets of rules on top of large-scale data models on
restricted hardware.Comment: pre-print version, published in the proceedings of MOMO-17 Worksho
Semantic-driven Configuration of Internet of Things Middleware
We are currently observing emerging solutions to enable the Internet of
Things (IoT). Efficient and feature rich IoT middeware platforms are key
enablers for IoT. However, due to complexity, most of these middleware
platforms are designed to be used by IT experts. In this paper, we propose a
semantics-driven model that allows non-IT experts (e.g. plant scientist, city
planner) to configure IoT middleware components easier and faster. Such tools
allow them to retrieve the data they want without knowing the underlying
technical details of the sensors and the data processing components. We propose
a Context Aware Sensor Configuration Model (CASCoM) to address the challenge of
automated context-aware configuration of filtering, fusion, and reasoning
mechanisms in IoT middleware according to the problems at hand. We incorporate
semantic technologies in solving the above challenges. We demonstrate the
feasibility and the scalability of our approach through a prototype
implementation based on an IoT middleware called Global Sensor Networks (GSN),
though our model can be generalized into any other middleware platform. We
evaluate CASCoM in agriculture domain and measure both performance in terms of
usability and computational complexity.Comment: 9th International Conference on Semantics, Knowledge & Grids (SKG),
Beijing, China, October, 201
P4CEP: Towards In-Network Complex Event Processing
In-network computing using programmable networking hardware is a strong trend
in networking that promises to reduce latency and consumption of server
resources through offloading to network elements (programmable switches and
smart NICs). In particular, the data plane programming language P4 together
with powerful P4 networking hardware has spawned projects offloading services
into the network, e.g., consensus services or caching services. In this paper,
we present a novel case for in-network computing, namely, Complex Event
Processing (CEP). CEP processes streams of basic events, e.g., stemming from
networked sensors, into meaningful complex events. Traditionally, CEP
processing has been performed on servers or overlay networks. However, we argue
in this paper that CEP is a good candidate for in-network computing along the
communication path avoiding detouring streams to distant servers to minimize
communication latency while also exploiting processing capabilities of novel
networking hardware. We show that it is feasible to express CEP operations in
P4 and also present a tool to compile CEP operations, formulated in our P4CEP
rule specification language, to P4 code. Moreover, we identify challenges and
problems that we have encountered to show future research directions for
implementing full-fledged in-network CEP systems.Comment: 6 pages. Author's versio
StreamLearner: Distributed Incremental Machine Learning on Event Streams: Grand Challenge
Today, massive amounts of streaming data from smart devices need to be
analyzed automatically to realize the Internet of Things. The Complex Event
Processing (CEP) paradigm promises low-latency pattern detection on event
streams. However, CEP systems need to be extended with Machine Learning (ML)
capabilities such as online training and inference in order to be able to
detect fuzzy patterns (e.g., outliers) and to improve pattern recognition
accuracy during runtime using incremental model training. In this paper, we
propose a distributed CEP system denoted as StreamLearner for ML-enabled
complex event detection. The proposed programming model and data-parallel
system architecture enable a wide range of real-world applications and allow
for dynamically scaling up and out system resources for low-latency,
high-throughput event processing. We show that the DEBS Grand Challenge 2017
case study (i.e., anomaly detection in smart factories) integrates seamlessly
into the StreamLearner API. Our experiments verify scalability and high event
throughput of StreamLearner.Comment: Christian Mayer, Ruben Mayer, and Majd Abdo. 2017. StreamLearner:
Distributed Incremental Machine Learning on Event Streams: Grand Challenge.
In Proceedings of the 11th ACM International Conference on Distributed and
Event-based Systems (DEBS '17), 298-30
A Backend Framework for the Efficient Management of Power System Measurements
Increased adoption and deployment of phasor measurement units (PMU) has
provided valuable fine-grained data over the grid. Analysis over these data can
provide insight into the health of the grid, thereby improving control over
operations. Realizing this data-driven control, however, requires validating,
processing and storing massive amounts of PMU data. This paper describes a PMU
data management system that supports input from multiple PMU data streams,
features an event-detection algorithm, and provides an efficient method for
retrieving archival data. The event-detection algorithm rapidly correlates
multiple PMU data streams, providing details on events occurring within the
power system. The event-detection algorithm feeds into a visualization
component, allowing operators to recognize events as they occur. The indexing
and data retrieval mechanism facilitates fast access to archived PMU data.
Using this method, we achieved over 30x speedup for queries with high
selectivity. With the development of these two components, we have developed a
system that allows efficient analysis of multiple time-aligned PMU data
streams.Comment: Published in Electric Power Systems Research (2016), not available
ye
Adaptive Energy-aware Scheduling of Dynamic Event Analytics across Edge and Cloud Resources
The growing deployment of sensors as part of Internet of Things (IoT) is
generating thousands of event streams. Complex Event Processing (CEP) queries
offer a useful paradigm for rapid decision-making over such data sources. While
often centralized in the Cloud, the deployment of capable edge devices on the
field motivates the need for cooperative event analytics that span Edge and
Cloud computing. Here, we identify a novel problem of query placement on edge
and Cloud resources for dynamically arriving and departing analytic dataflows.
We define this as an optimization problem to minimize the total makespan for
all event analytics, while meeting energy and compute constraints of the
resources. We propose 4 adaptive heuristics and 3 rebalancing strategies for
such dynamic dataflows, and validate them using detailed simulations for 100 -
1000 edge devices and VMs. The results show that our heuristics offer
O(seconds) planning time, give a valid and high quality solution in all cases,
and reduce the number of query migrations. Furthermore, rebalance strategies
when applied in these heuristics have significantly reduced the makespan by
around 20 - 25%.Comment: 11 pages, 7 figure
- …