Knowledge-infused and Consistent Complex Event Processing over Real-time and Persistent Streams

Emerging applications in Internet of Things (IoT) and Cyber-Physical Systems (CPS) present novel challenges to Big Data platforms for performing online analytics. Ubiquitous sensors from IoT deployments are able to generate data streams at high velocity, that include information from a variety of domains, and accumulate to large volumes on disk. Complex Event Processing (CEP) is recognized as an important real-time computing paradigm for analyzing continuous data streams. However, existing work on CEP is largely limited to relational query processing, exposing two distinctive gaps for query specification and execution: (1) infusing the relational query model with higher level knowledge semantics, and (2) seamless query evaluation across temporal spaces that span past, present and future events. These allow accessible analytics over data streams having properties from different disciplines, and help span the velocity (real-time) and volume (persistent) dimensions. In this article, we introduce a Knowledge-infused CEP (X-CEP) framework that provides domain-aware knowledge query constructs along with temporal operators that allow end-to-end queries to span across real-time and persistent streams. We translate this query model to efficient query execution over online and offline data streams, proposing several optimizations to mitigate the overheads introduced by evaluating semantic predicates and in accessing high-volume historic data streams. The proposed X-CEP query model and execution approaches are implemented in our prototype semantic CEP engine, SCEPter. We validate our query model using domain-aware CEP queries from a real-world Smart Power Grid application, and experimentally analyze the benefits of our optimizations for executing these queries, using event streams from a campus-microgrid IoT deployment.Comment: 34 pages, 16 figures, accepted in Future Generation Computer Systems, October 27, 201

Efficient and Effective Event Pattern Management

The goal of this thesis is to reduce the barriers stopping more enterprises from accessing CEP technology by providing additional support in managing relevant business situations. Therefore we outline the role of event pattern management and present a methodology, methods and tools aiming at an efficient and effective event pattern management. We provide a meta model for event patterns, an event pattern life cycle methodology, methods for guidance, refinement and evolution

An Interface-based Modular Approach for Designing Distributed Event-based Systems

A Distributed Event-based System (DEBS) exhibits its desired behavior through its functional components collaborating with each other via event exchanging. Due to loose-coupling and flexibility, DEBS applications have become increasingly popular. Indeed, such systems are expected to appear in various application domains such as large-scale Internet applications and ubiquitous computing. Notwithstanding their popularity, current DEBS applications are still often developed in an informal process and are not modularized. On the individual event level, current DEBS developers can define what events a component can accept and publish, and, by registering event handlers, what action an event can trigger. Currently, developers lack structuring mechanisms for representing event interactions and dependencies in a modular way. While current research has made fruitful contributions to various aspects in the DEBS paradigm, such as, event delivery, event detection and composition, event visibility, its emphasis is on the individual event level. In this thesis, we advocate that by designing a new DEBS metamodel with extended behavioral interfaces and high-level structure mechanisms, we can (1) define an interface-based modular approach to model and design DEBS applications, (2) implement a prototype framework on a P2P network that provides built-in support to our proposed interface-based DEBS development, and (3) provide case studies illustrating the interface-based development process and the applicability of our proposed approach

Intégration de données temps-réel issues de capteurs dans un entrepôt de données géo-décisionnel

Nous avons pu, au cours des dernières années, assister à une augmentation du nombre de capteurs utilisés pour mesurer des phénomènes de plus en plus variés. En effet, nous pouvons aujourd'hui utiliser les capteurs pour mesurer un niveau d'eau, une position (GPS), une température et même le rythme cardiaque d'un individu. La grande diversité de capteurs fait d'eux aujourd'hui des outils par excellence en matière d'acquisition de données. En parallèle à cette effervescence, les outils d'analyse ont également évolué depuis les bases de données transactionnelles et ont mené à l'apparition d'une nouvelle famille d’outils, appelés systèmes d’analyse (systèmes décisionnels), qui répond à des besoins d’analyse globale sur les données. Les entrepôts de données et outils OLAP (On-Line Analytical Processing), qui font partie de cette famille, permettent dorénavant aux décideurs d'analyser l'énorme volume de données dont ils disposent, de réaliser des comparaisons dans le temps et de construire des graphiques statistiques à l’aide de simples clics de la souris. Les nombreux types de capteurs peuvent certainement apporter de la richesse à une analyse, mais nécessitent de longs travaux d'intégration pour les amener jusqu'à un entrepôt géo-décisionnel, qui est au centre du processus de prise de décision. Les différents modèles de capteurs, types de données et moyens de transférer les données sont encore aujourd'hui des obstacles non négligeables à l'intégration de données issues de capteurs dans un entrepôt géo-décisionnel. Également, les entrepôts de données géo-décisionnels actuels ne sont pas initialement conçus pour accueillir de nouvelles données sur une base fréquente. Puisque l'utilisation de l'entrepôt par les utilisateurs est restreinte lors d'une mise à jour, les nouvelles données sont généralement ajoutées sur une base hebdomadaire, mensuelle, etc. Il existe pourtant des entrepôts de données capables d'être mis à jour plusieurs fois par jour sans que les performances lors de leur exploitation ne soient atteintes, les entrepôts de données temps-réel (EDTR). Toutefois, cette technologie est encore aujourd’hui peu courante, très coûteuse et peu développée. Ces travaux de recherche visent donc à développer une approche permettant de publier et standardiser les données temps-réel issues de capteurs et de les intégrer dans un entrepôt géo-décisionnel conventionnel. Une stratégie optimale de mise à jour de l'entrepôt a également été développée afin que les nouvelles données puissent être ajoutées aux analyses sans que la qualité de l'exploitation de l'entrepôt par les utilisateurs ne soit remise en cause.In the last decade, the use of sensors for measuring various phenomenons has greatly increased. As such, we can now make use of sensors to measure GPS position, temperature and even the heartbeats of a person. Nowadays, the wide diversity of sensor makes them the best tools to gather data. Along with this effervescence, analysis tools have also advanced since the creation of transactional databases, leading to a new category of tools, analysis systems (Business Intelligence (BI)), which respond to the need of the global analysis of the data. Data warehouses and OLAP (On-Line Analytical Processing) tools, which belong to this category, enable users to analyze big volumes of data, execute time-based requests and build statistic graphs in a few simple mouse clicks. Although the various types of sensor can surely enrich any analysis, such data requires heavy integration processes to be driven into the data warehouse, centerpiece of any decision-making process. The different data types produced by sensors, sensor models and ways to transfer such data are even today significant obstacles to sensors data streams integration in a geo-decisional data warehouse. Also, actual geo-decisional data warehouses are not initially built to welcome new data on a high frequency. Since the performances of a data warehouse are restricted during an update, new data is usually added weekly, monthly, etc. However, some data warehouses, called Real-Time Data Warehouses (RTDW), are able to be updated several times a day without letting its performance diminish during the process. But this technology is not very common, very costly and in most of cases considered as "beta" versions. Therefore, this research aims to develop an approach allowing to publish and normalize real-time sensors data streams and to integrate it into a classic data warehouse. An optimized update strategy has also been developed so the frequent new data can be added to the analysis without affecting the data warehouse performances

Web-oriented Event Processing

How can the Web be made situation-aware? Event processing is a suitable technology for gaining the necessary real-time results. The Web, however, has many users and many application domains. Thus, we developed multi-schema friendly data models allowing the re-use and mix from diverse users and application domains. Furthermore, our methods describe protocols to exchange events on the Web, algorithms to execute the language and to calculate access rights

Integration of Event Processing with Service-oriented Architectures and Business Processes

Data sources like the Internet of Things or Cyber-physical Systems provide enormous amounts of real-time information in form of streams of events. The use of such event streams enables reactive software components as building blocks in a new generation of systems. Businesses, for example, can benefit from the integration of event streams; new services can be provided to customers, or existing business processes can be improved. The development of reactive systems and the integration with existing application landscapes, however, is challenging. While traditional system components follow a pull-based request/reply interaction style, event-based systems follow a push-based interaction scheme; events arrive continuously and application logic is triggered implicitly. To benefit from push-based and pull-based interactions together, an intuitive software abstraction is necessary to integrate push-based application logic with existing systems. In this work we introduce such an abstraction: we present Event Stream Processing Units (SPUs) - a container model for the encapsulation of event-processing application logic at the technical layer as well as at the business process layer. At the technical layer SPUs provide a service-like abstraction and simplify the development of scalable reactive applications. At the business process layer SPUs make event processing explicitly representable. SPUs have a managed lifecycle and are instantiated implicitly - upon arrival of appropriate events - or explicitly upon request. At the business process layer SPUs encapsulate application logic for event stream processing and enable a seamless transition between process models, executable process representations, and components at the IT layer. Throughout this work, we focus on different aspects of the SPU container model: we first introduce the SPU container model and its execution semantics. Since SPUs rely on a publish/subscribe system for event dissemination, we discuss quality of service requirements in the context of event processing. SPUs rely on input in form of events; in event-based systems, however, event production is logically decoupled, i.e., event producers are not aware of the event consumers. This influences the system development process and requires an appropriate methodology. Fur this purpose we present a requirements engineering approach that takes the specifics of event-based applications into account. The integration of events with business processes leads to new business opportunities. SPUs can encapsulate event processing at the abstraction level of business functions and enable a seamless integration with business processes. For this integration, we introduce extensions to the business process modeling notations BPMN and EPCs to model SPUs. We also present a model-to-execute workflow for SPU-containing process models and implementation with business process modeling software. The SPU container model itself is language-agnostic; thus, we present Eventlets as SPU implementation based on Java Enterprise technology. Eventlets are executed inside a distributed middleware and follow a lifecycle. They reduce the development effort of scalable event processing applications as we show in our evaluation. Since the SPU container model introduces an additional layer of abstraction we analyze the overhead in terms of performance and show that Eventlets can compete with traditional event processing approaches in terms of performance. SPUs can be used to process sensitive data, e.g., in health care environments. Thus, privacy protection is an important requirement for certain use cases and we sketch the application of a privacy-preserving event dissemination scheme to protect event consumers and producers from curious brokers. We also quantify the resulting overhead introduced by a privacy-preserving brokering scheme in an evaluation

E-CARe (une méthode d'ingénierie des systèmes d'information ubiquitaires)

L'apparition des Systèmes d'Information ubiquitaires ou pervasifs est issue de l'émergence de nouvelles technologies fournissant au système une vision de son environnement, de l'environnement de ses utilisateurs ainsi que de leurs profils. Grâce à ces données formant le contexte de l'application, il est possible de fournir des services personnalisés, pertinents et ciblés. Mais, le problème qui se pose à ce niveau concerne le degré d'adaptation, de prise de décision à la place de l'utilisateur et de l'identification des données contextuelles nécessaires et suffisantes pour ces services. Ceci est dû à un déséquilibre entre les avancées des technologies et de leurs applications (qui reçoivent un grand intérêt de la part de la recherche et de l'industrie) et les méthodes et démarches de développement et d'ingénierie spécifiques aux systèmes ubiquitaires. Notre objectif dans ce travail de thèse est de proposer une méthode d'ingénierie des SI ubiquitaires en considérant les différentes exigences reliées à la nature mobile et grande échelle de ces systèmes. Cette méthode est basée sur une démarche de développement qui fait usage d'un ensemble de métamodèles et de langages génériques favorisant la spécification complète de ces systèmes. Cette démarche sépare les spécifications fonctionnelles, techniques et ubiquitaires. Les spécifications ubiquitaires permettent de définir des modèles structurels et événementiels du contexte respectant les exigences des utilisateurs et les contraintes de sécurité et supportant les fonctionnalités d'adaptation et de sensibilité au contexte. Cette approche orientée évènements est consolidée par l'adoption d'une architecture de traitement des évènements. Notre démarche E-CARe est une intégration des spécifications ubiquitaires dans une démarche classique de conception des SI pour garantir la couverture des spécifications fonctionnelles et techniques. Les applications d'assistance représentent un cas d'étude idéal pour cette démarche qui s'intéresse au domaine des transports, fortement dépendant de l'environnement et des évènements ambiants.Ubiquitous Information Systems appeared as a consequence to emerging and evolving communication and mobile technologies providing the system with information on its environment, the environment of its users and their profiles. These data constitute the application context and are used to provide personalized, targeted and relevant services. However, ubiquitous services face some difficulties and challenges concerning specially needed contextual data, adaptation degree and computerized decision making. This is due to the gap between advanced ubiquitous services and their applications, and methods and processes for developing and engineering ubiquitous systems. Our goal in this thesis is to propose an engineering method for ubiquitous Information Systems considering different requirements resulting from the mobile and high scalable nature of these systems. The proposed method is based on a development process and a set of generic metamodels and languages facilitating a complete system specification and implementation. The proposed process separates functional, technical and ubiquitous specifications. Ubiquitous specifications enable the structural and event based context models definition while considering user requirements and security requirements. Adaptation and context awareness functionalities are supported by structural and dynamic context models. The proposed event oriented approach is enhanced by the adoption of an event processing architecture. Ubiquitous specifications are integrated into a classical information systems engineering process to constitute the E-CARe process including functional and technical specifications. Our propositions are used to design a user assistance application in the transport domain, highly dependent on the ambient environment and events.SAVOIE-SCD - Bib.électronique (730659901) / SudocGRENOBLE1/INP-Bib.électronique (384210012) / SudocGRENOBLE2/3-Bib.électronique (384219901) / SudocSudocFranceF

Process Models for Distributed Event-Based Systems

Distributed Event-Based Systems (DEBSs) are middleware supporting the interaction of publisher and subscriber components via events. In DEBSs, the subscribers to be notified when an event is announced are decided at run-time without requiring publisher components to know the name or locations of the subscribers, nor the subscribers to know the name or locations of the publishers. This low coupling between components makes DEBSs suitable for applications with a large or unpredictable number of autonomous components. The development of applications in DEBSs is an ad hoc process poorly supported by current software engineering methodologies. Moreover, the behaviours exhibited by these systems and their applications are not well understood, and no suitable models exist where these behaviours can be described and analyzed. The main concern of this thesis is the development of such models. Specifically, we develop formalisms and models supporting the specification, prediction, and validation of the behaviour exhibited by the middleware and the applications executing on it. Our main contributions to the area are: new formalisms for the representation of DEBSs and their applications, and for the specification of both, system and application properties; a categorization of the features related to the definition, announcement, and notification of events in DEBSs and, in general, event-based systems; models representing the categorized DEBS features; case studies detailing models and properties for specific systems; a prototype tool for the verification of DEBSs and applications. The formalisms developed expose the location of the actions in the modelled systems and support the specification of several forms of location-awareness and adaptive behaviour