CEP-DTHP : A Complex Event Processing using the Dual-Tier Hybrid Paradigm Over the Stream Mining Process

CEP is a widely used technique for the reliability and recognition of arbitrarily complex patterns in enormous data streams with great performance in real time. Real-time detection of crucial events and rapid response to them are the key goals of sophisticated event processing.  The performance of event processing systems can be improved by parallelizing CEP evaluation procedures. Utilizing CEP in parallel while deploying a multi-core or distributed environment is one of the most popular and widely recognized tackles to accomplish the goal. This paper demonstrates the ability to use an unusual parallelization strategy to effectively process complicated events over streams of data. This method depends on a dual-tier hybrid paradigm that combines several parallelism levels. Thread-level or task-level parallelism (TLP) and Data-level parallelism (DLP) were combined in this research. Many threads or instruction sequences from a comparable application can run concurrently under the TLP paradigm. In the DLP paradigm, instruc-tions from a single stream operate on several data streams at the same time. In our suggested model, there are four major stages: data mining, pre-processing, load shedding, and optimization. The first phase is online data mining, following which the data is materialized into a publicly available solution that combines a CEP engine with a library. Next, data pre-processing encompasses the efficient adaptation of the content or format of raw data from many, perhaps diverse sources. Finally, parallelization approaches have been created to reduce CEP processing time. By providing this two-type parallelism, our proposed solution combines the benefits of DLP and TLP while addressing their constraints. The JAVA tool will be used to assess the suggested technique. The performance of the suggested technique is compared to that of other current ways for determining the efficacy and efficiency of the proposed algorithm

Event detection from social network streams using frequent pattern mining with dynamic support values

Detecting events from streams of data is challenging due to the characteristics of such streams: data elements arrive in real-time and at high velocity, and the size of the streams is typically unbounded while it is not possible to backtrack over past data elements or maintain and review the entire history. Social networks are a good source for event identification as they generate huge amount of timely information representing what users are posting and discussing. In this research, we are developing methods for event detection from streams of data. More specifically, we are presenting a framework for detecting the daily occurring events or topics occurring in social network streams related to major events. Our approach utilizes the Frequent Pattern Mining method to detect the daily occurring frequent patterns, which are going to be our detected events. In addition, we propose a dynamic support definition method to replace the fixed given one. An experiment was run on two streams relating to two different major events to examine the detected events and to test our support definition method. The UK General Elections 2015 stream holds more than one million tweets, and the Greece Crisis 2015 stream contains more than 150k tweets. The detected events were evaluated against news headlines published the same day the event was found. The results revealed that the higher the streaming level (bigger window size), the more accurate the detected events. We also show that for too small sized windows, a more strict support definition method is needed to avoid detecting false or insignificant events

Real-time Content Identification for Events and Sub-Events from Microblogs.

PhDIn an age when people are predisposed to report real-world events through their social media accounts, many researchers value the advantages of mining such unstructured and informal data from social media. Compared with the traditional news media, online social media services, such as Twitter, can provide more comprehensive and timely information about real-world events. Existing Twitter event monitoring systems analyse partial event data and are unable to report the underlying stories or sub-events in realtime. To ll this gap, this research focuses on the automatic identi cation of content for events and sub-events through the analysis of Twitter streams in real-time. To full the need of real-time content identification for events and sub-events, this research First proposes a novel adaptive crawling model that retrieves extra event content from the Twitter Streaming API. The proposed model analyses the characteristics of hashtags and tweets collected from live Twitter streams to automate the expansion of subsequent queries. By investigating the characteristics of Twitter hashtags, this research then proposes three Keyword Adaptation Algorithms (KwAAs) which are based on the term frequency (TF-KwAA), the tra c pattern (TP-KwAA), and the text content of associated tweets (CS-KwAA) of the emerging hashtags. Based on the comparison between traditional keyword crawling and adaptive crawling with di erent KwAAs, this thesis demonstrates that the KwAAs retrieve extra event content about sub-events in real-time for both planned and unplanned events. To examine the usefulness of extra event content for the event monitoring system, a Twitter event monitoring solution is proposed. This \Detection of Sub-events by Twit- ter Real-time Monitoring (DSTReaM)" framework concurrently runs multiple instances of a statistical-based event detection algorithm over different stream components. By evaluating the detection performance using detection accuracy and event entropy, this research demonstrates that better event detection can be achieved with a broader coverage of event content.School of Electronic Engineering Computer Science (EECS), Queen Mary University of London (QMUL) China Scholarship Council (CSC)

Heuristics Miners for Streaming Event Data

More and more business activities are performed using information systems. These systems produce such huge amounts of event data that existing systems are unable to store and process them. Moreover, few processes are in steady-state and due to changing circumstances processes evolve and systems need to adapt continuously. Since conventional process discovery algorithms have been defined for batch processing, it is difficult to apply them in such evolving environments. Existing algorithms cannot cope with streaming event data and tend to generate unreliable and obsolete results. In this paper, we discuss the peculiarities of dealing with streaming event data in the context of process mining. Subsequently, we present a general framework for defining process mining algorithms in settings where it is impossible to store all events over an extended period or where processes evolve while being analyzed. We show how the Heuristics Miner, one of the most effective process discovery algorithms for practical applications, can be modified using this framework. Different stream-aware versions of the Heuristics Miner are defined and implemented in ProM. Moreover, experimental results on artificial and real logs are reported

Processing count queries over event streams at multiple time granularities

Management and analysis of streaming data has become crucial with its applications in web, sensor data, network tra c data, and stock market. Data streams consist of mostly numeric data but what is more interesting is the events derived from the numerical data that need to be monitored. The events obtained from streaming data form event streams. Event streams have similar properties to data streams, i.e., they are seen only once in a fixed order as a continuous stream. Events appearing in the event stream have time stamps associated with them in a certain time granularity, such as second, minute, or hour. One type of frequently asked queries over event streams is count queries, i.e., the frequency of an event occurrence over time. Count queries can be answered over event streams easily, however, users may ask queries over di erent time granularities as well. For example, a broker may ask how many times a stock increased in the same time frame, where the time frames specified could be hour, day, or both. This is crucial especially in the case of event streams where only a window of an event stream is available at a certain time instead of the whole stream. In this paper, we propose a technique for predicting the frequencies of event occurrences in event streams at multiple time granularities. The proposed approximation method e ciently estimates the count of events with a high accuracy in an event stream at any time granularity by examining the distance distributions of event occurrences. The proposed method has been implemented and tested on di erent real data sets and the results obtained are presented to show its e ectiveness