Investigating the memory requirements for publish/subscribe filtering algorithms

Various filtering algorithms for publish/subscribe systems have been proposed. One distinguishing characteristic is their internal representation of Boolean subscriptions: They either require conversions to disjunctive normal forms (canonical approaches) or are directly exploited in event filtering (non-canonical approaches). In this paper, we present a detailed analysis and comparison of the memory requirements of canonical and non-canonical filtering algorithms. This includes a theoretical analysis of space usages as well as a verification of our theoretical results by an evaluation of a practical implementation. This practical analysis also considers time (filter) efficiency, which is the other important quality measure of filtering algorithms. By correlating the results of space and time efficiency, we conclude when to use non-canonical and canonical approaches

Dimension-Based Subscription Pruning for Publish/Subscribe Systems

Subscription pruning has been proven as valuable routing optimization for Boolean subscriptions in publish/ subscribe systems. It aims at optimizing subscriptions independently of each other and is thus applicable for all kinds of subscriptions regardless of their individual and collective structures. The original subscription pruning approach tries to optimize the event routing process based on the expected increase in network load. However, a closer look at pruning-based routing reveals its further applicability to optimizations in respect to other dimensions. In this paper, we introduce and investigate subscription pruning based on three dimensions of optimization: network load, memory usage, and system throughput. We present the algorithms to perform prunings based on these dimensions and discuss the results of a series of practical experiments. Our analysis reveals the advantages and disadvantages of the different dimensions of optimization and allows conclusions about the suitability of dimension-based pruning for different application requirements

AWARE: Platform for Autonomous self-deploying and operation of Wireless sensor-actuator networks cooperating with unmanned AeRial vehiclEs

This paper presents the AWARE platform that seeks to enable the cooperation of autonomous aerial vehicles with ground wireless sensor-actuator networks comprising both static and mobile nodes carried by vehicles or people. Particularly, the paper presents the middleware, the wireless sensor network, the node deployment by means of an autonomous helicopter, and the surveillance and tracking functionalities of the platform. Furthermore, the paper presents the first general experiments of the AWARE project that took place in March 2007 with the assistance of the Seville fire brigades

Engineering Crowdsourced Stream Processing Systems

A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream. This can be seen as enabling crowdsourcing work to be applied on a sample of large-scale data at high speed, or equivalently, enabling stream processing to employ human intelligence. It also leads to a substantial expansion of the capabilities of data processing systems. Engineering a CSP system requires the combination of human and machine computation elements. From a general systems theory perspective, this means taking into account inherited as well as emerging properties from both these elements. In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design patterns. We showcase the capabilities of CSP systems by performing a case study that applies our proposed framework to the design and analysis of a real system (AIDR) that classifies social media messages during time-critical crisis events. Results show that compared to a pure stream processing system, AIDR can achieve a higher data classification accuracy, while compared to a pure crowdsourcing solution, the system makes better use of human workers by requiring much less manual work effort

Arbitrary boolean advertisements: the final step in supporting the boolean publish/subscribe model

Publish/subscribe systems allow for an efficient filtering of incoming information. This filtering is based on the specifications of subscriber interests, which are registered with the system as subscriptions. Publishers conversely specify advertisements, describing the messages they will send later on. What is missing so far is the support of arbitrary Boolean advertisements in publish/subscribe systems. Introducing the opportunity to specify these richer Boolean advertisements increases the accuracy of publishers to state their future messages compared to currently supported conjunctive advertisements. Thus, the amount of subscriptions forwarded in the network is reduced. Additionally, the system can more time efficiently decide whether a subscription needs to be forwarded and more space efficiently store and index advertisements. In this paper, we introduce a publish/subscribe system that supports arbitrary Boolean advertisements and, symmetrically, arbitrary Boolean subscriptions. We show the advantages of supporting arbitrary Boolean advertisements and present an algorithm to calculate the practically required overlapping relationship among subscriptions and advertisements. Additionally, we develop the first optimization approach for arbitrary Boolean advertisements, advertisement pruning. Advertisement pruning is tailored to optimize advertisements, which is a strong contrast to current optimizations for conjunctive advertisements. These recent proposals mainly apply subscription-based optimization ideas, which is leading to the same disadvantages. In the second part of this paper, our evaluation of practical experiments, we analyze the efficiency properties of our approach to determine the overlapping relationship. We also compare conjunctive solutions for the overlapping problem to our calculation algorithm to show its benefits. Finally, we present a detailed evaluation of the optimization potential of advertisement pruning. This includes the analysis of the effects of additionally optimizing subscriptions on the advertisement pruning optimization

Fuzzy logic-based approximate event notification in sparse MANETs

Mobile Ad-Hoc Networks (MANETs) are an important communication infrastructure to support emergency and rescue operations. To address the frequent disconnections and network partitions that might occur, we have developed a distributed event notification service (DENS) for sparse MANETs. In most event notification solutions, subscriptions are formed with crisp values or crisp value ranges. However, in emergency and rescue operations subscribers may not always have time to give crisp values or crisp value ranges. Moreover, subscriber's interests in queries have gradual nature and subjective measure that calls for computing by words. Therefore, we design and implement a simple fuzzy concept based subscription language allowing more expressive subscriptions and more sophisticated event-filtering. It is built on two new ideas: using features as multi-attribute indexes of the subscription and predicate patterns for processing subscriptions with arbitrary Boolean operators. However, requiring more computational efforts, fuzzy logic introduces performance penalties in the whole network. The proposed services have been evaluated for run-time, space and scalability efficiency. The proposed design framework is extensible to the user- and application-semantics and configurable to the dynamics in data that publish/subscribe paradigm imposes at runtime