Engineering Crowdsourced Stream Processing Systems

A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream. This can be seen as enabling crowdsourcing work to be applied on a sample of large-scale data at high speed, or equivalently, enabling stream processing to employ human intelligence. It also leads to a substantial expansion of the capabilities of data processing systems. Engineering a CSP system requires the combination of human and machine computation elements. From a general systems theory perspective, this means taking into account inherited as well as emerging properties from both these elements. In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design patterns. We showcase the capabilities of CSP systems by performing a case study that applies our proposed framework to the design and analysis of a real system (AIDR) that classifies social media messages during time-critical crisis events. Results show that compared to a pure stream processing system, AIDR can achieve a higher data classification accuracy, while compared to a pure crowdsourcing solution, the system makes better use of human workers by requiring much less manual work effort

DRS: Dynamic Resource Scheduling for Real-Time Analytics over Fast Streams

In a data stream management system (DSMS), users register continuous queries, and receive result updates as data arrive and expire. We focus on applications with real-time constraints, in which the user must receive each result update within a given period after the update occurs. To handle fast data, the DSMS is commonly placed on top of a cloud infrastructure. Because stream properties such as arrival rates can fluctuate unpredictably, cloud resources must be dynamically provisioned and scheduled accordingly to ensure real-time response. It is quite essential, for the existing systems or future developments, to possess the ability of scheduling resources dynamically according to the current workload, in order to avoid wasting resources, or failing in delivering correct results on time. Motivated by this, we propose DRS, a novel dynamic resource scheduler for cloud-based DSMSs. DRS overcomes three fundamental challenges: (a) how to model the relationship between the provisioned resources and query response time (b) where to best place resources; and (c) how to measure system load with minimal overhead. In particular, DRS includes an accurate performance model based on the theory of \emph{Jackson open queueing networks} and is capable of handling \emph{arbitrary} operator topologies, possibly with loops, splits and joins. Extensive experiments with real data confirm that DRS achieves real-time response with close to optimal resource consumption.Comment: This is the our latest version with certain modificatio

What Automated Planning Can Do for Business Process Management

Business Process Management (BPM) is a central element of today organizations. Despite over the years its main focus has been the support of processes in highly controlled domains, nowadays many domains of interest to the BPM community are characterized by ever-changing requirements, unpredictable environments and increasing amounts of data that influence the execution of process instances. Under such dynamic conditions, BPM systems must increase their level of automation to provide the reactivity and flexibility necessary for process management. On the other hand, the Artificial Intelligence (AI) community has concentrated its efforts on investigating dynamic domains that involve active control of computational entities and physical devices (e.g., robots, software agents, etc.). In this context, Automated Planning, which is one of the oldest areas in AI, is conceived as a model-based approach to synthesize autonomous behaviours in automated way from a model. In this paper, we discuss how automated planning techniques can be leveraged to enable new levels of automation and support for business processing, and we show some concrete examples of their successful application to the different stages of the BPM life cycle

User recommendations for the optimized execution of business processes

In order to be able to flexibly adjust a company's business processes (BPs) there is an increasing interest in flexible process-aware information systems (PAISs). This increasing flexibility, however, typically implies decreased user guidance by the PAIS and thus poses significant challenges to its users. As a major contribution of this work, we propose a recommendation system which assists users during process execution to optimize performance goals of the processes. The recommendation system is based on a constraint-based approach for planning and scheduling the BP activities and considers both the control-flow and the resource perspective. To evaluate the proposed constraint-based approach different algorithms are applied to a range of test models of varying complexity. The results indicate that, although the optimization of process execution is a highly constrained problem, the proposed approach produces a satisfactory number of suitable solutions.Ministerio de Ciencia e Innovación TIN2009-1371

Decision-making support for input data in business processes according to former instances

Business Processes facilitate the execution of a set of activities to achieve the strategic plans of a company. During the execution of a business process model, several decisions can be made that frequently involve the values of the input data of certain activities. The decision regarding the value of these input data concerns not only the correct execution of the business process in terms of consistency, but also the compliance with the strategic plans of the company. Smart decision-support systems provide information by analyzing the process model and the business rules to be satisfied, but other elements, such as the previous temporal variation of the data during the former executed instances of similar processes, can also be employed to guide the input data decisions at instantiation time. Our proposal consists of learning the evolution patterns of the temporal variation of the data values in a process model extracted from previous process instances by applying Constraint Programming techniques. The knowledge obtained is applied in a Decision Support System (DSS) which helps in the maintenance of the alignment of the process execution with the organizational strategic plans, through a framework and a methodology. Finally, to present a proof of concept, the proposal has been applied to a complete case study.Ministerio de Ciencia y Tecnología RTI2018-094283-B-C3

Shared Memory Parallel Subgraph Enumeration

The subgraph enumeration problem asks us to find all subgraphs of a target graph that are isomorphic to a given pattern graph. Determining whether even one such isomorphic subgraph exists is NP-complete---and therefore finding all such subgraphs (if they exist) is a time-consuming task. Subgraph enumeration has applications in many fields, including biochemistry and social networks, and interestingly the fastest algorithms for solving the problem for biochemical inputs are sequential. Since they depend on depth-first tree traversal, an efficient parallelization is far from trivial. Nevertheless, since important applications produce data sets with increasing difficulty, parallelism seems beneficial. We thus present here a shared-memory parallelization of the state-of-the-art subgraph enumeration algorithms RI and RI-DS (a variant of RI for dense graphs) by Bonnici et al. [BMC Bioinformatics, 2013]. Our strategy uses work stealing and our implementation demonstrates a significant speedup on real-world biochemical data---despite a highly irregular data access pattern. We also improve RI-DS by pruning the search space better; this further improves the empirical running times compared to the already highly tuned RI-DS.Comment: 18 pages, 12 figures, To appear at the 7th IEEE Workshop on Parallel / Distributed Computing and Optimization (PDCO 2017

Flexible runtime support of business processes under rolling planning horizons

This work has been motivated by the needs we discovered when analyzing real-world processes from the healthcare domain that have revealed high flexibility demands and complex temporal constraints. When trying to model these processes with existing languages, we learned that none of the latter was able to fully address these needs. This motivated us to design TConDec-R, a declarative process modeling language enabling the specification of complex temporal constraints. Enacting business processes based on declarative process models, however, introduces a high complexity due to the required optimization of objective functions, the handling of various temporal constraints, the concurrent execution of multiple process instances, the management of crossinstance constraints, and complex resource allocations. Consequently, advanced user support through optimized schedules is required when executing the instances of such models. In previous work, we suggested a method for generating an optimized enactment plan for a given set of process instances created from a TConDec-R model. However, this approach was not applicable to scenarios with uncertain demands in which the enactment of newly created process instances starts continuously over time, as in the considered healthcare scenarios. Here, the process instances to be planned within a specific timeframe cannot be considered in isolation from the ones planned for future timeframes. To be able to support such scenarios, this article significantly extends our previous work by generating optimized enactment plans under a rolling planning horizon. We evaluate the approach by applying it to a particularly challenging healthcare process scenario, i.e., the diagnostic procedures required for treating patients with ovarian carcinoma in a Woman Hospital. The application of the approach to this sophisticated scenario allows avoiding constraint violations and effectively managing shared resources, which contributes to reduce the length of patient stays in the hospital.Ministerio de Economía y Competitividad TIN2016-76956-C3-2-RMinisterio de Ciencia e Innovación PID2019-105455 GB-C3