Vectorizing Instance-Based Integration Processes

The inefficiency of integration processes as an abstraction of workflow-based integration tasks is often reasoned by low resource utilization and significant waiting times for external systems. Due to the increasing use of integration processes within IT infrastructures, the throughput optimization has high influence on the overall performance of such an infrastructure. In the area of computational engineering, low resource utilization is addressed with vectorization techniques. In this paper, we introduce the concept of vectorization in the context of integration processes in order to achieve a higher degree of parallelism. Here, transactional behavior and serialized execution must be ensured.In conclusion of our evaluation, the message throughput can be significantly increased

Cost-Based Vectorization of Instance-Based Integration Processes

The inefficiency of integration processes - as an abstraction of workflow-based integration tasks - is often reasoned by low resource utilization and significant waiting times for external systems. With the aim to overcome these problems, we proposed the concept of process vectorization. There, instance-based integration processes are transparently executed with the pipes-and-filters execution model. Here, the term vectorization is used in the sense of processing a sequence (vector) of messages by one standing process. Although it has been shown that process vectorization achieves a significant throughput improvement, this concept has two major drawbacks. First, the theoretical performance of a vectorized integration process mainly depends on the performance of the most cost-intensive operator. Second, the practical performance strongly depends on the number of available threads. In this paper, we present an advanced optimization approach that addresses the mentioned problems. Therefore, we generalize the vectorization problem and explain how to vectorize process plans in a cost-based manner. Due to the exponential complexity, we provide a heuristic computation approach and formally analyze its optimality. In conclusion of our evaluation, the message throughput can be significantly increased compared to both the instance-based execution as well as the rule-based process vectorization

On the Complexity of Process Pipeline Scheduling

Abstract. This paper explores how process pipeline scheduling may become a viable strategy for executing workflows. It first details a workflow optimization and execution algorithm that reduces runtime space. The optimization strategy pipelines the communication between as many processes as possible, within the bounds of the storage space available, and depends on generic properties of datasets and processes. Then, the paper proves that the process pipeline scheduling problem is NP-Complete. Finally, it presents a greedy process pipeline scheduling algorithm which has a viable performance. 1