A characterization of workflow management systems for extreme-scale applications

Automation of the execution of computational tasks is at the heart of improving scientific productivity. Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications. By allowing scientists to model and express entire data processing steps and their dependencies, workflow management systems relieve scientists from the details of an application and manage its execution on a computational infrastructure. As the resource requirements of today’s computational and data science applications that process vast amounts of data keep increasing, there is a compelling case for a new generation of advances in high-performance computing, commonly termed as extreme-scale computing, which will bring forth multiple challenges for the design of workflow applications and management systems. This paper presents a novel characterization of workflow management systems using features commonly associated with extreme-scale computing applications. We classify 15 popular workflow management systems in terms of workflow execution models, heterogeneous computing environments, and data access methods. The paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems

Effective Computation Resilience in High Performance and Distributed Environments

The work described in this paper aims at effective computation resilience for complex simulations in high performance and distributed environments. Computation resilience is a complicated and delicate area; it deals with many types of simulation cores, many types of data on various input levels and also with many types of end-users, which have different requirements and expectations. Predictions about system and computation behaviors must be done based on deep knowledge about underlying infrastructures, and simulations' mathematical and realization backgrounds. Our conceptual framework is intended to allow independent collaborations between domain experts as end-users and providers of the computational power by taking on all of the deployment troubles arising within a given computing environment. The goal of our work is to provide a generalized approach for effective scalable usage of the computing power and to help domain-experts, so that they could concentrate more intensive on their domain solutions without the need of investing efforts in learning and adapting to the new IT backbone technologies

Population Diversity in Ant-inspired Optimization Algorithms

Finding a balance between exploration and exploitation is very important in the case of metaheuristics optimization, especially in the systems leveraging population of individuals expressing (as in Evolutionary Algorithms, etc.) or constructing (as in Ant Colony Optimization) solutions. Premature convergence is a real problem and finding means of its automatic detection and counteracting are of great importance. Measuring diversity in Evolutionary Algorithms working in real-value search space is often computationally complex, but feasible while measuring diversity in combinatorial domain is practically impossible (cf. Closest String Problem). Nevertheless, we propose several practical and feasible diversity measurement techniques dedicated to Ant Colony Optimization algorithms, leveraging the fact that even though analysis of the search space is at least an NP problem, we can focus on the pheromone table, where the direct outcomes of the search are expressed and can be analyzed. Besides proposing the measurement techniques, we apply them to assess the diversity of several variants of ACO, and closely analyze their features for the classic ACO. The discussion of the results is the first step towards applying the proposed measurement techniques in auto-adaptation of the parameters affecting directly the exploitation and exploration features in ACO in the future