A compositional method for reliability analysis of workflows affected by multiple failure modes

We focus on reliability analysis for systems designed as workflow based compositions of components. Components are characterized by their failure profiles, which take into account possible multiple failure modes. A compositional calculus is provided to evaluate the failure profile of a composite system, given failure profiles of the components. The calculus is described as a syntax-driven procedure that synthesizes a workflows failure profile. The method is viewed as a design-time aid that can help software engineers reason about systems reliability in the early stage of development. A simple case study is presented to illustrate the proposed approach

Performance optimization and energy efficiency of big-data computing workflows

Next-generation e-science is producing colossal amounts of data, now frequently termed as Big Data, on the order of terabyte at present and petabyte or even exabyte in the predictable future. These scientific applications typically feature data-intensive workflows comprised of moldable parallel computing jobs, such as MapReduce, with intricate inter-job dependencies. The granularity of task partitioning in each moldable job of such big data workflows has a significant impact on workflow completion time, energy consumption, and financial cost if executed in clouds, which remains largely unexplored. This dissertation conducts an in-depth investigation into the properties of moldable jobs and provides an experiment-based validation of the performance model where the total workload of a moldable job increases along with the degree of parallelism. Furthermore, this dissertation conducts rigorous research on workflow execution dynamics in resource sharing environments and explores the interactions between workflow mapping and task scheduling on various computing platforms. A workflow optimization architecture is developed to seamlessly integrate three interrelated technical components, i.e., resource allocation, job mapping, and task scheduling. Cloud computing provides a cost-effective computing platform for big data workflows where moldable parallel computing models are widely applied to meet stringent performance requirements. Based on the moldable parallel computing performance model, a big-data workflow mapping model is constructed and a workflow mapping problem is formulated to minimize workflow makespan under a budget constraint in public clouds. This dissertation shows this problem to be strongly NP-complete and designs i) a fully polynomial-time approximation scheme for a special case with a pipeline-structured workflow executed on virtual machines of a single class, and ii) a heuristic for a generalized problem with an arbitrary directed acyclic graph-structured workflow executed on virtual machines of multiple classes. The performance superiority of the proposed solution is illustrated by extensive simulation-based results in Hadoop/YARN in comparison with existing workflow mapping models and algorithms. Considering that large-scale workflows for big data analytics have become a main consumer of energy in data centers, this dissertation also delves into the problem of static workflow mapping to minimize the dynamic energy consumption of a workflow request under a deadline constraint in Hadoop clusters, which is shown to be strongly NP-hard. A fully polynomial-time approximation scheme is designed for a special case with a pipeline-structured workflow on a homogeneous cluster and a heuristic is designed for the generalized problem with an arbitrary directed acyclic graph-structured workflow on a heterogeneous cluster. This problem is further extended to a dynamic version with deadline-constrained MapReduce workflows to minimize dynamic energy consumption in Hadoop clusters. This dissertation proposes a semi-dynamic online scheduling algorithm based on adaptive task partitioning to reduce dynamic energy consumption while meeting performance requirements from a global perspective, and also develops corresponding system modules for algorithm implementation in the Hadoop ecosystem. The performance superiority of the proposed solutions in terms of dynamic energy saving and deadline missing rate is illustrated by extensive simulation results in comparison with existing algorithms, and further validated through real-life workflow implementation and experiments using the Oozie workflow engine in Hadoop/YARN systems

What's next? : operational support for business process execution

In the last decade flexibility has become an increasingly important in the area of business process management. Information systems that support the execution of the process are required to work in a dynamic environment that imposes changing demands on the execution of the process. In academia and industry a variety of paradigms and implementations has been developed to support flexibility. While on the one hand these approaches address the industry demands in flexibility, on the other hand, they result in confronting the user with many choices between different alternatives. As a consequence, methods to support users in selecting the best alternative during execution have become essential. In this thesis we introduce a formal framework for providing support to users based on historical evidence available in the execution log of the process. This thesis focuses on support by means of (1) recommendations that provide the user an ordered list of execution alternatives based on estimated utilities and (2) predictions that provide the user general statistics for each execution alternative. Typically, estimations are not an average over all observations, but they are based on observations for "similar" situations. The main question is what similarity means in the context of business process execution. We introduce abstractions on execution traces to capture similarity between execution traces in the log. A trace abstraction considers some trace characteristics rather than the exact trace. Traces that have identical abstraction values are said to be similar. The challenge is to determine those abstractions (characteristics) that are good predictors for the parameter to be estimated in the recommendation or prediction. We analyse the dependency between values of an abstraction and the mean of the parameter to be estimated by means of regression analysis. With regression we obtain a set of abstractions that explain the parameter to be estimated. Dependencies do not only play a role in providing predictions and recommendations to instances at run-time, but they are also essential for simulating the effect of changes in the environment on the processes, both locally and globally. We use stochastic simulation models to simulate the effect of changes in the environment, in particular changed probability distribution caused by recommendations. The novelty of these models is that they include dependencies between abstraction values and simulation parameters, which are estimated from log data. We demonstrate that these models give better approximations of reality than traditional models. A framework for offering operational support has been implemented in the context of the process mining framework ProM

IT Supported Construction Project Management Methodology Based on Process and Product Model and Quality Management

Computer Integrated Construction Project Management (CPM) supported by product and process models can be seen as a future type of integration structure facilitating the solution of various management problems in the fragmented Construction Industry. The key to success is directly correlated with the comprehensive integration of currently isolated IT applications. However, despite that a number of initiatives have been developed, no fully generic models have yet to be formally standardized. This topic has been the subject of intensive research during the last decades. In this thesis a Computer Integrated CPM approach, which is supported by IFC (Industry Foundation Classes) and ISO9001:2000 Quality Management System, is proposed. The main aim is to provide integration of product, process and organizational information to help achieve the interoperability of the involved actors and tools in a concurrent environment. According to implied requirements which are represented in the ‘state of the art’ section, the fundamental concepts are presented in two parts as: (1) realization of CPM in an IT concept and (2) formalization of IFC Views for software interoperability on the example of Bidding Preparation Phase. In order to realize a generic framework using a high-level process core model named Organizational Management Process (OMP) model, different aspects have been brought together into a consistent life cycle structure. These are: (1) a set of layered processes based on ISO procedural definitions, (2) software integration requirements based on Construction Management Phases, (3) application methods of the Procurement System and (4) Organizational data. This provides for synchronizing technical products, processes, documents, and actors in their inter-relationship. The framework is hierarchically structured in three layers Phases – Processes - Product data. The developed IT Management Processes (ITMP) which are used as a baseline for the IFC Views implementation are derived from the OMP. Moreover, in order to support completeness, a mapping structure between processes and scenarios based on the Procurement Systems was constituted. The representation of OMP and ITMP is provided by using the ARIS eEPC (extended event-driven process chain) modeling method. On the basis of a generalized representation of product data, a system-wide integration model for heterogeneous client applications which supports different CPM areas can be achieved. IFC Product Data Model integrates different domains thereby enabling coordination of bidding preparations. However, there is a need to realize individual model subsets. i.e. views of the product model. In this context, adaptable views were developed based on ITMP. The defined resources’ relevancies to IFC Objects are examined by realizing central information elements. These provide a mapping structure between process resources and IFC Classes. On that basis integration of process and product models can be accomplished. In order to realize IFC Views, IFC Concepts and IFC Instance Diagrams were developed based on IFC View Definition Format. The grouping of IFC Concepts enables the implementation of the adaptable IFC Views that are required for standardized system integration. This is achieved with the help of formal specification using the Generalized Subset Definition Schema. The validation has been made based on an alphanumerical comparison. The selected 3D full-model and the developed IFC View for Product Catalog models are compared in this context. There are two consequences observed. In the first case, which also addresses Unit Price Procurement systems, the desired results were obtained by filtering the required data. However, when the results were compared for Design & Build and Lump-sum Procurement Systems (contracts), an extension need was observed in the IFC Model. The solution is provided via formalization of cost data and material analysis information by an extension of IFC Concept namely ‘IfcConstructionResource’ with new classes and with new relations. Thereby a common information model based on the data schema of the IFC standard is constituted.Das von Produkt- und Prozessmodellen unterstützte computerintegrierte Bauprojektmanagement (CPM) kann als der zukünftige Typ der Integrationsstruktur angesehen werden, der die Lösung verschiedener Baumanagementprobleme in der fragmentierten Bauindustrie erleichtern kann. Der Schlüssel zum Erfolg steht in direkter Beziehung zu einer umfassenden Integration derzeit getrennter IT-Anwendungen. Trotz zahlreich entwickelter Ansätze, die zur Verfügung gestellt wurden, sind bisher noch keine vollständig generischen Modelle formell standardisiert worden, obwohl dies in den letzten Jahrzehnten ein Thema intensiver Forschung war. In dieser Promotionsschrift wird eine computerintegrierte CPM-Methode, die auf Basis der IFC (Industry Foundation Classes) und dem Qualitätsmanagement ISO 9001:2000 aufbaut, vorgeschlagen. Das Hauptziel besteht in der Schaffung der Integration von Produkt-, Prozess- und Organisationsinformationen, um die Interoperabilität der beteiligten Akteure und Tools in einer parallelen Umgebung erreichen zu können. Entsprechend den Anforderungen, die im Abschnitt „Stand der Technik“ aufgeführt sind, werden die vorgeschlagenen, grundlegenden Konzepte in zwei Bereiche aufgeteilt: (1) Umsetzung der CPM-Prozesse in ein IT-Konzept und (2) Formalisierung der IFC-Sichten für die Interoperabilität von Software, beispielhaft ausgeführt für die der Ausschreibungsphase. Um einen generischen Rahmen unter Verwendung eines hochrangigen Prozesskernmodells, das als organisatorischer Managementprozess (OMP) bezeichnet wird, zu realisieren, werden zuerst die verschiedenen Aspekte in einer konsistenten Lebenszyklenstruktur zusammengefügt. Diese sind: (1) eine Menge hierarchisch geschichteter Prozesse, erstellt auf der Grundlage der Verfahrensdefinitionen von ISO 9001, (2) die Softwareintegrationsanforderungen auf der Grundlage der Baumanagementphasen, (3) die Anwendungsmethoden des Beschaffungssystems und (4) die Organisationsdaten. Dadurch wird die Synchronisation der in Wechselbeziehung stehenden technischen Produkte, Prozesse, Dokumente und Akteure geschaffen. Das gesamte System ist hierarchisch in die drei Ebenen Phasen – Prozesse – Produktdaten strukturiert. Die entwickelten IT-Managementprozesse (ITMP), die als Grundlage für die IFC-Implementierungssichten dienen, werden aus dem OMP hergeleitet. Der Vollständigkeit halber, wird eine Abbildungsstruktur zwischen den Prozessen und den Szenarien, die die Beschaffungssysteme beschreiben, entwickelt. Die Darstellung der OMP und ITMP erfolgt unter Verwendung der erweiterten ereignisgesteuerten Prozessketten (eEPK) nach der ARIS-Modelliermethode. Auf der Grundlage einer verallgemeinerten Darstellung der Prozessdaten kann das systemweite Integrationsmodell für heterogene Client-Anwendungen, das verschiedene CPM-Bereiche unterstützt, erreicht werden. Das IFC-Produktdatenmodell integriert verschiedene Domänen und ermöglicht somit die Koordinierung der hier beispielhaft gewählten Ausschreibungsbearbeitungen. Hierzu ist es notwendig, Teilmodelle, d. h. Sichten des Produktmodells zu erzeugen. Entsprechend wurden anpassbare Sichten auf der Grundlage von ITMP entwickelt. Die Bedeutung der in diesem Zusammenhang identifizierten Informationsprozessressourcen in Bezug auf die IFC-Objekte wurde durch die Einführung zentraler Informationselemente, sog. IFC Concepts, untersucht. Diese stellen eine Abbildungsstruktur zwischen den Prozessressourcen und IFC-Klassen zur Verfügung. Auf dieser Grundlage konnte die Integration von Prozess- und Produktmodellen erreicht werden. Um die IFC-Sichten zu realisieren, wurden auf der Grundlage des IFC-Sichtendefinitionsformats IFC-Konzepte und IFC-Instanzendiagramme entwickelt. Die Gruppierung in IFC-Konzepten ermöglichte die Implementierung von anpassbaren IFC-Sichten, die für die standardisierte Systemintegration erforderlich sind. Diese wird mit Hilfe einer formellen Spezifikation unter Verwendung der verallgemeinerten Subset-Definitionsschema-Methode (GMSD) erreicht. Die Validierung erfolgte auf der Grundlage eines alphanumerischen Vergleichs, in dem ein ausgewähltes 3D-Produktmodell und die daraus entwickelte IFC-Sicht für das Produktkatalogmodell verglichen wurden. Es ergaben sich zwei Schlussfolgerungen. Im ersten Fall, der auch das Einheitspreisbeschaffungssystem betrifft, konnten die gewünschten Ergebnisse direkt durch Filterung der erforderlichen Daten erhalten werden. Beim Vergleich der Ergebnisse sowohl für Pauschal-, als auch für Entwurfs- und Baubeschaffungssysteme (Verträge) wurde jedoch festgestellt, dass für das IFC-Modell ein Erweiterungsbedarf besteht. Eine Lösung wurde über die Formalisierung der Kostendaten und Materialanalyseinformationen durch Erweiterung des IFC-Konzepts IfcBauRessource mit neuen Klassen und mit neuen Beziehungen erreicht. Somit erhält man ein allgemeines Informationsmodell auf der Grundlage des Datenschemas des IFC-Standards

Algorithms for Scheduling Problems

This edited book presents new results in the area of algorithm development for different types of scheduling problems. In eleven chapters, algorithms for single machine problems, flow-shop and job-shop scheduling problems (including their hybrid (flexible) variants), the resource-constrained project scheduling problem, scheduling problems in complex manufacturing systems and supply chains, and workflow scheduling problems are given. The chapters address such subjects as insertion heuristics for energy-efficient scheduling, the re-scheduling of train traffic in real time, control algorithms for short-term scheduling in manufacturing systems, bi-objective optimization of tortilla production, scheduling problems with uncertain (interval) processing times, workflow scheduling for digital signal processor (DSP) clusters, and many more

Probabilistic Estimation of Shale-Oil Resources in 93 Global Formations in 36 Countries

An in-depth probabilistic study of global shale-oil resources is presently absent in literature. In 2013, the Energy Information Agency (EIA) reported the total volumes of shale oil available in 36 countries to be 388 billion barrels of oil (BBO) following a volumetric assessment of several potentially shale-oil-rich global formations. However, their study did not take into consideration the inherent heterogeneity of shale formations and the resulting uncertainties in reservoir parameters used in the assessment. I employed a probabilistic approach to the volumetric estimation of the original-in-place shale oil (OOIPsh_oil) in 93 formations from these 36 countries (and the US) belonging to seven geographical regions. This was followed by reservoir-simulation studies of five US formations whereby generalized-recovery-factor (RF) distributions were established for three values of hydraulic-fracture stage spacing. These generalized- RF distributions were used to compute the technically-recoverable resources of shale oil (TRRsh_oil) from the 93 formations considered. The results were aggregated to regional and global levels assuming 100% dependence (arithmetic aggregation) as well as 100% independence (statistical aggregation) between summand formation-wise resource distributions. The arithmetically-aggregated volumes (P10-P50-P90) of OOIPsh_oil in the assessed global formations are 900-8,700-67,700 BBO (P90/P10 = 75), whereas the arithmetically-aggregated volumes of TRRsh_oil, using 50-ft stage spacing, are computed to be 25-374-3,906 BBO (P90/P10 = 159). Using statistical aggregation, the total volumes (P10-P50-P90) of OOIPsh_oil in the assessed global formations are 15,000-23,600-43,900 BBO (P90/P10 = 3), whereas the aggregated volumes of TRRsh_oil, using 50-ft stage spacing, are computed to be 250-1,300-3,100 BBO (P90/P10 = 12). The high values of P90/P10 highlight the large uncertainty in both arithmetically- and statistically aggregated estimates. Since true dependency between the resource distributions of aggregated formations is unknown, the actual resource estimates are expected to fall between the statistically- and arithmetically-aggregated estimates. Because this study does not consider the uncertainties in, and possible correlations between, all variables, the wider resources distributions from arithmetic aggregation may be deemed more appropriate than from statistical aggregation. This study accounts for only 41% of all global petroleum basins. Thus, actual world resources are likely to be considerably larger