A probabilistic evaluation procedure for process model matching techniques

Process model matching refers to the automatic identification of corresponding activities between two process models. It represents the basis for many advanced process model analysis techniques such as the identification of similar process parts or process model search. A central problem is how to evaluate the performance of process model matching techniques. Current evaluation methods require a binary gold standard that clearly defines which correspondences are correct. The problem is that often not even humans can agree on a set of correct correspondences. Hence, evaluating the performance of matching techniques based on a binary gold standard does not take the true complexity of the matching problem into account and does not fairly assess the capabilities of a matching technique. In this paper, we propose a novel evaluation procedure for process model matching techniques. In particular, we build on the assessments of multiple annotators to define the notion of a non-binary gold standard. In this way, we avoid the problem of agreeing on a single set of correct correspondences. Based on this non-binary gold standard, we introduce probabilistic versions of precision, recall, and F-measure as well as a distance-based performance measure. We use a dataset from the Process Model Matching Contest 2015 and a total of 16 matching systems to assess and compare the insights that can be obtained by using our evaluation procedure. We find that our probabilistic evaluation procedure allows us to gain more detailed insights into the performance of matching systems than a traditional evaluation based on a binary gold standard

Quantifying the Similarity of Non-bisimilar Labelled Transition Systems

International audienceEquivalence checking is an established technique for automatically verifying that two behavioural models (Labelled Transition Systems, LTSs) are equivalent from the point of view of an external observer. When these models are not equivalent, the checker returns a Boolean result with a counterexample, which is a sequence of actions leading to a state where the equivalence relation is not satisfied. However, this counterexample does not give any indication of how far the two LTSs are one from another. One can wonder whether they are almost identical or totally different, which is quite different from a design or debugging point of view. In this paper, we present an approach for measuring the similarity between two LTS models. The set of metrics is computed automatically using a tool we implemented. Beyond presenting the foundations of the proposed solution, we will show how it can be applied to two concrete application domains for supporting the construction of IoT applications on the one hand and for contributing to the process model matching problem on the other

On the expressive power of behavioral profiles

Behavioral profiles have been proposed as a behavioral abstraction of dynamic systems, specifically in the context of business process modeling. A behavioral profile can be seen as a complete graph over a set of task labels, where each edge is annotated with one relation from a given set of binary behavioral relations. Since their introduction, behavioral profiles were argued to provide a convenient way for comparing pairs of process models with respect to their behavior or computing behavioral similarity between process models. Still, as of today, there is little understanding of the expressive power of behavioral profiles. Via counter-examples, several authors have shown that behavioral profiles over various sets of behavioral relations cannot distinguish certain systems up to trace equivalence, even for restricted classes of systems represented as safe workflow nets. This paper studies the expressive power of behavioral profiles from two angles. Firstly, the paper investigates the expressive power of behavioral profiles and systems captured as acyclic workflow nets. It is shown that for unlabeled acyclic workflow net systems, behavioral profiles over a simple set of behavioral relations are expressive up to configuration equivalence. When systems are labeled, this result does not hold for any of several previously proposed sets of behavioral relations. Secondly, the paper compares the expressive power of behavioral profiles and regular languages. It is shown that for any set of behavioral relations, behavioral profiles are strictly less expressive than regular languages, entailing that behavioral profiles cannot be used to decide trace equivalence of finite automata and thus Petri nets

Reconciling Matching Networks of Conceptual Models

Conceptual models such as database schemas, ontologies or process models have been established as a means for effective engineering of information systems. Yet, for complex systems, conceptual models are created by a variety of stakeholders, which calls for techniques to manage consistency among the different views on a system. Techniques for model matching generate correspondences between the elements of conceptual models, thereby supporting effective model creation, utilization, and evolution. Although various automatic matching tools have been developed for different types of conceptual models, their results are often incomplete or erroneous. Automatically generated correspondences, therefore, need to be reconciled, i.e., validated by a human expert. We analyze the reconciliation process in a network setting, where a large number of conceptual models need to be matched. Then, the network induced by the generated correspondences shall meet consistency expectations in terms of mutual reinforcing relations between the correspondences. We develop a probabilistic model to identify the most uncertain correspondences in order to guide the expert's validation work. We also show how to construct a set of high-quality correspondences, even if the expert does not validate all generated correspondences. We demonstrate the efficiency of our techniques for real-world datasets in the domains of schema matching and ontology alignment

Agenda-driven case management

Im Gegensatz zu Routinetätigkeiten lassen sich wissensintensive Geschäftsprozesse – also Prozesse mit einem hohen Anteil an wissensintensiven Tätigkeiten, die von sogenannten Wissensarbeitern durchgeführt werden – nur schwer durch IT unterstützen. Das liegt vor allem daran, dass über den konkreten Lösungsweg und die dafür benötigten Daten nichts oder nur wenig im Vorfeld bekannt ist. Zwei wesentliche Ursachen hierfür sind, dass erstens der Ablauf von sehr vielen Parametern abhängig ist und dass zweitens diese Parameter sich auch über die Zeit verändern können. Solche Prozesse lassen sich unter anderem bei Trägern von Sozialleistungen oder in der privaten Versicherungswirtschaft beobachten. Dort steuern als Case Manager bezeichnete Wissensarbeiter komplizierte Leistungsfälle und koordinieren erforderliche Maßnahmen so, dass die Leistungen wirtschaftlich und bedarfsgerecht erbracht werden. Case Manager sind aufgrund ihrer Erfahrung, ihres breitgefächerten Fachwissens und der starken Vernetzung mit anderen Experten in der Lage, die wesentlichen Parameter der Prozesse zu erkennen, deren Veränderung stets nachzuverfolgen und den Ablauf entsprechend anzupassen. Wie in der Dissertation gezeigt wird, können wissensintensive Prozesse nicht mit den herkömmlichen Methoden des Process Mining analysiert und mit Workflow-Managementsystemen unterstützt werden. Deshalb werden neue Konzepte und alternative Ansätze vorgestellt und erprobt, um solche Prozesse analysierbar zu machen und Case Manager bei deren Ausführung zu unterstützen. Die zentralen Beiträge der Dissertation sind ein Metamodell mit den adCM-Grundkonzepten, ein Konzept zur anwendungsübergreifenden Protokollierung der Aktivitäten eines Case Managers unter Berücksichtigung des Metamodells (Monitoring), eine Methode zur Messung von Ereignisprotokollkomplexität, eine Methode zur Erhebung von Wissen über den Prozess auf Grundlage der Ereignisprotokolle (Discovery) und eine Werkzeugarchitektur zur operativen Unterstützung von Wissensarbeitern, um das Wissen über den Prozess kontextbezogen bereitzustellen

A Foundational Approach for Managing Process Variability

Abstract. A business process often shows different variations in a large organisation, due to different legal requirements in different countries, deviations in the IT infrastructure, or organisational differences. These variants are documented in separate independent process models. Management of these variants imposes various challenges. Invariant behaviour needs to be identified and redundancies among the variants have to be avoided. In this paper, we address these questions by defining a set-algebra for behavioural profiles. These profiles represent a behavioural abstraction of process models that can be computed efficiently. We trace back many questions of process variability management to set-theoretic operations and relations defined for behavioural profiles. As a validation, we apply our approach to an industry model collection.