Towards an evaluation framework for process mining algorithms

Although there has been a lot of progress in developing process mining algorithms in recent years, no effort has been put in developing a common means of assessing the quality of the models discovered by these algorithms. In this paper, we outline elements of an evaluation framework that is intended to enable (a) process mining researchers to compare the performance of their algorithms, and (b) end users to evaluate the validity of their process mining results. Furthermore, we describe two possible approaches to evaluate a discovered model (i) using existing comparison metrics that have been developed by the process mining research community, and (ii) based on the so-called k-fold-cross validation known from the machine learning community. To illustrate the application of these two approaches, we compared a set of models discovered by different algorithms based on a simple example log

The need for a process mining evaluation framework in research and practice

Although there has been much progress in developing process mining algorithms in recent years, no effort has been put in developing a common means of assessing the quality of the models discovered by these algorithms. In this paper, we motivate the need for such an evaluation mechanism, and outline elements of an evaluation framework that is intended to enable (a) process mining researchers to compare the performance of their algorithms, and (b) end users to evaluate the validity of their process mining results

ProM : the process mining toolkit

Nowadays, all kinds of information systems store detailed information in logs. Process mining has emerged as a way to analyze these systems based on these detailed logs. Unlike classical data mining, the focus of process mining is on processes. First, process mining allows us to extract a process model from an event log. Second, it allows us to detect discrepancies between a modeled process (as it was envisioned to be) and an event log (as it actually is). Third, it can enrich an existing model with knowledge derived from an event log. This paper presents our tool ProM, which is the world-leading tool in the area of process mining

Learning Hybrid Process Models From Events: Process Discovery Without Faking Confidence

Process discovery techniques return process models that are either formal (precisely describing the possible behaviors) or informal (merely a "picture" not allowing for any form of formal reasoning). Formal models are able to classify traces (i.e., sequences of events) as fitting or non-fitting. Most process mining approaches described in the literature produce such models. This is in stark contrast with the over 25 available commercial process mining tools that only discover informal process models that remain deliberately vague on the precise set of possible traces. There are two main reasons why vendors resort to such models: scalability and simplicity. In this paper, we propose to combine the best of both worlds: discovering hybrid process models that have formal and informal elements. As a proof of concept we present a discovery technique based on hybrid Petri nets. These models allow for formal reasoning, but also reveal information that cannot be captured in mainstream formal models. A novel discovery algorithm returning hybrid Petri nets has been implemented in ProM and has been applied to several real-life event logs. The results clearly demonstrate the advantages of remaining "vague" when there is not enough "evidence" in the data or standard modeling constructs do not "fit". Moreover, the approach is scalable enough to be incorporated in industrial-strength process mining tools.Comment: 25 pages, 12 figure

Methodological issues in cross-cultural research

Regardless of whether the research goal is to establish cultural universals or to identify and explain cross-cultural differences, researchers need measures that are comparable across different cultures when conducting cross-cultural studies. In this chapter, we describe two major strategies for enhancing cross-cultural comparability. First, we discuss a priori methods to ensure the comparability of data in cross-cultural surveys. In particular, we review findings on cross-cultural differences based on the psychology of survey response and provide suggestions on how to deal with these cultural differences in the survey design stage. Second, we discuss post hoc methods to ascertain data comparability and enable comparisons in the presence of threats to equivalence

Process Evaluation of a Dutch Community Intervention to improve Health Related Behaviour in deprived neighbourhoods

Objectives: To assess whether a community intervention on health related behaviour in deprived neighbourhoods was delivered as planned and the extent of exposure to the intervention programme. Methods: Data were gathered throughout the intervention period using minutes of meetings, registration forms and a postal questionnaire among residents in intervention and comparison neighbourhoods. Results: Overall, the intervention was delivered according to the key principles of a "community approach", although community participation could have been improved. Neighbourhood coalitions organized more than 50 health related activities in the neighbourhoods over a two-year period. Most activities were directed at attracting attention, providing information, and increasing awareness and knowledge, and at changing behaviours. Programme awareness and programme participation were 24% respectively 3% among residents in the intervention neighbourhoods. Conclusions: The process evaluation indicated that it was feasible to implement a community intervention according to the key principles of the "community approach" in deprived neighbourhoods. However, it is unlikely that the total package of intervention activities had enough strength and sufficient exposure to attain community-wide health behaviour change