Business process models are commonly used to document a company's operations. They describe internal processes in a chronological and logical order. Business process model matching refers to the automatic detection of semantically similar correspondences in process models. The output of those matching techniques is the basis for many applications. Currently, most research effort has been undertaken to improve the performance of such matching techniques. However, to support the improvement of process model matching techniques further, efficient and fair evaluation strategies are required. Moreover, information about the matching task, regarding the complexity of a data set have to be gathered. In the current literature, complexity is mostly associated with different level of granularity, thus 1:m or n:m correspondences. However, the evaluation should also account for different complexity aspects of the matching task, for example syntactical overlap of correspondences. Moreover, the evaluation of matching results actually strongly depends on the application. In this thesis, we therefore propose an application dependent evaluation. On the one hand, we introduce a non-binary evaluation, which better reflects the uncertainty of a gold standard and propose evaluation metrics, based on this non-binary gold standard which take different application scenarios into account.
On the other hand, we propose a conceptually novel evaluation procedure, which offers detailed information about strength and weaknesses of matchers without manually processing the matcher output. It therefore helps to find optimal application scenarios for specific matching techniques. It can further serve as a basis for a prediction for future matching tasks. We conduct experiments to show the insights gained by the introduced evaluation metrics and methods. Moreover, we apply the metrics at the OAEI 2016 and 2017
