Evaluating learning algorithms for stochastic finite automata - comparative empirical analyses on learning models for technical systems

Finite automata are used to model a large variety of technical systems and form the basis of important tasks such as model-based development, early simulations and model-based diagnosis. However, such models are today still mostly derived manually, in an expensive and time-consuming manner. Therefore in the past twenty years, several successful algorithms have been developed for learning various types of finite automata. These algorithms use measurements of the technical systems to automatically derive the underlying automata models. However, today users face a serious problem when looking for such model learning algorithm: Which algorithm to choose for which problem and which technical system? This papers closes this gap by comparative empirical analyses of the most popular algorithms (i) using two real-world production facilities and (ii) using artificial datasets to analyze the algorithms' convergence and scalability. Finally, based on these results, several observat ions for choosing an appropriate automaton learning algorithm for a specific problem are given