Efficient tuning in supervised machine learning

The tuning of learning algorithm parameters has become more and more important during the last years. With the fast growth of computational power and available memory databases have grown dramatically. This is very challenging for the tuning of parameters arising in machine learning, since the training can become very time-consuming for large datasets. For this reason efficient tuning methods are required, which are able to improve the predictions of the learning algorithms. In this thesis we incorporate model-assisted optimization techniques, for performing efficient optimization on noisy datasets with very limited budgets. Under this umbrella we also combine learning algorithms with methods for feature construction and selection. We propose to integrate a variety of elements into the learning process. E.g., can tuning be helpful in learning tasks like time series regression using state-of-the-art machine learning algorithms? Are statistical methods capable to reduce noise e ffects? Can surrogate models like Kriging learn a reasonable mapping of the parameter landscape to the quality measures, or are they deteriorated by disturbing factors? Summarizing all these parts, we analyze if superior learning algorithms can be created, with a special focus on efficient runtimes. Besides the advantages of systematic tuning approaches, we also highlight possible obstacles and issues of tuning. Di fferent tuning methods are compared and the impact of their features are exposed. It is a goal of this work to give users insights into applying state-of-the-art learning algorithms profitably in practiceBundesministerium f ür Bildung und Forschung (Germany), Cologne University of Applied Sciences (Germany), Kind-Steinm uller-Stiftung (Gummersbach, Germany)Algorithms and the Foundations of Software technolog

Proceedings. 19. Workshop Computational Intelligence, Dortmund, 2. - 4. Dezember 2009

Dieser Tagungsband enthält die Beiträge des 19. Workshops „Computational Intelligence“ des Fachausschusses 5.14 der VDI/VDE-Gesellschaft für Mess- und Automatisierungstechnik (GMA) und der Fachgruppe „Fuzzy-Systeme und Soft-Computing“ der Gesellschaft für Informatik (GI), der vom 2.-4. Dezember 2009 im Haus Bommerholz bei Dortmund stattfindet

A modular genetic programming system

Genetic Programming (GP) is an evolutionary algorithm for the automatic discovery of symbolic expressions, e.g. computer programs or mathematical formulae, that encode solutions to a user-defined task. Recent advances in GP systems and computer performance made it possible to successfully apply this algorithm to real-world applications. This work offers three main contributions to the state-of-the art in GP systems: (I) The documentation of RGP, a state-of-the art GP software implemented as an extension package to the popular R environment for statistical computation and graphics. GP and RPG are introduced both formally and with a series of tutorial examples. As R itself, RGP is available under an open source license. (II) A comprehensive empirical analysis of modern GP heuristics based on the methodology of Sequential Parameter Optimization. The effects and interactions of the most important GP algorithm parameters are analyzed and recommendations for good parameter settings are given. (III) Two extensive case studies based on real-world industrial applications. The first application involves process control models in steel production, while the second is about meta-model-based optimization of cyclone dust separators. A comparison with traditional and modern regression methods reveals that GP offers equal or superior performance in both applications, with the additional benefit of understandable and easy to deploy models. Main motivation of this work is the advancement of GP in real-world application areas. The focus lies on a subset of application areas that are known to be practical for GP, first of all symbolic regression and classification. It has been written with practitioners from academia and industry in mind