Hybrid Model Based on Genetic Algorithms and SVM Applied to Variable Selection Within Fruit Juice Classification

Research article[Abstract] Given the background of the use of Neural Networks in problems of apple juice classification, this paper aim at implementing a newly developed method in the field of machine learning: the Support Vector Machines (SVM). Therefore, a hybrid model that combines genetic algorithms and support vector machines is suggested in such a way that, when using SVM as a fitness function of the Genetic Algorithm (GA), the most representative variables for a specific classification problem can be selected

N+3 Small Commercial Efficient and Quiet Transportation for Year 2030-2035

This study develops a future scenario that enables convenient point-to-point commercial air travel via a large network of community airports and a new class of small airliners. A network demand and capacity study identifies current and future air travel demands and the capacity of this new network to satisfy these demands. A current technology small commercial airliner is defined to meet the needs of the new network, as a baseline for evaluating the improvement brought about by advanced technologies. Impact of this new mode of travel on the infrastructure and surrounding communities of the small airports in this new N+3 network are also evaluated. Year 2030-2035 small commercial airliner technologies are identified and a trade study conducted to evaluate and select those with the greatest potential for enhancing future air travel and the study metrics. The selected advanced air vehicle concept is assessed against the baseline aircraft, and an advanced, but conventional aircraft, and the study metrics. The key technologies of the selected advanced air vehicle are identified, their impact quantified, and risk assessments and roadmaps defined

Advances in boosting of temporal and spatial models

Boosting is an iterative algorithm for functional approximation and numerical optimization which can be applied to solve statistical regression-type problems. By design, boosting can mimic the solutions of many conventional statistical models, such as the linear model, the generalized linear model, and the generalized additive model, but its strength is to enhance these models or even go beyond. It enjoys increasing attention since a) it is a generic algorithm, easily extensible to exciting new problems, and b) it can cope with``difficult'' data where conventional statistical models fail. In this dissertation, we design autoregressive time series models based on boosting which capture nonlinearity in the mean and in the variance, and propose new models for multi-step forecasting of both. We use a special version of boosting, called componentwise gradient boosting, which is innovative in the estimation of the conditional variance of asset returns by sorting out irrelevant (lagged) predictors. We propose a model which enables us not only to identify the factors which drive market volatility, but also to assess the specific nature of their impact. Therefore, we gain a deeper insight into the nature of the volatility processes. We analyze four broad asset classes, namely, stocks, commodities, bonds, and foreign exchange, and use a wide range of potential macro and financial drivers. The proposed model for volatility forecasting performs very favorably for stocks and commodities relative to the common GARCH(1,1) benchmark model. The advantages are particularly convincing for longer forecasting horizons. To our knowledge, the application of boosting to multi-step forecasting of either the mean or the variance has not been done before. In a separate study, we focus on the conditional mean of German industrial production. With boosting, we improve the forecasting accuracy when compared to several competing models including the benchmark in this field, the linear autoregressive model. In an exhaustive simulation study we show that boosting of high-order nonlinear autoregressive time series can be very competitive in terms of goodness-of-fit when compared to alternative nonparametric models. Finally, we apply boosting in a spatio-temporal context to data coming from outside the econometric field. We estimate the browsing pressure on young beech trees caused by the game species within the borders of the Bavarian Forest National Park ``Bayerischer Wald,'' Germany. We found that using the geographic coordinates of the browsing cases contributes considerably to the fit. Furthermore, this bivariate geographic predictor is better suited for prediction if it allows for abrupt changes in the browsing pressure

Automation and Control Architecture for Hybrid Pipeline Robots

The aim of this research project, towards the automation of the Hybrid Pipeline Robot (HPR), is the development of a control architecture and strategy, based on reconfiguration of the control strategy for speed-controlled pipeline operations and self-recovering action, while performing energy and time management. The HPR is a turbine powered pipeline device where the flow energy is converted to mechanical energy for traction of the crawler vehicle. Thus, the device is flow dependent, compromising the autonomy, and the range of tasks it can perform. The control strategy proposes pipeline operations supervised by a speed control, while optimizing the energy, solved as a multi-objective optimization problem. The states of robot cruising and self recovering, are controlled by solving a neuro-dynamic programming algorithm for energy and time optimization, The robust operation of the robot includes a self-recovering state either after completion of the mission, or as a result of failures leading to the loss of the robot inside the pipeline, and to guaranteeing the HPR autonomy and operations even under adverse pipeline conditions Two of the proposed models, system identification and tracking system, based on Artificial Neural Networks, have been simulated with trial data. Despite the satisfactory results, it is necessary to measure a full set of robot’s parameters for simulating the complete control strategy. To solve the problem, an instrumentation system, consisting on a set of probes and a signal conditioning board, was designed and developed, customized for the HPR’s mechanical and environmental constraints. As a result, the contribution of this research project to the Hybrid Pipeline Robot is to add the capabilities of energy management, for improving the vehicle autonomy, increasing the distances the device can travel inside the pipelines; the speed control for broadening the range of operations; and the self-recovery capability for improving the reliability of the device in pipeline operations, lowering the risk of potential loss of the robot inside the pipeline, causing the degradation of pipeline performance. All that means the pipeline robot can target new market sectors that before were prohibitive

Gradient boosting in automatic machine learning: feature selection and hyperparameter optimization

Das Ziel des automatischen maschinellen Lernens (AutoML) ist es, alle Aspekte der Modellwahl in prädiktiver Modellierung zu automatisieren. Diese Arbeit beschäftigt sich mit Gradienten Boosting im Kontext von AutoML mit einem Fokus auf Gradient Tree Boosting und komponentenweisem Boosting. Beide Techniken haben eine gemeinsame Methodik, aber ihre Zielsetzung ist unterschiedlich. Während Gradient Tree Boosting im maschinellen Lernen als leistungsfähiger Vorhersagealgorithmus weit verbreitet ist, wurde komponentenweises Boosting im Rahmen der Modellierung hochdimensionaler Daten entwickelt. Erweiterungen des komponentenweisen Boostings auf multidimensionale Vorhersagefunktionen werden in dieser Arbeit ebenfalls untersucht. Die Herausforderung der Hyperparameteroptimierung wird mit Fokus auf Bayesianische Optimierung und effiziente Stopping-Strategien diskutiert. Ein groß angelegter Benchmark über Hyperparameter verschiedener Lernalgorithmen, zeigt den kritischen Einfluss von Hyperparameter Konfigurationen auf die Qualität der Modelle. Diese Daten können als Grundlage für neue AutoML- und Meta-Lernansätze verwendet werden. Darüber hinaus werden fortgeschrittene Strategien zur Variablenselektion zusammengefasst und eine neue Methode auf Basis von permutierten Variablen vorgestellt. Schließlich wird ein AutoML-Ansatz vorgeschlagen, der auf den Ergebnissen und Best Practices für die Variablenselektion und Hyperparameteroptimierung basiert. Ziel ist es AutoML zu vereinfachen und zu stabilisieren sowie eine hohe Vorhersagegenauigkeit zu gewährleisten. Dieser Ansatz wird mit AutoML-Methoden, die wesentlich komplexere Suchräume und Ensembling Techniken besitzen, verglichen. Vier Softwarepakete für die statistische Programmiersprache R sind Teil dieser Arbeit, die neu entwickelt oder erweitert wurden: mlrMBO: Ein generisches Paket für die Bayesianische Optimierung; autoxgboost: Ein AutoML System, das sich vollständig auf Gradient Tree Boosting fokusiert; compboost: Ein modulares, in C++ geschriebenes Framework für komponentenweises Boosting; gamboostLSS: Ein Framework für komponentenweises Boosting additiver Modelle für Location, Scale und Shape.The goal of automatic machine learning (AutoML) is to automate all aspects of model selection in (supervised) predictive modeling. This thesis deals with gradient boosting techniques in the context of AutoML with a focus on gradient tree boosting and component-wise gradient boosting. Both techniques have a common methodology, but their goal is quite different. While gradient tree boosting is widely used in machine learning as a powerful prediction algorithm, component-wise gradient boosting strength is in feature selection and modeling of high-dimensional data. Extensions of component-wise gradient boosting to multidimensional prediction functions are considered as well. Focusing on Bayesian optimization and efficient early stopping strategies the challenge of hyperparameter optimization for these algorithms is discussed. Difficulty in the optimization of these algorithms is shown by a large scale random search on hyperparameters for machine learning algorithms, that can build the foundation of new AutoML and metalearning approaches. Furthermore, advanced feature selection strategies are summarized and a new method based on shadow features is introduced. Finally, an AutoML approach based on the results and best practices for feature selection and hyperparameter optimization is proposed, with the goal of simplifying and stabilizing AutoML while maintaining high prediction accuracy. This is compared to AutoML approaches using much more complex search spaces and ensembling techniques. Four software packages for the statistical programming language R have been newly developed or extended as a part of this thesis: mlrMBO: A general framework for Bayesian optimization; autoxgboost: An automatic machine learning framework that heavily utilizes gradient tree boosting; compboost: A modular framework for component-wise boosting written in C++; gamboostLSS: A framework for component-wise boosting for generalized additive models for location scale and shape