Intelligent Data Mining using Kernel Functions and Information Criteria

Radial Basis Function (RBF) Neural Networks and Support Vector Machines (SVM) are two powerful kernel related intelligent data mining techniques. The current major problems with these methods are over-fitting and the existence of too many free parameters. The way to select the parameters can directly affect the generalization performance(test error) of theses models. Current practice in how to choose the model parameters is an art, rather than a science in this research area. Often, some parameters are predetermined, or randomly chosen. Other parameters are selected through repeated experiments that are time consuming, costly, and computationally very intensive. In this dissertation, we provide a two-stage analytical hybrid-training algorithm by building a bridge among regression tree, EM algorithm, and Radial Basis Function Neural Networks together. Information Complexity (ICOMP) criterion of Bozdogan along with other information based criteria are introduced and applied to control the model complexity, and to decide the optimal number of kernel functions. In the first stage of the hybrid, regression tree and EM algorithm are used to determine the kernel function parameters. In the second stage of the hybrid, the weights (coefficients) are calculated and information criteria are scored. Kernel Principal Component Analysis (KPCA) using EM algorithm for feature selection and data preprocessing is also introduced and studied. Adaptive Support Vector Machines (ASVM) and some efficient algorithms are given to deal with massive data sets in support vector classifications. Versatility and efficiency of the new proposed approaches are studied on real data sets and via Monte Carlo sim- ulation experiments

Kernel-based Information Criterion

This paper introduces Kernel-based Information Criterion (KIC) for model selection in regression analysis. The novel kernel-based complexity measure in KIC efficiently computes the interdependency between parameters of the model using a variable-wise variance and yields selection of better, more robust regressors. Experimental results show superior performance on both simulated and real data sets compared to Leave-One-Out Cross-Validation (LOOCV), kernel-based Information Complexity (ICOMP), and maximum log of marginal likelihood in Gaussian Process Regression (GPR).Comment: We modified the reference 17, and the subcaptions of Figure

Hybrid Autoregressive Integrated Moving Average-Support Vector Regression for Stock Price Forecasting

Stock investment provides high-profit opportunities but also has a high risk of loss. Investors use various decision-making methods to minimize this risk, such as stock price forecasting. This research aims to predict daily closing stock prices using a hybrid Autoregressive Integrated Moving Average (ARIMA)-Support Vector Regression (SVR) model and compare it with the single model of ARIMA and SVR, as well as compiling the R-shiny web for the hybrid ARIMA-SVR model which makes it easier for investors to use the model to support investment decision making. The hybrid ARIMA-SVR model is composed of two components: the linear component from the results of stock price forecasting using the Autoregressive Integrated Moving Average (ARIMA) model and the nonlinear components from the residual forecasting results of the ARIMA model using the Support Vector Regression (SVR) model. The data used was closing stock price data from April 1, 2019, to April 1, 2021, from PT Unilever Indonesia Tbk (UNVR.JK), PT Perusahaan Gas Negara Tbk (PGAS.JK), and PT Telekomunikasi Indonesia Tbk (TLKM.JK), from the Yahoo Finance website. The research results conclude that the hybrid ARIMA-SVR model has excellent capabilities in forecasting stock prices with the MAPE values ​​for UNVR, PGAS, and TLKM stocks, respectively of 0.797%, 2.213%, and 0.993%, which are lower than the MAPE values of ARIMA-GARCH and SVR models. The hybrid model can be an alternative model with excellent capabilities in forecasting stock prices

Machine Learning (ML) module

Lectures notes of the machine learning content of the course TOML (Topics on Optimization and Machine Learning) at Master in Innovation and Research in Informatics (MIRI) at FIB, UPC.2023/202

Support Vector Machine and Its Difficulties From Control Field of View

The application of the Support Vector Machine (SVM) classification algorithm to large-scale datasets is limited due to its use of a large number of support vectors and dependency of its performance on its kernel parameter. In this paper, SVM is redefined as a control system and Iterative Learning Control (ILC) method is used to optimize SVM’s kernel parameter. The ILC technique first defines an error equation and then iteratively updates the kernel function and its regularization parameter using the training error and the previous state of the system. The closed-loop structure of the proposed algorithm increases the robustness of the technique to uncertainty and improves its convergence speed. Experimental results were generated using nine standard benchmark datasets covering a wide range of applications. Experimental results show that the proposed method generates superior or very competitive results in term of accuracy than those of classical and stateof-the-art SVM-based techniques while using a significantly smaller number of support vectors