Intelligent Data Mining using Kernel Functions and Information Criteria

Radial Basis Function (RBF) Neural Networks and Support Vector Machines (SVM) are two powerful kernel related intelligent data mining techniques. The current major problems with these methods are over-fitting and the existence of too many free parameters. The way to select the parameters can directly affect the generalization performance(test error) of theses models. Current practice in how to choose the model parameters is an art, rather than a science in this research area. Often, some parameters are predetermined, or randomly chosen. Other parameters are selected through repeated experiments that are time consuming, costly, and computationally very intensive. In this dissertation, we provide a two-stage analytical hybrid-training algorithm by building a bridge among regression tree, EM algorithm, and Radial Basis Function Neural Networks together. Information Complexity (ICOMP) criterion of Bozdogan along with other information based criteria are introduced and applied to control the model complexity, and to decide the optimal number of kernel functions. In the first stage of the hybrid, regression tree and EM algorithm are used to determine the kernel function parameters. In the second stage of the hybrid, the weights (coefficients) are calculated and information criteria are scored. Kernel Principal Component Analysis (KPCA) using EM algorithm for feature selection and data preprocessing is also introduced and studied. Adaptive Support Vector Machines (ASVM) and some efficient algorithms are given to deal with massive data sets in support vector classifications. Versatility and efficiency of the new proposed approaches are studied on real data sets and via Monte Carlo sim- ulation experiments

Latent sentiment model for weakly-supervised cross-lingual sentiment classification

In this paper, we present a novel weakly-supervised method for crosslingual sentiment analysis. In specific, we propose a latent sentiment model (LSM) based on latent Dirichlet allocation where sentiment labels are considered as topics. Prior information extracted from English sentiment lexicons through machine translation are incorporated into LSM model learning, where preferences on expectations of sentiment labels of those lexicon words are expressed using generalized expectation criteria. An efficient parameter estimation procedure using variational Bayes is presented. Experimental results on the Chinese product reviews show that the weakly-supervised LSM model performs comparably to supervised classifiers such as Support vector Machines with an average of 81% accuracy achieved over a total of 5484 review documents. Moreover, starting with a generic sentiment lexicon, the LSM model is able to extract highly domainspecific polarity words from text

Clasificación de puntos de operación de un sistema de bombeo con la ayuda de máquinas de soporte vectorial

This thesis tries to obtain information about how works the Support Vector Machines method for statistical learning applied to a pump within a hydraulic installation. Support Vector Machines, known as SVM, is a method that teaches a learning machine how to classify the data of a system. The system under study, in this case the pump, is tested in the loop and measured for a later classification. This method has been used in many applications due to its predicting function. Support Vector Machines involves a mode for turning a machine (the computer) able to make difference between the values of a parameter. So, the pumping sytem can be monitored and measured and then, recording the vibrations that the pump produces, the machine is trained for recognizing the vibrations and associate them with a value of the properties of the pumping system. In this way, the learning machine predict the values of the desired magnitude: ow rate or rotation speed among other. The machine has in advance the measures and learn from them and their values, that is why is called supervised learning. The accuracy that the learning machine develops is obtained comparing the predicted values with the actuals, and it is represented using a visual tool called confusion matrix. For the vibration is used a sensor coupled to the pump that takes the signal and provides it in order to be recorded with the values of the rest of the parameters measured. This project has been assisted by two different computer programs. First, LabView was used for the recording as an interface and then, Python applied the SVM method to the data. The sensors fixed in the workbench measure the properties of the pumping system and parallel, the accelerometer take the signal of the vibrations produced in the pump. In the LabView program, loops of 100 seconds are recorded with a established flow rate, rotation speed and sensor position. The measures are recorded varying the values of the mentioned magnitudes in order to make different groups of classes. After that, the data achieved feed the Python program. The classifications are done attending to different criteria and are analyzed in order to extract as much information and conclusions as possible.Ingeniería Industria

KERT: Automatic Extraction and Ranking of Topical Keyphrases from Content-Representative Document Titles

We introduce KERT (Keyphrase Extraction and Ranking by Topic), a framework for topical keyphrase generation and ranking. By shifting from the unigram-centric traditional methods of unsupervised keyphrase extraction to a phrase-centric approach, we are able to directly compare and rank phrases of different lengths. We construct a topical keyphrase ranking function which implements the four criteria that represent high quality topical keyphrases (coverage, purity, phraseness, and completeness). The effectiveness of our approach is demonstrated on two collections of content-representative titles in the domains of Computer Science and Physics.Comment: 9 page

Model Selection for Support Vector Machine Classification

We address the problem of model selection for Support Vector Machine (SVM) classification. For fixed functional form of the kernel, model selection amounts to tuning kernel parameters and the slack penalty coefficient $C$. We begin by reviewing a recently developed probabilistic framework for SVM classification. An extension to the case of SVMs with quadratic slack penalties is given and a simple approximation for the evidence is derived, which can be used as a criterion for model selection. We also derive the exact gradients of the evidence in terms of posterior averages and describe how they can be estimated numerically using Hybrid Monte Carlo techniques. Though computationally demanding, the resulting gradient ascent algorithm is a useful baseline tool for probabilistic SVM model selection, since it can locate maxima of the exact (unapproximated) evidence. We then perform extensive experiments on several benchmark data sets. The aim of these experiments is to compare the performance of probabilistic model selection criteria with alternatives based on estimates of the test error, namely the so-called ``span estimate'' and Wahba's Generalized Approximate Cross-Validation (GACV) error. We find that all the ``simple'' model criteria (Laplace evidence approximations, and the Span and GACV error estimates) exhibit multiple local optima with respect to the hyperparameters. While some of these give performance that is competitive with results from other approaches in the literature, a significant fraction lead to rather higher test errors. The results for the evidence gradient ascent method show that also the exact evidence exhibits local optima, but these give test errors which are much less variable and also consistently lower than for the simpler model selection criteria

Active Sampling-based Binary Verification of Dynamical Systems

Nonlinear, adaptive, or otherwise complex control techniques are increasingly relied upon to ensure the safety of systems operating in uncertain environments. However, the nonlinearity of the resulting closed-loop system complicates verification that the system does in fact satisfy those requirements at all possible operating conditions. While analytical proof-based techniques and finite abstractions can be used to provably verify the closed-loop system's response at different operating conditions, they often produce conservative approximations due to restrictive assumptions and are difficult to construct in many applications. In contrast, popular statistical verification techniques relax the restrictions and instead rely upon simulations to construct statistical or probabilistic guarantees. This work presents a data-driven statistical verification procedure that instead constructs statistical learning models from simulated training data to separate the set of possible perturbations into "safe" and "unsafe" subsets. Binary evaluations of closed-loop system requirement satisfaction at various realizations of the uncertainties are obtained through temporal logic robustness metrics, which are then used to construct predictive models of requirement satisfaction over the full set of possible uncertainties. As the accuracy of these predictive statistical models is inherently coupled to the quality of the training data, an active learning algorithm selects additional sample points in order to maximize the expected change in the data-driven model and thus, indirectly, minimize the prediction error. Various case studies demonstrate the closed-loop verification procedure and highlight improvements in prediction error over both existing analytical and statistical verification techniques.Comment: 23 page

Kratkoročna prognoza potrošnje električne energije zasnovana na metodama veštačke inteligencije

The topic of this dissertation is a short-term load forecasting using artificial intelligence methods. Three new models with least squares support vector machines for nonlinear regression are proposed. First proposed model is a model with forecasting in two stages. This model use additioal feature, maximum daily load which is not known for day ahead. Forecating of maximum daily load is obtained in the first stage. This forecasted value is used in second stage, where forecasting of hourly load is done. Model with feature selection, using mutual information for selection criteria, is a second proposed model. This model tries to find an optimal feature set for a given problem. Forecasting model based on an incremental update scheme is a third proposed model. This model is based on the incremental update of the initial training set by adding new instances into it as soon as they become available and throwing out the old ones. Then the model is trained with new training set. By this approach the evolving nature of the load pattern is followed and the model performance is preserved and improved. For models evaluation, the forecasting of hourly loads for one year is done. Electrical consumption data for the City of Niš, which have about 260000 habitans and average daily demand of 182 MW, is used for testing. Double sesonal ARIMA and Holt-Winters as representatives of clasical models and artificial neural networks, least squares support vector machines and relevance vector machines as representatives of artificial models, are used for models evaluation. For a measure of accuracy, mean absolute percentage error, symetrical mean absolute percentage error, square root mean error and absolute percentage error are used. Obtained results show that the best model is model with incremental update scheme, followed by double sesonal ARIMA and artificial neural networks models. The worst results are obtained by relevance vector machines and double sesonal Holt-Winters models. It has been shown that the best model could be successfully used with the short-term load forecasting problem

Star-galaxy separation in the AKARI NEP Deep Field

Context: It is crucial to develop a method for classifying objects detected in deep surveys at infrared wavelengths. We specifically need a method to separate galaxies from stars using only the infrared information to study the properties of galaxies, e.g., to estimate the angular correlation function, without introducing any additional bias. Aims. We aim to separate stars and galaxies in the data from the AKARI North Ecliptic Pole (NEP) Deep survey collected in nine AKARI / IRC bands from 2 to 24 {\mu}m that cover the near- and mid-infrared wavelengths (hereafter NIR and MIR). We plan to estimate the correlation function for NIR and MIR galaxies from a sample selected according to our criteria in future research. Methods: We used support vector machines (SVM) to study the distribution of stars and galaxies in the AKARIs multicolor space. We defined the training samples of these objects by calculating their infrared stellarity parameter (sgc). We created the most efficient classifier and then tested it on the whole sample. We confirmed the developed separation with auxiliary optical data obtained by the Subaru telescope and by creating Euclidean normalized number count plots. Results: We obtain a 90% accuracy in pinpointing galaxies and 98% accuracy for stars in infrared multicolor space with the infrared SVM classifier. The source counts and comparison with the optical data (with a consistency of 65% for selecting stars and 96% for galaxies) confirm that our star/galaxy separation methods are reliable. Conclusions: The infrared classifier derived with the SVM method based on infrared sgc- selected training samples proves to be very efficient and accurate in selecting stars and galaxies in deep surveys at infrared wavelengths carried out without any previous target object selection.Comment: 8 pages, 8 figure