Feature network methods for machine learning

We develop a graph structure for feature vectors in machine learning, which we denote as a feature network (FN); this is different from sample-based networks, in which nodes simply represent samples. FNs reveal the underlying relationship among feature vector components and re-represent features as functions on a network. Our study focuses on using FN structures to extract underlying information and thus improve machine learning performance. Upon the representation of feature vectors as such functions, so-called graph signal processing, or graph functional analytic techniques can be implemented, consisting of analytic operations including differentiation and integration of feature vectors. Our motivation originated from a study using infrared spectroscopy data, where domain experts prefer using the second derivative information rather than the original data; this is an illustration of the potential power of understanding the underlying feature structure. We begin by developing a classification method based on the premise that is assuming data from different classes (e.g., different cancer subtypes) will have distinct underlying graph structures, for graphs consisting of genes as nodes and gene covariances as edges. That is, a feature vector from one class will tend to be "smooth" on the related FN, and "fluctuate" in the other FNs. This method, using an entirely new set of features from standard ones, on its own proves to somewhat outperform SVM and KNN in classifying cancer subtypes in infrared spectroscopy data and gene expression data. We are effectively also projecting high-dimensional data into a low dimensional representation of graph smoothness, providing a unique way of data visualization. Additionally, FNs represent new ways of thinking about data. With a graph structure for feature vectors, graphical functional analysis can be used to extract various types of information not apparent in the original feature vectors. Specifically, operations such as calculus, Fourier transforms, and convolutions can be performed on the graph vertex domain. We introduce a family of calculus-like operators in reproducing kernel Hilbert spaces for feature vector regularization to deal with two types of data deficiency, which we designate as noise and blurring. Such operations are generalized from widely used ones in computer vision. The derivative operations on feature vectors provide additional information by amplifying differences between highly correlated features. Integrating feature vectors smooths and denoises them. Applications show that those denoising and deblurring operators can improve classification algorithms. The feature network with deep learning can be naturally extended to graph convolutional networks. We proposed a deep multiscale clustering structure with small learning complexity on general graph distance structures. This framework substantially reduces the number of parameters, and it allows the introduction of general machine learning algorithms such as SVM to feed-forward in this deep structure

Towards machine learning applied to time series based network traffic forecasting

This TFG will explore some specific use cases of the application of Machine Learning techniques to Software-Define Networks, in particular to overlay protocols such as LISP, VXLAN, etc.The aim of this project is to implement a network traffic forecasting model using time series and improve its performance with machine learning techniques, offering a better prediction based in outlier correction. This is a project developed in the Computer Architecture Department (DAC) at the Universitat Politècnica de Catalunya (UPC). Time Series modeling methodology is able to shape a trend and take care of any existing outlier, however it does not cover outlier impact on forecasting. In order to achieve more precision and better confidence intervals, the model combines outlier detection methodology and Artificial Neural Networks to quantify and predict outliers. A study is realized over external data to find out if there is an improvement and its effect on the predictions. Machine learning techniques as Artificial Neural Networks has proven to be an improvement of the current methodology to realize forecasting using Time Series modeling. Future work will be oriented to create an improved standard of this system focused on generalize the model.El objetivo de este proyecto es implementar un modelo de previsión de tráfico de red utilizando series temporales y mejorar su rendimiento con técnicas de aprendizaje automático, generando una mejor predicción basada en la corrección de valores atípicos. Se trata de un proyecto desarrollado en el Departamento de Arquitectura de Computadores (DAC) de la Universidad Politécnica de Cataluña (UPC). La metodología de modelado de series temporales es capaz de predecir una tendencia y hacerse cargo de cualquier valor atípico ya existente, sin embargo, no cubre el impacto de estos sobre la predicción. Con el fin de lograr una mayor precisión y mejores intervalos de confianza, el modelo combina la metodología de detección de valores atípicos y redes neuronales artificiales para cuantificar y predecir los atípicos. Un estudio se realiza sobre datos externos para averiguar si hay una mejora y su efecto sobre las predicciones. Las técnicas de aprendizaje automático, como redes neuronales artificiales, han demostrado ser una mejora de la metodología actual para realizar la predicción utilizando modelos de series de tiempo. El trabajo futuro se orientará para crear un mejor nivel de este sistema se centró en generalizar el modelo.L'objectiu d'aquest projecte és implementar un model de previsió de tràfic de xarxa utilitzant sèries temporals i millorar el seu rendiment amb tècniques d'aprenentatge automàtic, generant una millor predicció basada en la correcció de valors atípics. Es tracta d'un projecte desenvolupat al Departament d'Arquitectura de Computadors (DAC) de la Universitat Politècnica de Catalunya (UPC). La metodologia de modelatge de sèries temporals és capaç de predir una tendència i fer-se càrrec de qualsevol valor atípic ja existent, però, no cobreix l'impacte d'aquests sobre la predicció. Per tal d'aconseguir una major precisió i millors intervals de confiança, el model combina la metodologia de detecció de valors atípics i xarxes neuronals artificials per quantificar i predir els atípics. Un estudi es realitza sobre dades externes per esbrinar si hi ha una millora i el seu efecte sobre les prediccions. Les tècniques d'aprenentatge automàtic, com xarxes neuronals artificials, han demostrat ser una millora de la metodologia actual per a fer predicció utilitzant models de sèries de temps. El treball futur s'orientarà per crear un millor nivell d'aquest sistema es va centrar en generalitzar el model

Bayesian Cluster Enumeration Criterion for Unsupervised Learning

We derive a new Bayesian Information Criterion (BIC) by formulating the problem of estimating the number of clusters in an observed data set as maximization of the posterior probability of the candidate models. Given that some mild assumptions are satisfied, we provide a general BIC expression for a broad class of data distributions. This serves as a starting point when deriving the BIC for specific distributions. Along this line, we provide a closed-form BIC expression for multivariate Gaussian distributed variables. We show that incorporating the data structure of the clustering problem into the derivation of the BIC results in an expression whose penalty term is different from that of the original BIC. We propose a two-step cluster enumeration algorithm. First, a model-based unsupervised learning algorithm partitions the data according to a given set of candidate models. Subsequently, the number of clusters is determined as the one associated with the model for which the proposed BIC is maximal. The performance of the proposed two-step algorithm is tested using synthetic and real data sets.Comment: 14 pages, 7 figure