144 research outputs found
Auto-adaptive multi-scale Laplacian Pyramids for modeling non-uniform data
Kernel-based techniques have become a common way for describing the local and global relationships of data samples that are generated in real-world processes. In this research, we focus on a multi-scale kernel based technique named Auto-adaptive Laplacian Pyramids (ALP). This method can be useful for function approximation and interpolation. ALP is an extension of the standard Laplacian Pyramids model that incorporates a modified Leave-One-Out Cross Validation procedure, which makes the method stable and automatic in terms of parameters selection without extra cost. This paper introduces a new algorithm that extends ALP to fit datasets that are non-uniformly distributed. In particular, the optimal stopping criterion will be point-dependent with respect to the local noise level and the sample rate. Experimental results over real datasets highlight the advantages of the proposed multi-scale technique for modeling and learning complex, high dimensional dataThey wish to thank Prof. Ronald R. Coifman for helpful remarks. They 525 also gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at Universidad Autónoma de Madrid. Funding: This work was supported by Spanish grants of the Ministerio de Ciencia, Innovación y Universidades [grant numbers: TIN2013-42351-P, TIN2015-70308-REDT, TIN2016-76406-P]; project CASI-CAM-CM supported by Madri+d 530 [grant number: S2013/ICE-2845]; project FACIL supported by Fundación BBVA (2016); and the UAM–ADIC Chair for Data Science and Machine Learnin
Diffusion maps and local models for wind power prediction
The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-33266-1_70Proceedings of 22nd International Conference on Artificial Neural Networks, Lausanne, Switzerland, September 11-14, 2012In this work we will apply Diffusion Maps (DM), a recent technique for dimensionality reduction and clustering, to build local models for wind energy forecasting. We will compare ridge regression models for K–means clusters obtained over DM features, against the models obtained for clusters constructed over the original meteorological data or principal components, and also against a global model. We will see that a combination of the DM model for the low wind power region and the global model elsewhere outperforms other options.With partial support from grant TIN2010-21575-C02-01
of Spain’s Ministerio de Economía y Competitividad and the UAM–ADIC Chair for Machine Learning in Modelling and Prediction. The first author is also supported by an FPI-UAM grant and kindly thanks the Applied Mathematics Department
of Yale University for receiving her during a visit. The second author is
supported by the FPU-MEC grant AP2008-00167. We also thank Red Eléctrica de España, Spain’s TSO, for providing historic wind energy dat
Diffusion, methods and applications
Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura: junio de 2014Big Data, an important problem nowadays, can be understood in terms of a very large number of
patterns, a very large pattern dimension or, often, both. In this thesis, we will concentrate on the
high dimensionality issue, applying manifold learning techniques for visualizing and analyzing
such patterns.
The core technique will be Di usion Maps (DM) and its Anisotropic Di usion (AD) version,
introduced by Ronald R. Coifman and his school at Yale University, and of which we will give
a complete, systematic, compact and self-contained treatment. This will be done after a brief
survey of previous manifold learning methods.
The algorithmic contributions of the thesis will be centered in two computational challenges of
di usion methods: the potential high cost of the similarity matrix eigenanalysis that is needed
to define the di usion embedding coordinates, and the di culty of computing this embedding
over new patterns not available for the initial eigenanalysis. With respect to the first issue, we
will show how the AD set up can be used to skip it when looking for local models. In this case,
local patterns will be selected through a k-Nearest Neighbors search using a properly defined
local Mahalanobis distance, that enables neighbors to be found over the latent variable space
underlying the AD model while we can work directly with the observable patterns and, thus,
avoiding the potentially costly similarity matrix eigenanalysis.
The second proposed algorithm, that we will call Auto-adaptative Laplacian Pyramids (ALP),
focuses in the out-of-sample embedding extension and consists in a modification of the classical
Laplacian Pyramids (LP) method. In this new algorithm the LP iterations will be combined with
an estimate of the Leave One Out CV error, something that makes possible to directly define
during training a criterion to estimate the optimal stopping point of this iterative algorithm.
This thesis will also present several application contributions to important problems in renewable
energy and medical imaging. More precisely, we will show how DM is a good method
for dimensionality reduction of meteorological weather predictions, providing tools to visualize
and describe these data, as well as to cluster them in order to define local models.
In turn, we will apply our AD-based localized search method first to find the location in the
human body of CT scan images and then to predict wind energy ramps on both individual farms
and over the whole of Spain. We will see that, in both cases, our results improve on the current
state of the art methods.
Finally, we will compare our ALP proposal with the well-known Nyström method as well as
with LP on two large dimensional problems, the time compression of meteorological data and
the analysis of meteorological variables relevant in daily radiation forecasts. In both cases we
will show that ALP compares favorably with the other approaches for out-of-sample extension
problemsBig Data es un problema importante hoy en día, que puede ser entendido en términos de un
amplio número de patrones, una alta dimensión o, como sucede normalmente, de ambos. Esta
tesis se va a centrar en problemas de alta dimensión, aplicando técnicas de aprendizaje de
subvariedades para visualizar y analizar dichos patrones.
La técnica central será Di usion Maps (DM) y su versión anisotrópica, Anisotropic Di usion
(AD), introducida por Ronald R. Coifman y su escuela en la Universidad de Yale, la cual va a
ser tratada de manera completa, sistemática, compacta y auto-contenida. Esto se llevará a cabo
tras un breve repaso de métodos previos de aprendizaje de subvariedades.
Las contribuciones algorítmicas de esta tesis estarán centradas en dos de los grandes retos en
métodos de difusión: el potencial alto coste que tiene el análisis de autovalores de la matriz de
similitud, necesaria para definir las coordenadas embebidas; y la dificultad para calcular este
mismo embedding sobre nuevos datos que no eran accesibles cuando se realizó el análisis de
autovalores inicial. Respecto al primer tema, se mostrará cómo la aproximación AD se puede
utilizar para evitar el cálculo del embedding cuando estamos interesados en definir modelos locales.
En este caso, se seleccionarán patrones cercanos por medio de una búsqueda de vecinos
próximos (k-NN), usando como distancia una medida de Mahalanobis local que permita encontrar
vecinos sobre las variables latentes existentes bajo el modelo de AD. Todo esto se llevará
a cabo trabajando directamente sobre los patrones observables y, por tanto, evitando el costoso
cálculo que supone el cálculo de autovalores de la matriz de similitud.
El segundo algoritmo propuesto, que llamaremos Auto-adaptative Laplacian Pyramids (ALP),
se centra en la extensión del embedding para datos fuera de la muestra, y se trata de una modificación
del método denominado Laplacian Pyramids (LP). En este nuevo algoritmo, las iteraciones
de LP se combinarán con una estimación del error de Leave One Out CV, permitiendo definir
directamente durante el periodo de entrenamiento, un criterio para estimar el criterio de parada
óptimo para este método iterativo.
En esta tesis se presentarán también una serie de contribuciones de aplicación de estas técnicas
a importantes problemas en energías renovables e imágenes médicas. Más concretamente, se
muestra como DM es un buen método para reducir la dimensión de predicciones del tiempo
meteorológico, sirviendo por tanto de herramienta de visualización y descripción, así como de
clasificación de los datos con vistas a definir modelos locales sobre cada grupo descrito.
Posteriormente, se aplicará nuestro método de búsqueda localizada basado en AD tanto a la
búsqueda de la correspondiente posición de tomografías en el cuerpo humano, como para la
detección de rampas de energía eólica en parques individuales o de manera global en España.
En ambos casos se verá como los resultados obtenidos mejoran los métodos del estado del arte
actual.
Finalmente se comparará el algoritmo de ALP propuesto frente al conocido método de Nyström
y al método de LP, en dos problemas de alta dimensión: el problema de compresión temporal
de datos meteorológicos y el análisis de variables meteorológicas relevantes para la predicción
de la radiación diaria. En ambos casos se mostrará cómo ALP es comparativamente mejor que
otras aproximaciones existentes para resolver el problema de extensión del embedding a puntos
fuera de la muestr
Positive semi-definite embedding for dimensionality reduction and out-of-sample extensions
In machine learning or statistics, it is often desirable to reduce the
dimensionality of a sample of data points in a high dimensional space
. This paper introduces a dimensionality reduction method where
the embedding coordinates are the eigenvectors of a positive semi-definite
kernel obtained as the solution of an infinite dimensional analogue of a
semi-definite program. This embedding is adaptive and non-linear. A main
feature of our approach is the existence of a non-linear out-of-sample
extension formula of the embedding coordinates, called a projected Nystr\"om
approximation. This extrapolation formula yields an extension of the kernel
matrix to a data-dependent Mercer kernel function. Our empirical results
indicate that this embedding method is more robust with respect to the
influence of outliers, compared with a spectral embedding method.Comment: 16 pages, 5 figures. Improved presentatio
- …