Dynamic learning with neural networks and support vector machines

Neural network approach has proven to be a universal approximator for nonlinear continuous functions with an arbitrary accuracy. It has been found to be very successful for various learning and prediction tasks. However, supervised learning using neural networks has some limitations because of the black box nature of their solutions, experimental network parameter selection, danger of overfitting, and convergence to local minima instead of global minima. In certain applications, the fixed neural network structures do not address the effect on the performance of prediction as the number of available data increases. Three new approaches are proposed with respect to these limitations of supervised learning using neural networks in order to improve the prediction accuracy.;Dynamic learning model using evolutionary connectionist approach . In certain applications, the number of available data increases over time. The optimization process determines the number of the input neurons and the number of neurons in the hidden layer. The corresponding globally optimized neural network structure will be iteratively and dynamically reconfigured and updated as new data arrives to improve the prediction accuracy. Improving generalization capability using recurrent neural network and Bayesian regularization. Recurrent neural network has the inherent capability of developing an internal memory, which may naturally extend beyond the externally provided lag spaces. Moreover, by adding a penalty term of sum of connection weights, Bayesian regularization approach is applied to the network training scheme to improve the generalization performance and lower the susceptibility of overfitting. Adaptive prediction model using support vector machines . The learning process of support vector machines is focused on minimizing an upper bound of the generalization error that includes the sum of the empirical training error and a regularized confidence interval, which eventually results in better generalization performance. Further, this learning process is iteratively and dynamically updated after every occurrence of new data in order to capture the most current feature hidden inside the data sequence.;All the proposed approaches have been successfully applied and validated on applications related to software reliability prediction and electric power load forecasting. Quantitative results show that the proposed approaches achieve better prediction accuracy compared to existing approaches

LSTM Learning with Bayesian and Gaussian Processing for Anomaly Detection in Industrial IoT

The data generated by millions of sensors in Industrial Internet of Things (IIoT) is extremely dynamic, heterogeneous, and large scale. It poses great challenges on the real-time analysis and decision making for anomaly detection in IIoT. In this paper, we propose a LSTM-Gauss-NBayes method, which is a synergy of the long short-term memory neural network (LSTM-NN) and the Gaussian Bayes model for outlier detection in IIoT. In a nutshell, the LSTM-NN builds model on normal time series. It detects outliers by utilising the predictive error for the Gaussian Naive Bayes model. Our method exploits advantages of both LSTM and Gaussian Naive Bayes models, which not only has strong prediction capability of LSTM for future time point data, but also achieves an excellent classification performance of Gaussian Naive Bayes model through the predictive error. Empirical studies demonstrate our solution outperforms the best-known competitors, which is a preferable choice for detecting anomalies

Prediction Analysis of Floods Using Machine Learning Algorithms (NARX & SVM)

The changing patterns and behaviors of river water levels that may lead to flooding are an interesting and practical research area. They are configured to mitigate economic and societal implications brought about by floods. Non-linear (NARX) and Support Vector Machine (SVM) are machine learning algorithms suitable for predicting changes in levels of river water, thus detection of flooding possibilities. The two algorithms employ similar hydrological and flood resource variables such as precipitation amount, river inflow, peak gust, seasonal flow, flood frequency, and other relevant flood prediction variables. In the process of predicting floods, the water level is the most important hydrological research aspect. Prediction using machine-learning algorithms is effective due to its ability to utilize data from various sources and classify and regress it into flood and non-flood classes. This paper gives insight into mechanism of the two algorithm in perspective of flood estimation

Aprendizaje automático aplicado al modelado de viento y olas

Trabajo de Fin de Grado en Ingeniería del Software, Facultad de Informática UCM, Departamento de Arquitectura de Computadores y Automática, Curso 2019/2020In the fight against climate change, Offshore wind energy is at the forefront, in the development phase. The problem with turbines anchored to the seabed lies in the enormous cost of installation and maintenance, leading to the theoretical approach of floating offshore wind turbines. However, floating turbines are exposed to new wave loads and stronger wind loads. To enable their implementation while maximizing the electricity production and ensuring the protection of the structure, more accurate predictive models than the physical and statistical ones found in the literature are needed for the metocean (meteorological and oceanographic) variables involved. This project aims to model the wind speed in the time domain, the significant waves height in the frequency domain and the misalignment between wind and waves direction in the time domain, applying Machine Learning techniques. Offshore data collection as well as an exploratory data analysis and data cleaning phases have been carried out. Subsequently, the following algorithms were applied to train the models: Linear Regression, Support Vector Machines for Regression, Gaussian Process Regression and Neural Networks. Nonlinear Autoregressive with exogenous input neural networks (NARX) have proved to be the best algorithm both for wind speed and misalignment forecasting and the most accurate predictive model for significant waves height prediction has been the Gaussian Process Regression (GPR). In this project we demonstrated the ability of Machine Learning algorithms to model wind variables of a stochastic nature and waves. We emphasize the importance of evaluating the models through techniques such as Learning Curves to make better decisions to optimize them. This work not only makes predictive models available for later use, but it is also a pioneer in misalignment modelling, leaving a door open for future research.En la lucha contra el cambio climático, la energía eólica marina se sitúa en cabeza encontrándose en fase de desarrollo. El problema de las turbinas ancladas al lecho marino reside en el enorme coste de instalación y mantenimiento, llevando al planteamiento teórico de turbinas eólicas marinas flotantes. Estas, sin embargo, están expuestas a nuevas cargas de olas y cargas de viento más fuertes. Para hacer posible su implantación maximizando la producción eléctrica a la vez que asegurando la protección de la estructura, se necesita disponer de modelos predictivos más precisos que los físicos y estadísticos de la literatura para las variables metoceánicas (meteorológicas y oceánicas) implicadas. El objetivo de este proyecto es modelar la velocidad del viento en el dominio del tiempo, la altura significativa de la ola en el dominio de la frecuencia y la desalineación entre la dirección del viento y de las olas en el dominio temporal, aplicando técnicas de Aprendizaje Automático. Se ha llevado a cabo una fase de recopilación de datos medidos en alta mar, así como el análisis exploratorio y limpieza de los mismos. Posteriormente, para el entrenamiento de los modelos se aplicaron los algoritmos: Regresión Lineal, Máquinas de Vectores Soporte para Regresión, Proceso de Regresión Gausiano y Redes Neuronales. Las redes neuronales autorregresivas no lineales con entrada externa (NARX) han resultado ser el mejor algoritmo tanto para la predicción de la velocidad del viento como para la desalineación y para la altura significativa de la ola el modelo predictivo más preciso ha sido el proceso regresivo gausiano (GPR). En este proyecto demostramos la capacidad de los algoritmos de Aprendizaje Automático para modelar las variables del viento de naturaleza estocástica y del oleaje. Destacamos la importancia de la evaluación de los modelos mediante técnicas como las Curvas de Aprendizaje para tomar mejores decisiones en la optimización de los mismos. Este trabajo no pone solo a disposición modelos predictivos para su posterior uso, además es pionero en el modelado de la desalineación dejando una puerta abierta a futuras investigaciones.Depto. de Arquitectura de Computadores y AutomáticaFac. de InformáticaTRUEunpu

Neural network applications to reservoirs: Physics-based models and data models

International audienceEditoria

Machine learning for flow field measurements: a perspective

Advancements in machine-learning (ML) techniques are driving a paradigm shift in image processing. Flow diagnostics with optical techniques is not an exception. Considering the existing and foreseeable disruptive developments in flow field measurement techniques, we elaborate this perspective, particularly focused to the field of particle image velocimetry. The driving forces for the advancements in ML methods for flow field measurements in recent years are reviewed in terms of image preprocessing, data treatment and conditioning. Finally, possible routes for further developments are highlighted.Stefano Discetti acknowledges funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 949085). Yingzheng Liu acknowledges financial support from the National Natural Science Foundation of China (11725209)

Nonlinear forecasting of stream flows using a chaotic approach and artificial neural networks

This paper evaluates the forecasting performance of two nonlinear models, k-nearest neighbor (kNN) and feed-forward neural networks (FFNN), using stream flow data of the Kızılırmak River, the longest river in Turkey. For the kNN model, the required parameters are delay time, number of nearest neigh- bors and embedding dimension. The optimal delay time was obtained with the mutual information function; the number of nearest neighbors was obtained with the optimization process that minimi- zes RMSE as a function of the neighbor number and the embedding dimension was obtained with the correlation dimension method. The correlation dimension of the Kızılırmak River was d = 2.702, which was used in forming the input structure of the FFNN. The nearest integer above the correlation dimension (i.e., 3) provided the minimal number of required variables to characterize the system, and the maximum number of required variables was obtained with the nearest integer above the value 2d + 1 (Takens, 1981) (i.e., 7). Two FFNN models were developed that incorporate 3 and 7 lagged discharge values and the predicted performance compared to that of the kNN model. The results showed that the kNN model was superior to the FFNN model in stream flow forecasting. However, as a result from the kNN model structure, the model failed in the prediction of peak values. Additionally, it was found that the correlation dimension (if it existed) could successfully be used in time series where the determina- tion of the input structure is difficult because of high inter-dependency, as in stream flow time series. ResumenEste trabajo evalúa el desempeño de pronóstico de dos modelos no lineares, de método de clasificación no paramétrico kNN y de redes neuronales con alimentación avanzada (FNNN), usando datos de flujo del río Kizilirmak, el mayor de Turquía. Para el modelo kNN, los parámetros requeridos son tiempo de retraso, número de vecindarios cercanos y dimensión de encrustamiento. El tiempo óptimo de retraso fue obtenido con la función de información mutua; el número de vecindarios cercanos fue obtenido con la optimización de procesos que minimizan el RMSE como una función del número de vecindarios y la dimensión de incrus- tación fue obtenida con el método de dimensión correlativa. La dimensión de correlación del río Kizilirmak fue utilizado en la formación de la estructura de ingreso de las redes FFNN. La integración cercana sobre la dimensión de correlación proveyó el número mínimo de variables requeridas para caracterizar el sistema y el número máximo de variables requeridas fue obtenido con el número entero por encima del valor (Takens, 1981). Se desarrollaron dos modelos de redes FNNN que incorporan 3 y 7 valores de descargas retrasadas y el desempeño de predicción comparado con el modelo kNN. Los resultados muestran que el modelo kNN fue superior al modelo de redes FFNN en el flujo de pronósticos. Sin embargo, como un resultado del modelo de estructura kNN, el modelo falla en los valores pico. Adicionalmente, se encontró que la dimensión de correla- ción (de existir) podría ser usada eficientemente en series temporales donde la determinación de estructura de ingreso es difícil por la gran interdependencia, como en las series temporales de flujo

Maximally Machine-Learnable Portfolios

When it comes to stock returns, any form of predictability can bolster risk-adjusted profitability. We develop a collaborative machine learning algorithm that optimizes portfolio weights so that the resulting synthetic security is maximally predictable. Precisely, we introduce MACE, a multivariate extension of Alternating Conditional Expectations that achieves the aforementioned goal by wielding a Random Forest on one side of the equation, and a constrained Ridge Regression on the other. There are two key improvements with respect to Lo and MacKinlay's original maximally predictable portfolio approach. First, it accommodates for any (nonlinear) forecasting algorithm and predictor set. Second, it handles large portfolios. We conduct exercises at the daily and monthly frequency and report significant increases in predictability and profitability using very little conditioning information. Interestingly, predictability is found in bad as well as good times, and MACE successfully navigates the debacle of 2022