239 research outputs found
Evolutionary optimization of sparsely connected and time-lagged neural networks for time series forecasting
Time Series Forecasting (TSF) is an important tool to support decision mak- ing (e.g., planning production resources). Artificial Neural Networks (ANN) are innate candidates for TSF due to advantages such as nonlinear learn- ing and noise tolerance. However, the search for the best model is a complex task that highly affects the forecasting performance. In this work, we propose two novel Evolutionary Artificial Neural Networks (EANN) approaches for TSF based on an Estimation Distribution Algorithm (EDA) search engine. The first new approach consist of Sparsely connected Evolutionary ANN (SEANN), which evolves more flexible ANN structures to perform multi-step ahead forecasts. The second one, consists of an automatic Time lag feature selection EANN (TEANN) approach that evolves not only ANN parameters (e.g., input and hidden nodes, training parameters) but also which set of time lags are fed into the forecasting model. Several experiments were held, using a set of six time series, from different real-world domains. Also, two error metrics (i.e., Mean Squared Error and Symmetric Mean Absolute Per- centage Error) were analyzed. The two EANN approaches were compared against a base EANN (with no ANN structure or time lag optimization) and four other methods (Autoregressive Integrated Moving Average method, Random Forest, Echo State Network and Support Vector Machine). Overall, the proposed SEANN and TEANN methods obtained the best forecasting results. Moreover, they favor simpler neural network models, thus requiring less computational effort when compared with the base EANN.The research reported here has been supported by the Spanish Ministry of Science and Innovation under project TRA2010-21371-C03-03 and FCT - Fundacao para a Ciencia e Tecnologia within the Project Scope PEst- OE/EEI/UI0319/2014. The authors want to thank specially Martin Stepnicka and Lenka Vavrickova for all their help. The authors also want to thank Ramon Sagarna for introducing the subject of EDA
Data-driven discovery of coordinates and governing equations
The discovery of governing equations from scientific data has the potential
to transform data-rich fields that lack well-characterized quantitative
descriptions. Advances in sparse regression are currently enabling the
tractable identification of both the structure and parameters of a nonlinear
dynamical system from data. The resulting models have the fewest terms
necessary to describe the dynamics, balancing model complexity with descriptive
ability, and thus promoting interpretability and generalizability. This
provides an algorithmic approach to Occam's razor for model discovery. However,
this approach fundamentally relies on an effective coordinate system in which
the dynamics have a simple representation. In this work, we design a custom
autoencoder to discover a coordinate transformation into a reduced space where
the dynamics may be sparsely represented. Thus, we simultaneously learn the
governing equations and the associated coordinate system. We demonstrate this
approach on several example high-dimensional dynamical systems with
low-dimensional behavior. The resulting modeling framework combines the
strengths of deep neural networks for flexible representation and sparse
identification of nonlinear dynamics (SINDy) for parsimonious models. It is the
first method of its kind to place the discovery of coordinates and models on an
equal footing.Comment: 25 pages, 6 figures; added acknowledgment
Machine Learning for Fluid Mechanics
The field of fluid mechanics is rapidly advancing, driven by unprecedented
volumes of data from field measurements, experiments and large-scale
simulations at multiple spatiotemporal scales. Machine learning offers a wealth
of techniques to extract information from data that could be translated into
knowledge about the underlying fluid mechanics. Moreover, machine learning
algorithms can augment domain knowledge and automate tasks related to flow
control and optimization. This article presents an overview of past history,
current developments, and emerging opportunities of machine learning for fluid
mechanics. It outlines fundamental machine learning methodologies and discusses
their uses for understanding, modeling, optimizing, and controlling fluid
flows. The strengths and limitations of these methods are addressed from the
perspective of scientific inquiry that considers data as an inherent part of
modeling, experimentation, and simulation. Machine learning provides a powerful
information processing framework that can enrich, and possibly even transform,
current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202
Multi-step time series prediction intervals using neuroevolution
Multi-step time series forecasting (TSF) is a crucial element to support tactical decisions (e.g., designing production or marketing plans several months in advance). While most TSF research addresses only single-point prediction, prediction intervals (PIs) are useful to reduce uncertainty related to important decision making variables. In this paper, we explore a large set of neural network methods for multi-step TSF and that directly optimize PIs. This includes multi-step adaptations of recently proposed PI methods, such as lower--upper bound estimation (LUBET), its ensemble extension (LUBEXT), a multi-objective evolutionary algorithm LUBE (MLUBET) and a two-phase learning multi-objective evolutionary algorithm (M2LUBET). We also explore two new ensemble variants for the evolutionary approaches based on two PI coverage--width split methods (radial slices and clustering), leading to the MLUBEXT, M2LUBEXT, MLUBEXT2 and M2LUBEXT2 methods. A robust comparison was held by considering the rolling window procedure, nine time series from several real-world domains and with different characteristics, two PI quality measures (coverage error and width) and the Wilcoxon statistic. Overall, the best results were achieved by the M2LUBET neuroevolution method, which requires a reasonable computational effort for time series with a few hundreds of observations.This article is a result of the project NORTE-01-
0247-FEDER-017497, supported by Norte Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020
Partnership Agreement, through the European Regional Development
Fund (ERDF). We would also like to thank the anonymous reviewers
for their helpful suggestionsinfo:eu-repo/semantics/publishedVersio
Review of automated time series forecasting pipelines
Time series forecasting is fundamental for various use cases in different
domains such as energy systems and economics. Creating a forecasting model for
a specific use case requires an iterative and complex design process. The
typical design process includes the five sections (1) data pre-processing, (2)
feature engineering, (3) hyperparameter optimization, (4) forecasting method
selection, and (5) forecast ensembling, which are commonly organized in a
pipeline structure. One promising approach to handle the ever-growing demand
for time series forecasts is automating this design process. The present paper,
thus, analyzes the existing literature on automated time series forecasting
pipelines to investigate how to automate the design process of forecasting
models. Thereby, we consider both Automated Machine Learning (AutoML) and
automated statistical forecasting methods in a single forecasting pipeline. For
this purpose, we firstly present and compare the proposed automation methods
for each pipeline section. Secondly, we analyze the automation methods
regarding their interaction, combination, and coverage of the five pipeline
sections. For both, we discuss the literature, identify problems, give
recommendations, and suggest future research. This review reveals that the
majority of papers only cover two or three of the five pipeline sections. We
conclude that future research has to holistically consider the automation of
the forecasting pipeline to enable the large-scale application of time series
forecasting
Assessing the Volume of Changes to Banking Assets and Liabilities Using Genetic Algorithms in Additional Funds Needed Model
This paper investigates Small-Medium Banksâ (SMBs) business plans in accordance with the structure of Additional Funds Needed (AFN) model. The Key Profitability Variables (KPVs) are the size and structure of deposits, loans, and their interest rates. This study employs a Genetic Algorithm (GA) problem with hard constraints, to point out the limits to changes in the structure of deposits and loans and the effects of those changes on the P&L of a banking institution. After examining 10,000 iterations with Evolver, an innovative optimization software that uses GA, OptQuest, and linear programming, the alternations, have been narrowed down to 3700 which satisfy both, a) constraints and b) maximization of profits. Having also the distributions, this paper concludes that it is a useful methodology that must be further exploited by applying risk weights, mainly for credit risk, to the structural components of the Balance Sheet, and to other competitive institutions.
Keywords: banking institutions, genetic algorithms, additional funds needed, operational researc
Review of automated time series forecasting pipelines
Time series forecasting is fundamental for various use cases in different domains such as energy systems and economics. Creating a forecasting model for a specific use case requires an iterative and complex design process. The typical design process includes the five sections (1) data pre-processing, (2) feature engineering, (3) hyperparameter optimization, (4) forecasting method selection, and (5) forecast ensembling, which are commonly organized in a pipeline structure. One promising approach to handle the ever-growing demand for time series forecasts is automating this design process. The present paper, thus, analyzes the existing literature on automated time series forecasting pipelines to investigate how to automate the design process of forecasting models. Thereby, we consider both Automated Machine Learning (AutoML) and automated statistical forecasting methods in a single forecasting pipeline. For this purpose, we firstly present and compare the proposed automation methods for each pipeline section. Secondly, we analyze the automation methods regarding their interaction, combination, and coverage of the five pipeline sections. For both, we discuss the literature, identify problems, give recommendations, and suggest future research. This review reveals that the majority of papers only cover two or three of the five pipeline sections. We conclude that future research has to holistically consider the automation of the forecasting pipeline to enable the large-scale application of time series forecasting
New Strategies for Initialization and Training of Radial Basis Function Neural Networks
In this paper we proposed two new strategies for initialization and training of Radial Basis Function (RBF) Neural Network. The first approach takes into consideration the "error" between the input vector p of the network and the x-axis, which are the centers of radial functions. The second approach takes into account the "error" between the input vector p and the network output. In order to check the performances of these strategies, we used Brazilian financial market data for the RBF networks training, specifically the adjusted prices of the 10 greater weighted shares in the Bovespa index at the time of data collection - from April 8th , 2009 to October 31th , 2014. The first approach presented a 52% of improvement in the mean squared error (MSE) compared to the standard RBF network, while the improvement for the second approach was 38%. The strategies proved to be consistent for the time series tested, in addition to having a low computational cost. It is proposed that these strategies be improved by testing them with the Levenberg-Marquardt algorithm
- âŠ