A Nonlinear Technical Indicator Selection Approach for Stock Markets. Application to the Chinese Stock Market

In this paper we present a combinatorial nonlinear technical indicator approach for the identification of appropriate combinations of stock technical indicators as inputs in non-linear models. This approach is illustrated with the example of Chinese stock indexes and 35 different stock technical indicators using neural networks as the chosen non-linear method. Stock market technical indicators can generate contradictory signals regarding the future performance of the stock analyzed. Furthermore, some non-linear methods, such as neural networks, can have poor generalization power when dealing with problems of high dimensionality due to the issue of local minima. Therefore, non-linear approaches that can identify appropriate combinations of input variables are of clear importance. It will be shown that the proposed approach, when using neural networks as classifiers, generates error rates lower than those using directly neural networks without dimensionality reduction. It will also be shown that merely increasing the number of neurons does not increase the accuracy. The approach proposed in this article is illustrated with an application to the stock market using neural networks but it could be applied to other fields and it can also be used with other non-linear techniques such as for instance support vector machines.Ministerio de Ciencia e Innovación PID2019-106212RB-C41

Gaussian process models for SCADA data based wind turbine performance/condition monitoring

Wind energy has seen remarkable growth in the past decade, and installed wind turbine capacity is increasing significantly every year around the globe. The presence of an excellent offshore wind resource and the need to reduce carbon emissions from electricity generation are driving policy to increase offshore wind generation capacity in UK waters. Logistic and transport issues make offshore maintenance costlier than onshore and availability correspondingly lower, and as a result, there is a growing interest in wind turbine condition monitoring allowing condition based, rather than corrective or scheduled, maintenance.;Offshore wind turbine manufacturers are constantly increasing the rated size the turbines, and also their hub height in order to access higher wind speeds with lower turbulence. However, such scaling up leads to significant increments in terms of materials for both tower structure and foundations, and also costs required for transportation, installation, and maintenance. Wind turbines are costly affairs that comprise several complex systems connected altogether (e.g., hub, drive shaft, gearbox, generator, yaw system, electric drive and so on).;The unexpected failure of these components can cause significant machine unavailability and/or damage to other components. This ultimately increases the operation and maintenance (O&M) cost and subsequently cost of energy (COE). Therefore, identifying faults at an early stage before catastrophic damage occurs is the primary objective associated with wind turbine condition monitoring.;Existing wind turbine condition monitoring strategies, for example, vibration signal analysis and oil debris detection, require costly sensors. The additional costs can be significant depending upon the number of wind turbines typically deployed in offshore wind farms and also, costly expertise is generally required to interpret the results. By contrast, Supervisory Control and Data Acquisition (SCADA) data analysis based condition monitoring could underpin condition based maintenance with little or no additional cost to the wind farm operator.;A Gaussian process (GP) is a stochastic, nonlinear and nonparametric model whose distribution function is the joint distribution of a collection of random variables; it is widely suitable for classification and regression problems. GP is a machine learning algorithm that uses a measure of similarity between subsequent data points (via covariance functions) to fit and or estimate the future value from a training dataset. GP models have been applied to numerous multivariate and multi-task problems including spatial and spatiotemporal contexts.;Furthermore, GP models have been applied to electricity price and residential probabilistic load forecasting, solar power forecasting. However, the application of GPs to wind turbine condition monitoring has to date been limited and not much explored.;This thesis focuses on GP based wind turbine condition monitoring that utilises data from SCADA systems exclusively. The selection of the covariance function greatly influences GP model accuracy. A comparative analysis of different covariance functions for GP models is presented with an in-depth analysis of popularly used stationary covariance functions. Based on this analysis, a suitable covariance function is selected for constructing a GP model-based fault detection algorithm for wind turbine condition monitoring.;By comparing incoming operational SCADA data, effective component condition indicators can be derived where the reference model is based on SCADA data from a healthy turbine constructed and compared against incoming data from a faulty turbine. In this thesis, a GP algorithm is constructed with suitable covariance function to detect incipient turbine operational faults or failures before they result in catastrophic damage so that preventative maintenance can be scheduled in a timely manner.;In order to judge GP model effectiveness, two other methods, based on binning, have been tested and compared with the GP based algorithm. This thesis also considers a range of critical turbine parameters and their impact on the GP fault detection algorithm.;Power is well known to be influenced by air density, and this is reflected in the IEC Standard air density correction procedure. Hence, the proper selection of an air density correction approach can improve the power curve model. This thesis addresses this, explores the different types of air density correction approach, and suggests the best way to incorporate these in the GP models to improve accuracy and reduce uncertainty.;Finally, a SCADA data based fault detection algorithm is constructed to detect failures caused by the yaw misalignment. Two fault detection algorithms based on IEC binning methods (widely used within the wind industry) are developed to assess the performance of the GP based fault detection algorithm in terms of their capability to detect in advance (and by how much) signs of failure, and also their false positive rate by making use of extensive SCADA data and turbine fault and repair logs.;GP models are robust in identifying early anomalies/failures that cause the wind turbine to underperform. This early detection is helpful in preventing machines to reach the catastrophic stage and allow enough time to undertake scheduled maintenance, which ultimately reduces the O&M, cost and maximises the power performance of wind turbines. Overall, results demonstrate the effectiveness of the GP algorithm in improving the performance of wind turbines through condition monitoring.Wind energy has seen remarkable growth in the past decade, and installed wind turbine capacity is increasing significantly every year around the globe. The presence of an excellent offshore wind resource and the need to reduce carbon emissions from electricity generation are driving policy to increase offshore wind generation capacity in UK waters. Logistic and transport issues make offshore maintenance costlier than onshore and availability correspondingly lower, and as a result, there is a growing interest in wind turbine condition monitoring allowing condition based, rather than corrective or scheduled, maintenance.;Offshore wind turbine manufacturers are constantly increasing the rated size the turbines, and also their hub height in order to access higher wind speeds with lower turbulence. However, such scaling up leads to significant increments in terms of materials for both tower structure and foundations, and also costs required for transportation, installation, and maintenance. Wind turbines are costly affairs that comprise several complex systems connected altogether (e.g., hub, drive shaft, gearbox, generator, yaw system, electric drive and so on).;The unexpected failure of these components can cause significant machine unavailability and/or damage to other components. This ultimately increases the operation and maintenance (O&M) cost and subsequently cost of energy (COE). Therefore, identifying faults at an early stage before catastrophic damage occurs is the primary objective associated with wind turbine condition monitoring.;Existing wind turbine condition monitoring strategies, for example, vibration signal analysis and oil debris detection, require costly sensors. The additional costs can be significant depending upon the number of wind turbines typically deployed in offshore wind farms and also, costly expertise is generally required to interpret the results. By contrast, Supervisory Control and Data Acquisition (SCADA) data analysis based condition monitoring could underpin condition based maintenance with little or no additional cost to the wind farm operator.;A Gaussian process (GP) is a stochastic, nonlinear and nonparametric model whose distribution function is the joint distribution of a collection of random variables; it is widely suitable for classification and regression problems. GP is a machine learning algorithm that uses a measure of similarity between subsequent data points (via covariance functions) to fit and or estimate the future value from a training dataset. GP models have been applied to numerous multivariate and multi-task problems including spatial and spatiotemporal contexts.;Furthermore, GP models have been applied to electricity price and residential probabilistic load forecasting, solar power forecasting. However, the application of GPs to wind turbine condition monitoring has to date been limited and not much explored.;This thesis focuses on GP based wind turbine condition monitoring that utilises data from SCADA systems exclusively. The selection of the covariance function greatly influences GP model accuracy. A comparative analysis of different covariance functions for GP models is presented with an in-depth analysis of popularly used stationary covariance functions. Based on this analysis, a suitable covariance function is selected for constructing a GP model-based fault detection algorithm for wind turbine condition monitoring.;By comparing incoming operational SCADA data, effective component condition indicators can be derived where the reference model is based on SCADA data from a healthy turbine constructed and compared against incoming data from a faulty turbine. In this thesis, a GP algorithm is constructed with suitable covariance function to detect incipient turbine operational faults or failures before they result in catastrophic damage so that preventative maintenance can be scheduled in a timely manner.;In order to judge GP model effectiveness, two other methods, based on binning, have been tested and compared with the GP based algorithm. This thesis also considers a range of critical turbine parameters and their impact on the GP fault detection algorithm.;Power is well known to be influenced by air density, and this is reflected in the IEC Standard air density correction procedure. Hence, the proper selection of an air density correction approach can improve the power curve model. This thesis addresses this, explores the different types of air density correction approach, and suggests the best way to incorporate these in the GP models to improve accuracy and reduce uncertainty.;Finally, a SCADA data based fault detection algorithm is constructed to detect failures caused by the yaw misalignment. Two fault detection algorithms based on IEC binning methods (widely used within the wind industry) are developed to assess the performance of the GP based fault detection algorithm in terms of their capability to detect in advance (and by how much) signs of failure, and also their false positive rate by making use of extensive SCADA data and turbine fault and repair logs.;GP models are robust in identifying early anomalies/failures that cause the wind turbine to underperform. This early detection is helpful in preventing machines to reach the catastrophic stage and allow enough time to undertake scheduled maintenance, which ultimately reduces the O&M, cost and maximises the power performance of wind turbines. Overall, results demonstrate the effectiveness of the GP algorithm in improving the performance of wind turbines through condition monitoring

On the Problem of Local Minima in Recurrent Neural Networks

Many researchers have recently focused their efforts on devising efficient algorithms, mainly based on optimization schemes, for learning the weights of recurrent neural networks. Like for feedforward networks however, these learning algorithms may get stuck in local minima during gradient descent, thus discovering sub-optimal solutions. This paper analyzes the problem of optimal learning in recurrent networks by proposing some sufficient conditions which guarantee local minima free error surfaces. An example is given which also show the constructive role of the proposed theory in designing networks suitable for solving a given task. Moreover, a formal relationship between recurrent and static feedforward networks is established such that the examples of local minima for feedforward networks already known in the literature can be associated with analogous ones in recurrent networks. Index Terms- Backpropagation through time, feedforward networks, learning environment, linearly separabl..

On the problem of local minima in recurrent neural networks

Many researchers have recently focused their efforts on devising efficient algorithms, mainly based on optimization schemes, for learning the weights of recurrent neural networks. As in the case of feedforward networks, however, these learning algorithms may get stuck in local minima during gradient descent, thus discovering sub-optimal solutions. This paper analyses the problem of optimal learning in recurrent networks by proposing conditions that guarantee local minima free error surfaces. An example is given that also shows the constructive role of the proposed theory in designing networks suitable for solving a given task. Moreover, a formal relationship between recurrent and static feedforward networks is established such that the examples of local minima for feedforward networks already known in the literature can be associated with analogous ones in recurrent networks