Self-organizing linear output map (SOLO): An artificial neural network suitable for hydrologic modeling and analysis

Artificial neural networks (ANNs) can be useful in the prediction of hydrologic variables, such as streamflow, particularly when the underlying processes have complex nonlinear interrelationships. However, conventional ANN structures suffer from network training issues that significantly limit their widespread application. This paper presents a multivariate ANN procedure entitled self-organizing linear output map (SOLO), whose structure has been designed for rapid, precise, and inexpensive estimation of network structure/parameters and system outputs. More important, SOLO provides features that facilitate insight into the underlying processes, thereby extending its usefulness beyond forecast applications as a tool for scientific investigations. These characteristics are demonstrated using a classic rainfall-runoff forecasting problem. Various aspects of model performance are evaluated in comparison with other commonly used modeling approaches, including multilayer feedforward ANNs, linear time series modeling, and conceptual rainfall-runoff modeling

The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.Comment: under revie

Collinsville solar thermal project: yield forecasting (draft report)

The final report has been published and is available here. Executive Summary 1        Introduction This report’s primary aim is to provide yield projections for the proposed Linear Fresnel Reflector (LFR) technology plant at Collinsville, Queensland, Australia.  However, the techniques developed in this report to overcome inadequate datasets at Collinsville to produce the yield projections are of interest to a wider audience because inadequate datasets for renewable energy projects are commonplace.  The subsequent report called ‘Energy economics and dispatch forecasting’ (Bell, Wild & Foster 2014a) uses the yield projections from this report to produce long-term wholesale market price and dispatch forecasts for the plant.  2        Literature review The literature review discusses the four drivers for yield for LFR technology: DNI (Direct Normal Irradiance) Temperature Humidity Pressure Collinsville lacks complete historical datasets of the four drivers to develop yield projects but its three nearby neighbours do possess complete datasets, so could act as proxies for Collinsville.  However, analysing the four drivers for Collinsville and its three nearby sites shows that there is considerable difference in their climates.  This difference makes them unsuitable to act as proxies for yield calculations.  Therefore, the review investigates modelling the four drivers for Collinsville. We introduce the term “effective” DNI to help clarify and ameliorate concerns over the dust and dew effects on terrestrial DNI measurement and LFR technology. We also introduce a modified TMY technique to overcome technology specific Typical Metrological Year (TMY).  We discuss the effect of climate change and the El Nino Southern Oscillation (ENSO) on yield and their implications for a TMY. 2.1     Research questions Research question arising from the literature review include: The overarching research question: Can modelling the weather with limited datasets produce greater yield predictive power than using the historically more complete datasets from nearby sites? This overarching question has a number of smaller supporting research questions: Is BoM’s DNI satellite dataset adequately adjusted for cloud cover at Collinsville? Given the dust and dew effects, is using raw satellite data sufficient to model yield? Does elevation between Collinsville and nearby sites affect yield? How does the ENSO affect yield? Given the 2007-2012 constraint, will the TMY process provide a “Typical” year over the ENSO cycle? How does climate change affect yield? A further research question arises in the methodology but is included here for completeness. What is the expected frequency of oversupply from the Linear Fresnel Novatec Solar Boiler? 3        Methodology In the methodology section, we discuss the data preparation and the model selection process for the four drivers of yield. 4        Results and analysis In the results section we present the four driver models selected and the process that was undertaken to arrive at the models. 5        Discussion We analyse the extent to which the research questions are informed by the results. 6        Conclusion In this report, we have identified the key research questions and established a methodology to address these questions.  The models for the four drivers have been established allowing the calculation of the yield projections for Collinsville

European exchange trading funds trading with locally weighted support vector regression

In this paper, two different Locally Weighted Support Vector Regression (wSVR) algorithms are generated and applied to the task of forecasting and trading five European Exchange Traded Funds. The trading application covers the recent European Monetary Union debt crisis. The performance of the proposed models is benchmarked against traditional Support Vector Regression (SVR) models. The Radial Basis Function, the Wavelet and the Mahalanobis kernel are explored and tested as SVR kernels. Finally, a novel statistical SVR input selection procedure is introduced based on a principal component analysis and the Hansen, Lunde, and Nason (2011) model confidence test. The results demonstrate the superiority of the wSVR models over the traditional SVRs and of the v-SVR over the ε-SVR algorithms. We note that the performance of all models varies and considerably deteriorates in the peak of the debt crisis. In terms of the kernels, our results do not confirm the belief that the Radial Basis Function is the optimum choice for financial series

On the Benefit of Using Time Series Features for Choosing a Forecasting Method

In research of time series forecasting, a lot of uncertainty is still related to the question of which forecasting method to use in which situation. One thing is obvious: There is no single method that performs best on all time series. This work examines whether features extracted from time series can be exploited for a better understanding of different behaviour of forecasting algorithms. An extensive pool of automatically computable features is identified, which is submitted to feature selection algorithms. Finally, a possible relationship between these features and the performance of forecasting and forecast combination methods for the particular series is investigated

ART Neural Networks: Distributed Coding and ARTMAP Applications

ART (Adaptive Resonance Theory) neural networks for fast, stable learning and prediction have been applied in a variety of areas. Applications include airplane design and manufacturing, automatic target recognition, financial forecasting, machine tool monitoring, digital circuit design, chemical analysis, and robot vision. Supervised ART architectures, called ARTMAP systems, feature internal control mechanisms that create stable recognition categories of optimal size by maximizing code compression while minimizing predictive error in an on-line setting. Special-purpose requirements of various application domains have led to a number of ARTMAP variants, including fuzzy ARTMAP, ART-EMAP, Gaussian ARTMAP, and distributed ARTMAP. ARTMAP has been used for a variety of applications, including computer-assisted medical diagnosis. Medical databases present many of the challenges found in general information management settings where speed, efficiency, ease of use, and accuracy are at a premium. A direct goal of improved computer-assisted medicine is to help deliver quality emergency care in situations that may be less than ideal. Working with these problems has stimulated a number of ART architecture developments, including ARTMAP-IC [1]. This paper describes a recent collaborative effort, using a new cardiac care database for system development, has brought together medical statisticians and clinicians at the New England Medical Center with researchers developing expert systems and neural networks, in order to create a hybrid method for medical diagnosis. The paper also considers new neural network architectures, including distributed ART {dART), a real-time model of parallel distributed pattern learning that permits fast as well as slow adaptation, without catastrophic forgetting. Local synaptic computations in the dART model quantitatively match the paradoxical phenomenon of Markram-Tsodyks [2] redistribution of synaptic efficacy, as a consequence of global system hypotheses.Office of Naval Research (N00014-95-1-0409, N00014-95-1-0657

Incremental construction of LSTM recurrent neural network

Long Short--Term Memory (LSTM) is a recurrent neural network that uses structures called memory blocks to allow the net remember significant events distant in the past input sequence in order to solve long time lag tasks, where other RNN approaches fail. Throughout this work we have performed experiments using LSTM networks extended with growing abilities, which we call GLSTM. Four methods of training growing LSTM has been compared. These methods include cascade and fully connected hidden layers as well as two different levels of freezing previous weights in the cascade case. GLSTM has been applied to a forecasting problem in a biomedical domain, where the input/output behavior of five controllers of the Central Nervous System control has to be modelled. We have compared growing LSTM results against other neural networks approaches, and our work applying conventional LSTM to the task at hand.Postprint (published version