Model structure selection using an integrated forward orthogonal search algorithm assisted by squared correlation and mutual information

Model structure selection plays a key role in non-linear system identification. The first step in non-linear system identification is to determine which model terms should be included in the model. Once significant model terms have been determined, a model selection criterion can then be applied to select a suitable model subset. The well known Orthogonal Least Squares (OLS) type algorithms are one of the most efficient and commonly used techniques for model structure selection. However, it has been observed that the OLS type algorithms may occasionally select incorrect model terms or yield a redundant model subset in the presence of particular noise structures or input signals. A very efficient Integrated Forward Orthogonal Search (IFOS) algorithm, which is assisted by the squared correlation and mutual information, and which incorporates a Generalised Cross-Validation (GCV) criterion and hypothesis tests, is introduced to overcome these limitations in model structure selection

Model structure selection using an integrated forward orthogonal search algorithm interfered with squared correlation and mutual information

Model structure selection plays a key role in nonlinear system identification. The first step in nonlinear system identification is to determine which model terms should be included in the model. Once significant model terms have been determined, a model selection criterion can then be applied to select a suitable model subset. The well known orthogonal least squares type algorithms are one of the most efficient and commonly used techniques for model structure selection. However, it has been observed that the orthogonal least squares type algorithms may occasionally select incorrect model terms or yield a redundant model subset in the presence of particular noise structures or input signals. A very efficient integrated forward orthogonal searching (IFOS) algorithm, which is interfered with squared correlation and mutual information, and which incorporates a general cross-validation (GCV) criterion and hypothesis tests, is introduced to overcome these limitations in model structure selection

Training Echo State Networks with Regularization through Dimensionality Reduction

In this paper we introduce a new framework to train an Echo State Network to predict real valued time-series. The method consists in projecting the output of the internal layer of the network on a space with lower dimensionality, before training the output layer to learn the target task. Notably, we enforce a regularization constraint that leads to better generalization capabilities. We evaluate the performances of our approach on several benchmark tests, using different techniques to train the readout of the network, achieving superior predictive performance when using the proposed framework. Finally, we provide an insight on the effectiveness of the implemented mechanics through a visualization of the trajectory in the phase space and relying on the methodologies of nonlinear time-series analysis. By applying our method on well known chaotic systems, we provide evidence that the lower dimensional embedding retains the dynamical properties of the underlying system better than the full-dimensional internal states of the network

A proposal for a coordinated effort for the determination of brainwide neuroanatomical connectivity in model organisms at a mesoscopic scale

In this era of complete genomes, our knowledge of neuroanatomical circuitry remains surprisingly sparse. Such knowledge is however critical both for basic and clinical research into brain function. Here we advocate for a concerted effort to fill this gap, through systematic, experimental mapping of neural circuits at a mesoscopic scale of resolution suitable for comprehensive, brain-wide coverage, using injections of tracers or viral vectors. We detail the scientific and medical rationale and briefly review existing knowledge and experimental techniques. We define a set of desiderata, including brain-wide coverage; validated and extensible experimental techniques suitable for standardization and automation; centralized, open access data repository; compatibility with existing resources, and tractability with current informatics technology. We discuss a hypothetical but tractable plan for mouse, additional efforts for the macaque, and technique development for human. We estimate that the mouse connectivity project could be completed within five years with a comparatively modest budget.Comment: 41 page

An Exploration of Neural Network Modelling Options for the Upper River Ping, Thailand

This thesis reports results from a systematic experimental approach to evaluating aspects of the neural network modelling process to forecast river stage for a large, 23,600 km2 catchment in northern Thailand. The research is prompted by the absence of evidenced recommendations as to which of the array of input processes, validations and modelling procedures might be selected by a neural network forecaster. The flood issue for forecasters at Chiang Mai derives from the monsoon rainfall, which leads to serious out-of-bank flooding two to four times a year. Data for stage and rainfall is limited as the instrumentation is sparse and the historical flood record is limited in length. Neural network forecasting models are potentially very powerful forecasters where the data are limited. The challenge of this catchment is to provide adequate forecasts from data for relatively few storm events using three stage gauges and one rain gauge. Previous studies have reported forecasts with lead times of up to 18 hours. Thus, one research driver is to extend this lead time to give more warning. Eight input determination methods were systematically evaluated through thousands of model runs. The most successful method was found to be correlation and stepwise regression although the pattern was not consistent across all model runs. Cloud radar imagery was available for a few storm events. Rainfall data from a network was not available so it was decided to explore the value of the raw cloud reflectivity data as a catchment-wide surrogate for rainfall, to enhance the data record and potentially improve the forecast. The limited number of events makes drawing conclusions difficult, but for one event the forecast lead time was extended to 24-30 hours. The modelling also indicates that for this catchment where the monsoon may come from the south west or the north east, the direction of storm travel is important, indicating that developing two neural network models may be more appropriate. Internal model training and parameterisation tests suggest that future models should use Bayesian Regularization, and average across 50 runs. The number of hidden nodes should be less than the number input variables although for more complex problems, this was not necessarily the case. Ranges of normalisation made little difference. However, the minimum and maximum values used for normalisation appear to more important. The strength of the conclusions to be drawn from this research was recognised from the start as being limited by the data, but the results suggest that neural networks are both helpful modelling processes and can provide valuable forecasts in catchments with extreme rainfall and limited hydrological data. The systematic investigation of the alternative input determination methods, algorithms and internal parameters has enabled guidance to be given on appropriate model structures

Air Quality Prediction in Smart Cities Using Machine Learning Technologies Based on Sensor Data: A Review

The influence of machine learning technologies is rapidly increasing and penetrating almost in every field, and air pollution prediction is not being excluded from those fields. This paper covers the revision of the studies related to air pollution prediction using machine learning algorithms based on sensor data in the context of smart cities. Using the most popular databases and executing the corresponding filtration, the most relevant papers were selected. After thorough reviewing those papers, the main features were extracted, which served as a base to link and compare them to each other. As a result, we can conclude that: (1) instead of using simple machine learning techniques, currently, the authors apply advanced and sophisticated techniques, (2) China was the leading country in terms of a case study, (3) Particulate matter with diameter equal to 2.5 micrometers was the main prediction target, (4) in 41% of the publications the authors carried out the prediction for the next day, (5) 66% of the studies used data had an hourly rate, (6) 49% of the papers used open data and since 2016 it had a tendency to increase, and (7) for efficient air quality prediction it is important to consider the external factors such as weather conditions, spatial characteristics, and temporal features