Two Hidden Layers are Usually Better than One

This study investigates whether feedforward neural networks with two hidden layers generalise better than those with one. In contrast to the existing literature, a method is proposed which allows these networks to be compared empirically on a hidden-node-by-hidden-node basis. This is applied to ten public domain function approximation datasets. Networks with two hidden layers were found to be better generalisers in nine of the ten cases, although the actual degree of improvement is case dependent. The proposed method can be used to rapidly determine whether it is worth considering two hidden layers for a given problem

Short-term wind power forecasting using artificial neural networks-based ensemble model

Short-term wind power forecasting is crucial for the efficient operation of power systems with high wind power penetration. Many forecasting approaches have been developed in the past to forecast short-term wind power. In recent years, artificial neural network-based approaches (ANNs) have been one of the most effective and popular approaches for short-term wind power forecasting because of the availability of large amounts of historical data and strong computational power. Although ANNs usually perform well for short-term wind power forecasting, further improvement can be obtained by selecting suitable input features, model parameters, and using forecasting techniques like spatial correlation and ensemble for ANNs. In this research, the effect of input features, model parameters, spatial correlation and ensemble techniques on short-term wind power forecasting performance of the ANNs models was evaluated. Pearson correlation coefficients between wind speed and other meteorological variables, together with a basic ANN model, were used to determine the impact of different input features on the forecasting performance of the ANNs. The effect of training sample resolution and training sample size on the forecasting performance was also investigated. To separately investigate the impact of the number of hidden layers and the number of hidden neurons on short-term wind power forecasting and to keep a single variable for each experiment, the same number of hidden neurons was used in each hidden layer. The ANNs with a total of 20 hidden neurons are shown to be sufficient for the nonlinear multivariate wind power forecasting problems faced in this dissertation. The ANNs with two hidden layers performed better than the one with a single hidden layer because additional hidden layer adds nonlinearity to the model. However, the ANNs with more than two hidden layers have the same or worse forecasting performance than the one with two hidden layers. ANNs with too many hidden layers and hidden neurons can overfit the training data. Spatial correlation technique was used to include meteorological variables from highly correlated neighbouring stations as input features to provide more surrounding information to the ANNs. The advantages of input features, model parameters, and spatial correlation and ensemble techniques were combined to form an ANN-based ensemble model to further enhance the forecasting performance from an individual ANN model. The simulation results show that all the available meteorological variables have different levels of impact on forecasting performance. Wind speed has the most significant impact on both short-term wind speed and wind power forecasting, whereas air temperature, barometric pressure, and air density have the smallest effects. The ANNs perform better with a higher data resolution and a significantly larger training sample size. However, one requires more computational power and a longer training time to train the model with a higher data resolution and a larger training sample size. Using the meteorological variables from highly related neighbouring stations do significantly improve the forecasting accuracy of target stations. It is shown that an ANNs-based ensemble model can further enhance the forecasting performance of an individual ANN by obtaining a large amount of surrounding meteorological information in parallel without encountering the overfitting issue faced by a single ANN model

Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction

Click-Through Rate prediction is an important task in recommender systems, which aims to estimate the probability of a user to click on a given item. Recently, many deep models have been proposed to learn low-order and high-order feature interactions from original features. However, since useful interactions are always sparse, it is difficult for DNN to learn them effectively under a large number of parameters. In real scenarios, artificial features are able to improve the performance of deep models (such as Wide & Deep Learning), but feature engineering is expensive and requires domain knowledge, making it impractical in different scenarios. Therefore, it is necessary to augment feature space automatically. In this paper, We propose a novel Feature Generation by Convolutional Neural Network (FGCNN) model with two components: Feature Generation and Deep Classifier. Feature Generation leverages the strength of CNN to generate local patterns and recombine them to generate new features. Deep Classifier adopts the structure of IPNN to learn interactions from the augmented feature space. Experimental results on three large-scale datasets show that FGCNN significantly outperforms nine state-of-the-art models. Moreover, when applying some state-of-the-art models as Deep Classifier, better performance is always achieved, showing the great compatibility of our FGCNN model. This work explores a novel direction for CTR predictions: it is quite useful to reduce the learning difficulties of DNN by automatically identifying important features

Exploring the trend of New Zealand housing prices to support sustainable development

The New Zealand housing sector is experiencing rapid growth that has a significant impact on society, the economy, and the environment. In line with the growth, the housing market for both residential and business purposes has been booming, as have house prices. To sustain the housing development, it is critical to accurately monitor and predict housing prices so as to support the decision-making process in the housing sector. This study is devoted to applying a mathematical method to predict housing prices. The forecasting performance of two types of models: autoregressive integrated moving average (ARIMA) and multiple linear regression (MLR) analysis are compared. The ARIMA and regression models are developed based on a training-validation sample method. The results show that the ARIMA model generally performs better than the regression model. However, the regression model explores, to some extent, the significant correlations between house prices in New Zealand and the macro-economic conditions