DEK-Forecaster: A Novel Deep Learning Model Integrated with EMD-KNN for Traffic Prediction

Internet traffic volume estimation has a significant impact on the business policies of the ISP (Internet Service Provider) industry and business successions. Forecasting the internet traffic demand helps to shed light on the future traffic trend, which is often helpful for ISPs decision-making in network planning activities and investments. Besides, the capability to understand future trend contributes to managing regular and long-term operations. This study aims to predict the network traffic volume demand using deep sequence methods that incorporate Empirical Mode Decomposition (EMD) based noise reduction, Empirical rule based outlier detection, and $K$-Nearest Neighbour (KNN) based outlier mitigation. In contrast to the former studies, the proposed model does not rely on a particular EMD decomposed component called Intrinsic Mode Function (IMF) for signal denoising. In our proposed traffic prediction model, we used an average of all IMFs components for signal denoising. Moreover, the abnormal data points are replaced by $K$ nearest data points average, and the value for $K$ has been optimized based on the KNN regressor prediction error measured in Root Mean Squared Error (RMSE). Finally, we selected the best time-lagged feature subset for our prediction model based on AutoRegressive Integrated Moving Average (ARIMA) and Akaike Information Criterion (AIC) value. Our experiments are conducted on real-world internet traffic datasets from industry, and the proposed method is compared with various traditional deep sequence baseline models. Our results show that the proposed EMD-KNN integrated prediction models outperform comparative models.Comment: 13 pages, 9 figure

Exchange Rate Forecasting Using Entropy Optimized Multivariate Wavelet Denoising Model

Exchange rate is one of the key variables in the international economics and international trade. Its movement constitutes one of the most important dynamic systems, characterized by nonlinear behaviors. It becomes more volatile and sensitive to increasingly diversified influencing factors with higher level of deregulation and global integration worldwide. Facing the increasingly diversified and more integrated market environment, the forecasting model in the exchange markets needs to address the individual and interdependent heterogeneity. In this paper, we propose the heterogeneous market hypothesis- (HMH-) based exchange rate modeling methodology to model the micromarket structure. Then we further propose the entropy optimized wavelet-based forecasting algorithm under the proposed methodology to forecast the exchange rate movement. The multivariate wavelet denoising algorithm is used to separate and extract the underlying data components with distinct features, which are modeled with multivariate time series models of different specifications and parameters. The maximum entropy is introduced to select the best basis and model parameters to construct the most effective forecasting algorithm. Empirical studies in both Chinese and European markets have been conducted to confirm the significant performance improvement when the proposed model is tested against the benchmark models

Urban Safety: An Image-Processing and Deep-Learning-Based Intelligent Traffic Management and Control System

With the rapid growth and development of cities, Intelligent Traffic Management and Control (ITMC) is becoming a fundamental component to address the challenges of modern urban traffic management, where a wide range of daily problems need to be addressed in a prompt and expedited manner. Issues such as unpredictable traffic dynamics, resource constraints, and abnormal events pose difficulties to city managers. ITMC aims to increase the efficiency of traffic management by minimizing the odds of traffic problems, by providing real-time traffic state forecasts to better schedule the intersection signal controls. Reliable implementations of ITMC improve the safety of inhabitants and the quality of life, leading to economic growth. In recent years, researchers have proposed different solutions to address specific problems concerning traffic management, ranging from image-processing and deep-learning techniques to forecasting the traffic state and deriving policies to control intersection signals. This review article studies the primary public datasets helpful in developing models to address the identified problems, complemented with a deep analysis of the works related to traffic state forecast and intersection-signal-control models. Our analysis found that deep-learning-based approaches for short-term traffic state forecast and multi-intersection signal control showed reasonable results, but lacked robustness for unusual scenarios, particularly during oversaturated situations, which can be resolved by explicitly addressing these cases, potentially leading to significant improvements of the systems overall. However, there is arguably a long path until these models can be used safely and effectively in real-world scenarios

Forecasting Crude Oil Price with Multiscale Denoising Ensemble Model

Crude oil price becomes more volatile and sensitive to increasingly diversified influencing factors with higher level of deregulations worldwide. Current methodologies are being challenged as they have been constrained by traditional approaches assuming homogeneous time horizons and investment strategies. Approximations they provided over the long term time horizon no longer satisfy the accuracy requirement at shorter term and more microlevels. This paper proposes a novel crude oil price forecasting model based on the wavelet denoising ARMA models ensemble by least square support vector regression with the reduced forecasting matrix dimensions by independent component analysis. The proposed methodology combines the multi resolution analysis and nonlinear ensemble framework. The wavelet denoising based algorithm is introduced to separate and extract the underlying data components with distinct features, corresponding to investors with different investment scales, which are modeled with time series models of different specifications and parameters. Then least square support vector regression is introduced to nonlinearly ensemble results based on different wavelet families to further reduce the estimation biases and improve the forecasting generalizability. Empirical studies show the significant performance improvement when the proposed model is tested against the bench-mark models

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Forecasting the All-Weather Short-Term Metro Passenger Flow Based on Seasonal and Nonlinear LSSVM

Accurate metro ridership prediction can guide passengers in efficiently selecting their departure time and simultaneously help traffic operators develop a passenger organization strategy. However, short-term passenger flow prediction needs to consider many factors, and the results of the existing models for short-term subway passenger flow forecasting are often unsatisfactory. Along this line, we propose a parallel architecture, called the seasonal and nonlinear least squares support vector machine (SN-LSSVM), to extract the periodicity and nonlinearity characteristics of passenger flow. Various forecasting models, including auto-regressive integrated moving average, long short-term memory network, and support vector machine, are employed for evaluating the performance of the proposed architecture. Moreover, we first applied the method to the Tiyu Xilu station which is the most crowded station in the Guangzhou metro. The results indicate that the proposed model can effectively make all-weather and year-round passenger flow predictions, thus contributing to the management of the station

Fuzzy Prediction for Traffic Flow Based on Delta Test

This paper presents a novel approach to one-step-forward prediction of traffic flow based on fuzzy reasoning. The successful construction of a competent fuzzy inference system of Sugeno type largely relies on proper choice of input dimension and accurate estimation of structure parameters and rules. The first issue is addressed with a proposed method, based on δ-test, which can simultaneously determine input dimension and reduce noise level. In response to the second issue, two clustering techniques, based on nearest-neighbor clustering and Gaussian mixture models, are successively employed to determine the antecedent parameters and rules, and the estimation for the consequent parameters is achieved by the least square estimation technique. A number of experiments have been performed on the one-week data of traffic flow to evaluate the proposed approach in terms of denosing, prediction performances, overfitting, and so forth. The experimental results have demonstrated that the proposed prediction approach is effective in removing noise and constructing a competent and compact fuzzy inference system without significant overfitting

Short-Term Forecasting of Passenger Demand under On-Demand Ride Services: A Spatio-Temporal Deep Learning Approach

Short-term passenger demand forecasting is of great importance to the on-demand ride service platform, which can incentivize vacant cars moving from over-supply regions to over-demand regions. The spatial dependences, temporal dependences, and exogenous dependences need to be considered simultaneously, however, which makes short-term passenger demand forecasting challenging. We propose a novel deep learning (DL) approach, named the fusion convolutional long short-term memory network (FCL-Net), to address these three dependences within one end-to-end learning architecture. The model is stacked and fused by multiple convolutional long short-term memory (LSTM) layers, standard LSTM layers, and convolutional layers. The fusion of convolutional techniques and the LSTM network enables the proposed DL approach to better capture the spatio-temporal characteristics and correlations of explanatory variables. A tailored spatially aggregated random forest is employed to rank the importance of the explanatory variables. The ranking is then used for feature selection. The proposed DL approach is applied to the short-term forecasting of passenger demand under an on-demand ride service platform in Hangzhou, China. Experimental results, validated on real-world data provided by DiDi Chuxing, show that the FCL-Net achieves better predictive performance than traditional approaches including both classical time-series prediction models and neural network based algorithms (e.g., artificial neural network and LSTM). This paper is one of the first DL studies to forecast the short-term passenger demand of an on-demand ride service platform by examining the spatio-temporal correlations.Comment: 39 pages, 10 figure