Forecast evaluation for data scientists:common pitfalls and best practices

Machine Learning (ML) and Deep Learning (DL) methods are increasingly replacing traditional methods in many domains involved with important decision making activities. DL techniques tailor-made for specific tasks such as image recognition, signal processing, or speech analysis are being introduced at a fast pace with many improvements. However, for the domain of forecasting, the current state in the ML community is perhaps where other domains such as Natural Language Processing and Computer Vision were at several years ago. The field of forecasting has mainly been fostered by statisticians/econometricians; consequently the related concepts are not the mainstream knowledge among general ML practitioners. The different non-stationarities associated with time series challenge the data-driven ML models. Nevertheless, recent trends in the domain have shown that with the availability of massive amounts of time series, ML techniques are quite competent in forecasting, when related pitfalls are properly handled. Therefore, in this work we provide a tutorial-like compilation of the details of one of the most important steps in the overall forecasting process, namely the evaluation. This way, we intend to impart the information of forecast evaluation to fit the context of ML, as means of bridging the knowledge gap between traditional methods of forecasting and state-of-the-art ML techniques. We elaborate on the different problematic characteristics of time series such as non-normalities and non-stationarities and how they are associated with common pitfalls in forecast evaluation. Best practices in forecast evaluation are outlined with respect to the different steps such as data partitioning, error calculation, statistical testing, and others. Further guidelines are also provided along selecting valid and suitable error measures depending on the specific characteristics of the dataset at hand

Deep Learning Approaches for Long-Term Global Horizontal Irradiance Forecasting for Microgrids Planning

Providing sustainable energy to rural communities is considered in Sustainable Development Goal 7. Off-grid renewable energy systems arise as an affordable solution due to their portability and the availability of renewable sources for rural communities. In this work, to deal with the uncertainties of solar resources, we employ two deep learning models (feed forward and recurrent neural networks) to predict renewable sources in a long-term horizon. To this aim, the approach presented takes into account the necessity of a high enough resolution in the forecasting output. As a case study, we employ open source data for a location in Michoacan, Mexico as well as open source programming frameworks to ensure the replicability of the numerical experiments. The results show that our prediction model performs excellently with respect to the baseline methods (ARIMA, exponential smoothing, and seasonal naive) in terms of the evaluation metrics MASE (18.5% of reduction with respect to seasonal naive), RMSE (24.7%), WAPE (13.1%), MAE (12.9%), and APB (8.9%)

Evaluation of feature-based object identification for augmented reality applications on mobile devices

With the technological advancement in mobile computing industry the field of electronic commerce has gone through a paradigm shift. Today's customers are more inclined towards using sophisticated mobile assistants, to help them in shopping, indoor navigation etc. rather than struggling on their own. In the literature it can be found that several attempts have been taken to address this cause. Majority of those attempts are based on marker based object identification. But current scale and the trends of the electronic commerce industry demand shopping assistants who are following markerless object identification methods due to limitations like the limited number of objects that can be identified. In order to support more usable electronic commerce applications, this paper focuses on the feasibility of deploying common and readily available feature tracking algorithms which can provide the object identification capability without having fiducial markers.</p