Prediction of Daily Mean PM10 Concentrations Using Random Forest, CART Ensemble and Bagging Stacked by MARS

A novel framework for stacked regression based on machine learning was developed to predict the daily average concentrations of particulate matter (PM10), one of Bulgaria’s primary health concerns. The measurements of nine meteorological parameters were introduced as independent variables. The goal was to carefully study a limited number of initial predictors and extract stochastic information from them to build an extended set of data that allowed the creation of highly efficient predictive models. Four base models using random forest, CART ensemble and bagging, and their rotation variants, were built and evaluated. The heterogeneity of these base models was achieved by introducing five types of diversities, including a new simplified selective ensemble algorithm. The predictions from the four base models were then used as predictors in multivariate adaptive regression splines (MARS) models. All models were statistically tested using out-of-bag or with 5-fold and 10-fold cross-validation. In addition, a variable importance analysis was conducted. The proposed framework was used for short-term forecasting of out-of-sample data for seven days. It was shown that the stacked models outperformed all single base models. An index of agreement IA = 0.986 and a coefficient of determination of about 95% were achieved

Prediction of Daily Mean PM₁₀ Concentrations Using Random Forest, CART Ensemble and Bagging Stacked by MARS

A novel framework for stacked regression based on machine learning was developed to predict the daily average concentrations of particulate matter (PM10), one of Bulgaria’s primary health concerns. The measurements of nine meteorological parameters were introduced as independent variables. The goal was to carefully study a limited number of initial predictors and extract stochastic information from them to build an extended set of data that allowed the creation of highly efficient predictive models. Four base models using random forest, CART ensemble and bagging, and their rotation variants, were built and evaluated. The heterogeneity of these base models was achieved by introducing five types of diversities, including a new simplified selective ensemble algorithm. The predictions from the four base models were then used as predictors in multivariate adaptive regression splines (MARS) models. All models were statistically tested using out-of-bag or with 5-fold and 10-fold cross-validation. In addition, a variable importance analysis was conducted. The proposed framework was used for short-term forecasting of out-of-sample data for seven days. It was shown that the stacked models outperformed all single base models. An index of agreement IA = 0.986 and a coefficient of determination of about 95% were achieved

Multi-Step Ahead Ex-Ante Forecasting of Air Pollutants Using Machine Learning

In this study, a novel general multi-step ahead strategy is developed for forecasting time series of air pollutants. The values of the predictors at future moments are gathered from official weather forecast sites as independent ex-ante data. They are updated with new forecasted values every day. Each new sample is used to build- a separate single model that simultaneously predicts future pollution levels. The sought forecasts were estimated by averaging the actual predictions of the single models. The strategy was applied to three pollutants—PM10, SO2, and NO2—in the city of Pernik, Bulgaria. Random forest (RF) and arcing (Arc-x4) machine learning algorithms were applied to the modeling. Although there are many highly changing day-to-day predictors, the proposed averaging strategy shows a promising alternative to single models. In most cases, the root mean squared errors (RMSE) of the averaging models (aRF and aAR) for the last 10 horizons are lower than those of the single models. In particular, for PM10, the aRF’s RMSE is 13.1 vs. 13.8 micrograms per cubic meter for the single model; for the NO2 model, the aRF exhibits 21.5 vs. 23.8; for SO2, the aAR has 17.3 vs. 17.4; for NO2, the aAR’s RMSE is 22.7 vs. 27.5, respectively. Fractional bias is within the same limits of (−0.65, 0.7) for all constructed models

Discrete Wavelet Transform and Ensemble Tree Algorithms for Air Pollutant Modeling: A Case Study

Air pollution is one of the greatest environmental problems of our time on a global and local scale. Elevated and even low but constant levels of harmful emissions in the air in urbanized areas pose serious risks to the health of the population. This paper develops a new approach for statistical modeling of time series of air pollutants, depending on meteorological factors. A new framework based on discrete wavelet transform (DWT) is proposed for decomposing pollutant's time series as a sum of components that represent trend, seasonality, and other specific characteristics. A key element in the applied DWT is an adaptive scheme for selecting the threshold value to control the reverse DWT's accuracy for achieving better prediction of the time series values. The resulting components are modeled with cutting-edge predictive ensemble tree algorithms, including bagging, boosting, and stacking techniques. This approach is tested with real data measured with a mobile automated station in the Plovdiv region, Bulgaria. All models are evaluated, analyzed, and cross-validated. The models are applied for short-term pollution forecasts