Supported Trust Region Optimization for Offline Reinforcement Learning

Offline reinforcement learning suffers from the out-of-distribution issue and extrapolation error. Most policy constraint methods regularize the density of the trained policy towards the behavior policy, which is too restrictive in most cases. We propose Supported Trust Region optimization (STR) which performs trust region policy optimization with the policy constrained within the support of the behavior policy, enjoying the less restrictive support constraint. We show that, when assuming no approximation and sampling error, STR guarantees strict policy improvement until convergence to the optimal support-constrained policy in the dataset. Further with both errors incorporated, STR still guarantees safe policy improvement for each step. Empirical results validate the theory of STR and demonstrate its state-of-the-art performance on MuJoCo locomotion domains and much more challenging AntMaze domains.Comment: Accepted at ICML 202

Forecasting and alert of atmospheric bioaerosol concentration profile based on adaptive genetic algorithm back propagation neural network, atmospheric parameter and fluorescence lidar

Bioaerosols are biologically originated particles in the atmosphere, which is mainly composed of bacteria, fungi, viruses, pollen, spores, and the fragmentation and disintegration of plants and animals. Bioaerosols are easy to be spread in the lower atmosphere and cause various epidemic diseases, which is harmful to human health. The forecasting and alert of bioaerosols have important scientific significance and reality needs. In this paper, a method is proposed for estimating and predicting the concentration profile of atmospheric bioaerosols using fluorescence lidar observational data. Using the powerful nonlinear prediction ability of artificial neural networks and through repeated training, a mathematical model can be established for the relationship among atmospheric environment, meteorological parameters, and bioaerosol concentration profiles. The input parameters are temperature and humidity, aerosol extinction coefficient, backscatter coefficient, PM2.5, PM10, SO2, NO2, CO, O3, and wind speed, and outputs the concentration profile of bioaerosols. The prediction results with the measurement relative deviation of genetic algorithm back propagation (GA-BP) neural network and adaptive genetic algorithm back propagation (AGA-BP) neural network were analyzed. The results indicate that the AGA-BP neural network can effectively predict the concentration distribution of bioaerosols, and the predicted concentrations of bioaerosols are 1793 particles × m−3, 3088 particles × m−3, 5261 particles × m−3, 7410 particles × m−3 and 9133 particles × m−3 for air quality with superior, fine, mild contamination, middle level pollution and heavy pollution at an altitude of 0.315 km, respectively. We found that the predicted concentration of pollution weather is much higher than that of good weather. Furthermore, the AGA-BP neural network was used to predict the concentration profiles of atmospheric bioaerosols under different weather conditions, which provided a new research method for forecasting and alert of atmospheric bioaerosols