Revisiting Discrete Soft Actor-Critic

We study the adaption of soft actor-critic (SAC) from continuous action space to discrete action space. We revisit vanilla SAC and provide an in-depth understanding of its Q value underestimation and performance instability issues when applied to discrete settings. We thereby propose entropy-penalty and double average Q-learning with Q-clip to address these issues. Extensive experiments on typical benchmarks with discrete action space, including Atari games and a large-scale MOBA game, show the efficacy of our proposed method. Our code is at:https://github.com/coldsummerday/Revisiting-Discrete-SAC

Gain scheduled torque compensation of PMSG-based wind turbine for frequency regulation in an isolated grid

Frequency stability in an isolated grid can be easily impacted by sudden load or wind speed changes. Many frequency regulation techniques are utilized to solve this problem. However, there are only few studies designing torque compensation controllers based on power performances in different Speed Parts. It is a major challenge for a wind turbine generator (WTG) to achieve the satisfactory compensation performance in different Speed Parts. To tackle this challenge, this paper proposes a gain scheduled torque compensation strategy for permanent magnet synchronous generator (PMSG) based wind turbines. Our main idea is to improve the anti-disturbance ability for frequency regulation by compensating torque based on WTG speed Parts. To achieve higher power reserve in each Speed Part, an enhanced deloading method of WTG is proposed. We develop a new small-signal dynamic model through analyzing the steady-state performances of deloaded WTG in the whole range of wind speed. Subsequently, H∞ theory is leveraged in designing the gain scheduled torque compensation controller to effectively suppress frequency fluctuation. Moreover, since torque compensation brings about untimely power adjustment in over-rated wind speed condition, the conventional speed reference of pitch control system is improved. Our simulation and experimental results demonstrate that the proposed strategy can significantly improve frequency stability and smoothen power fluctuation resulting from wind speed variations. The minimum of frequency deviation with the proposed strategy is improved by up to 0.16 Hz at over-rated wind speed. Our technique can also improve anti-disturbance ability in frequency domain and achieve power balance

Multiple adaptive model predictive controllers for frequency regulation in wind farms

Frequent and inadequate power regulation could significantly impact the main shaft mechanical load and the fatigue of wind turbines, which imposes a stringent requirement to perform frequency regulation. However, the existing work on frequency regulation mainly uses torque compensation to improve the frequency response, while few of them consider the mechanical fatigue of the main shaft caused by torque compensation of the frequency controller. In this paper, the mechanical fatigue of the main shaft can be mitigated in all of the speed sections thanks to the proposed frequency regulation controllers. Precisely, a multiple adaptive model predictive controller (MAMPC), which seamlessly integrates the multiple model predictive control (MMPC) and the real-time AutoRegressive with eXogenous inputs (ARX) model, is proposed. It nicely handles the rate of change in compensation torque to mitigate the mechanical load on the shaft in all of the speed sections. The effectiveness of our method is verified through extensive simulations. With the proposed method, the minimum frequency deviation can be reduced, and the number of fatigue cycles of the main shaft can be extended

Optimal dispatch based on prediction of distributed electric heating storages in combined electricity and heat networks

The volatility of wind power generations could significantly challenge the economic and secure operation of combined electricity and heat networks. To tackle this challenge, this paper proposes a framework of optimal dispatch with distributed electric heating storage based on a correlation-based long short-term memory prediction model. The prediction model of distributed electric heating storage is developed to model its behavior characteristics which are obtained by the autocorrelation and correlation analysis with external factors including weather and time-of-use price. An optimal dispatch model of combined electricity and heat networks is then formulated and resolved by a constraint reduction technique with clustering and classification. Our method is verified through numerous simulations. The results show that, compared with the state-of-the-art techniques of support vector machine and recurrent neural networks, the mean absolute percentage error with the proposed correlation-based long short-term memory can be reduced by 1.009 and 0.481 respectively. Compared with conventional method, the peak wind power curtailment with dispatching distributed electric heating storage is reduced by nearly 30% and 50% in two cases respectively

Upper Arm Motion High-Density sEMG Recognition Optimization Based on Spatial and Time-Frequency Domain Features

Background. Spatial characteristics of sEMG signals are obtained by high-density matrix sEMG electrodes for further complex upper arm movement classification. Multiple electrode channels of the high-density sEMG acquisition device aggravate the burden of the microprocessor and deteriorate control system’s real-time performance at the same time. A shoulder motion recognition optimization method based on the maximizing mutual information from multiclass CSP selected spatial feature channels and wavelet packet features extraction is proposed in this study. Results. The relationship between the number of channels and recognition rate is obtained by the recognition optimization method. The original 64 electrodes channels are reduced to only 4-5 active signal channels with the accuracy over 92%. Conclusion. The shoulder motion recognition optimization method is combined with the spatial-domain and time-frequency-domain features. In addition, the spatial feature channel selection is independent of feature extraction and classification algorithm. Therefore, it is more convenient to use less channels to achieve the desired classification accuracy

A Simulation Study of the Impact of Urban Street Greening on the Thermal Comfort in Street Canyons on Hot and Cold Days

The urban heat island effect has become a widely concerning issue worldwide. Many researchers have made great efforts to improve the summer thermal comfort of urban street canyons by optimizing street greening. Relatively less research has focused on how to improve the thermal comfort of street canyons by optimizing street greening during cold days. Many researchers have proposed strategies to improve the summer thermal comfort of street canyons using road greening. This may have a significant negative impact on the winter thermal comfort of street canyons due to the lack of consideration of the impact on hot and cold days simultaneously, especially when the road green space is mainly composed of evergreen tree species. We aimed to explore the impacts of urban street greening on thermal comfort on hot and cold days at the same time. We used Zhutang West Road in Changsha, China, as an example and built six different models to explore the impacts of the street vegetation types, number of street trees, tree heights, crown widths, and Leaf Area Index on the thermal comfort of the street canyon. In addition, we also considered the impact of different building features and wind directions on the thermal comfort of the street canyon. We employed ENVI-met (version 5.5.1) to simulate different urban street greening models. The results show that the model with a high tree canopy density, tall trees, large and dense crowns, and sufficient building shade has good thermal comfort on hot days (the average physiological equivalent temperature (PET) is 31.1 °C for the study period) and bad thermal comfort on cold days (the PET is 13.3 °C) when it is compared with the other models (the average PETs are 36.2 °C, 31.5 °C, 41.5 °C, 36.2 °C, and 35.5 °C, respectively, on hot days and for other models). In addition, the model with a very large number of short hedges has a positive impact on thermal comfort during hot days (the PET is 31.1 °C). The PET value of another comparable model which does not have hedges is 31.5 °C. Even if the model with a small building area has good ventilation, the small building shade area in the model has a more obvious impact and the model has relatively good thermal comfort during cold days (the PET is 14.2 °C) when it is compared to models with bigger building areas (the PET is 13.9 °C). In summer, when the wind is parallel to the direction of the street canyon, the wind speed in the street canyon is high and the model has relatively good thermal comfort (the PET is 35.5 °C) compared with another model which has different wind direction and lower wind speed at the street canyon (the PET is 36.2 °C). In winter, when the wind is perpendicular to the direction of the street canyon, buildings and trees have a strong windproof effect and this is beneficial to the improvement of thermal comfort (the PET is 15.3 °C for this model and 13.9 °C for another comparable model). This research lays a solid foundation and encourages people to think about the impact of building and tree composition and configuration on the thermal comfort of street canyons during hot and cold days simultaneously