Deep Reinforcement Learning for Wind and Energy Storage Coordination in Wholesale Energy and Ancillary Service Markets

Global power systems are increasingly reliant on wind energy as a mitigation strategy for climate change. However, the variability of wind energy causes system reliability to erode, resulting in the wind being curtailed and, ultimately, leading to substantial economic losses for wind farm owners. Wind curtailment can be reduced using battery energy storage systems (BESS) that serve as onsite backup sources. Yet, this auxiliary role may significantly hamper the BESS's capacity to generate revenues from the electricity market, particularly in conducting energy arbitrage in the Spot market and providing frequency control ancillary services (FCAS) in the FCAS markets. Ideal BESS scheduling should effectively balance the BESS's role in absorbing onsite wind curtailment and trading in the electricity market, but it is difficult in practice because of the underlying coordination complexity and the stochastic nature of energy prices and wind generation. In this study, we investigate the bidding strategy of a wind-battery system co-located and participating simultaneously in both the Spot and Regulation FCAS markets. We propose a deep reinforcement learning (DRL)-based approach that decouples the market participation of the wind-battery system into two related Markov decision processes for each facility, enabling the BESS to absorb onsite wind curtailment while simultaneously bidding in the wholesale Spot and FCAS markets to maximize overall operational revenues. Using realistic wind farm data, we validated the coordinated bidding strategy for the wind-battery system and find that our strategy generates significantly higher revenue and responds better to wind curtailment compared to an optimization-based benchmark. Our results show that joint-market bidding can significantly improve the financial performance of wind-battery systems compared to individual market participation

Optimal Energy Storage Scheduling for Wind Curtailment Reduction and Energy Arbitrage: A Deep Reinforcement Learning Approach

Wind energy has been rapidly gaining popularity as a means for combating climate change. However, the variable nature of wind generation can undermine system reliability and lead to wind curtailment, causing substantial economic losses to wind power producers. Battery energy storage systems (BESS) that serve as onsite backup sources are among the solutions to mitigate wind curtailment. However, such an auxiliary role of the BESS might severely weaken its economic viability. This paper addresses the issue by proposing joint wind curtailment reduction and energy arbitrage for the BESS. We decouple the market participation of the co-located wind-battery system and develop a joint-bidding framework for the wind farm and BESS. It is challenging to optimize the joint-bidding because of the stochasticity of energy prices and wind generation. Therefore, we leverage deep reinforcement learning to maximize the overall revenue from the spot market while unlocking the BESS's potential in concurrently reducing wind curtailment and conducting energy arbitrage. We validate the proposed strategy using realistic wind farm data and demonstrate that our joint-bidding strategy responds better to wind curtailment and generates higher revenues than the optimization-based benchmark. Our simulations also reveal that the extra wind generation used to be curtailed can be an effective power source to charge the BESS, resulting in additional financial returns.Comment: 2023 IEEE Power & Energy Society General Meeting (PESGM). arXiv admin note: text overlap with arXiv:2212.1336

N-Gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding

The first step to apply deep learning techniques for symbolic music understanding is to transform musical pieces (mainly in MIDI format) into sequences of predefined tokens like note pitch, note velocity, and chords. Subsequently, the sequences are fed into a neural sequence model to accomplish specific tasks. Music sequences exhibit strong correlations between adjacent elements, making them prime candidates for N-gram techniques from Natural Language Processing (NLP). Consider classical piano music: specific melodies might recur throughout a piece, with subtle variations each time. In this paper, we propose a novel method, NG-Midiformer, for understanding symbolic music sequences that leverages the N-gram approach. Our method involves first processing music pieces into word-like sequences with our proposed unsupervised compoundation, followed by using our N-gram Transformer encoder, which can effectively incorporate N-gram information to enhance the primary encoder part for better understanding of music sequences. The pre-training process on large-scale music datasets enables the model to thoroughly learn the N-gram information contained within music sequences, and subsequently apply this information for making inferences during the fine-tuning stage. Experiment on various datasets demonstrate the effectiveness of our method and achieved state-of-the-art performance on a series of music understanding downstream tasks. The code and model weights will be released at https://github.com/CinqueOrigin/NG-Midiformer.Comment: 8 pages, 2 figures, aaai202

Model-Free Approach to Fair Solar PV Curtailment Using Reinforcement Learning

The rapid adoption of residential solar photovoltaics (PV) has resulted in regular overvoltage events, due to correlated reverse power flows. Currently, PV inverters prevent damage to electronics by curtailing energy production in response to overvoltage. However, this disproportionately affects households at the far end of the feeder, leading to an unfair allocation of the potential value of energy produced. Globally optimizing for fair curtailment requires accurate feeder parameters, which are often unknown. This paper investigates reinforcement learning, which gradually optimizes a fair PV curtailment strategy by interacting with the system. We evaluate six fairness metrics on how well they can be learned compared to an optimal solution oracle. We show that all definitions permit efficient learning, suggesting that reinforcement learning is a promising approach to achieving both safe and fair PV coordination

Nonlinear vibrations of beams with spring and damping delayed feedback control

The primary, subharmonic, and superharmonic resonances of an Euler–Bernoulli beam subjected to harmonic excitations are studied with damping and spring delayed-feedback controllers. By method of multiple scales, the non-linear governing partial differential equation is transformed into linear differential equations directly. Effects of the feedback gains and time-delays on the steady state responses are investigated. The velocity and displacement delayed-feedback controllers are employed to suppress the primary and superharmonic resonances of the forced nonlinear oscillator. The stable vibration regions of the feedback gains and time-delays are worked out based on stablility conditions of the resonances. It is found that proper selection of feedback gains and time-delays can enhance the control performance of beam’s nonlinear vibration. Position of the bifurcation point can be changed or the bifurcation can be eliminated

Are Diffusion Models Vulnerable to Membership Inference Attacks?

Diffusion-based generative models have shown great potential for image synthesis, but there is a lack of research on the security and privacy risks they may pose. In this paper, we investigate the vulnerability of diffusion models to Membership Inference Attacks (MIAs), a common privacy concern. Our results indicate that existing MIAs designed for GANs or VAE are largely ineffective on diffusion models, either due to inapplicable scenarios (e.g., requiring the discriminator of GANs) or inappropriate assumptions (e.g., closer distances between synthetic samples and member samples). To address this gap, we propose Step-wise Error Comparing Membership Inference (SecMI), a query-based MIA that infers memberships by assessing the matching of forward process posterior estimation at each timestep. SecMI follows the common overfitting assumption in MIA where member samples normally have smaller estimation errors, compared with hold-out samples. We consider both the standard diffusion models, e.g., DDPM, and the text-to-image diffusion models, e.g., Latent Diffusion Models and Stable Diffusion. Experimental results demonstrate that our methods precisely infer the membership with high confidence on both of the two scenarios across multiple different datasets. Code is available at https://github.com/jinhaoduan/SecMI.Comment: To appear in ICML 202

PO-107 Applied Research on Heart Rate Variability in Monitoring Sports Fatigue of Boxing Athletes

Objective Based on the diagnosis of sports fatigue using physiological and biochemical indicators, to detect the changes of heart rate variability (HRV) index before and after heavy load training in boxing athletes, and observe the effect of heavy load training on cardiac autonomic nerves. The purpose of this study was to investigate the application of HRV to monitor boxing athletes’ sports fatigue. Methods 16 athletes from Shanghai men's boxing team were recruited. The coach organized a 4-week heavy load training, on Monday morning before and after heavy load training, to evaluate whether athletes have exercise fatigue by testing white blood cell (WBC), red blood cell (RBC), hemoglobin (Hb), blood testosterone (T), cortisol (C), testosterone/cortisol ratio (T/C), creatine kinase (CK), blood urea (BU) and morning pulse. Heart rate variability (HRV) indicators were detected simultaneously. The data were analyzed by SPSS 19.0 statistical software. Pearson correlation analysis was used to compare the correlation between HRV and physiological and biochemical indexes. The paired sample T test was used to compare the differences between the indicators, P<0.05, P<0.01 was statistically significant. Results After heavy load training, when compared with indexes before heavy load training, T and T/C ratios decreased significantly (-38%, -52.7%, p<0.01), C and morning pulse increased significantly (+32.4%, +20.4%, p<0.05), BU and CK had an increasing trend but no statistical significance (+16.5%, +52.7%, p>0.05), while WBC, RBC and Hb showed no statistical significance (p>0.05), these changes in physiological and biochemical indexes can diagnose sports fatigue of boxing athletes after heavy load training. SDNN of HRV index was significantly correlated with morning pulse (p<0.05), RMSSD was significantly correlated with CK (p<0.05), LF was significantly correlated with Hb (p<0.05), and LF/HF was significantly correlated with T, C, T/C, morning pulse, CK (p<0.05). After heavy load training, LF and LF/HF of HRV index in boxing athletes were significantly increased than that before heavy load training (1744.7±1526.3 ms2 vs. 1134.5±1003.3 ms2, 2.5±1.3 vs. 1.6±1.0, p<0.05), the other HRV indexes showed no statistical significance (p>0.05). Conclusions The LF and LF/HF changed significantly when boxing athletes appeared sports fatigue, suggesting that the sympathetic nervous system had enhanced activity and increased tension, the imbalance between Sympathetic and parasympathetic tend to predominate in sympathetic activity. LF and LF/HF are sensitive HRV indicators for monitoring sports fatigue in boxing athletes