587 research outputs found
GPU-accelerated stochastic predictive control of drinking water networks
Despite the proven advantages of scenario-based stochastic model predictive
control for the operational control of water networks, its applicability is
limited by its considerable computational footprint. In this paper we fully
exploit the structure of these problems and solve them using a proximal
gradient algorithm parallelizing the involved operations. The proposed
methodology is applied and validated on a case study: the water network of the
city of Barcelona.Comment: 11 pages in double column, 7 figure
A Deep Reinforcement Learning-Based Charging Scheduling Approach with Augmented Lagrangian for Electric Vehicle
This paper addresses the problem of optimizing charging/discharging schedules
of electric vehicles (EVs) when participate in demand response (DR). As there
exist uncertainties in EVs' remaining energy, arrival and departure time, and
future electricity prices, it is quite difficult to make charging decisions to
minimize charging cost while guarantee that the EV's battery
state-of-the-charge (SOC) is within certain range. To handle with this dilemma,
this paper formulates the EV charging scheduling problem as a constrained
Markov decision process (CMDP). By synergistically combining the augmented
Lagrangian method and soft actor critic algorithm, a novel safe off-policy
reinforcement learning (RL) approach is proposed in this paper to solve the
CMDP. The actor network is updated in a policy gradient manner with the
Lagrangian value function. A double-critics network is adopted to synchronously
estimate the action-value function to avoid overestimation bias. The proposed
algorithm does not require strong convexity guarantee of examined problems and
is sample efficient. Comprehensive numerical experiments with real-world
electricity price demonstrate that our proposed algorithm can achieve high
solution optimality and constraints compliance
Approximate non-linear model predictive control with safety-augmented neural networks
Model predictive control (MPC) achieves stability and constraint satisfaction
for general nonlinear systems, but requires computationally expensive online
optimization. This paper studies approximations of such MPC controllers via
neural networks (NNs) to achieve fast online evaluation. We propose safety
augmentation that yields deterministic guarantees for convergence and
constraint satisfaction despite approximation inaccuracies. We approximate the
entire input sequence of the MPC with NNs, which allows us to verify online if
it is a feasible solution to the MPC problem. We replace the NN solution by a
safe candidate based on standard MPC techniques whenever it is infeasible or
has worse cost. Our method requires a single evaluation of the NN and forward
integration of the input sequence online, which is fast to compute on
resource-constrained systems. The proposed control framework is illustrated on
three non-linear MPC benchmarks of different complexity, demonstrating
computational speedups orders of magnitudes higher than online optimization. In
the examples, we achieve deterministic safety through the safety-augmented NNs,
where naive NN implementation fails
Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability
A trustworthy reinforcement learning algorithm should be competent in solving
challenging real-world problems, including {robustly} handling uncertainties,
satisfying {safety} constraints to avoid catastrophic failures, and
{generalizing} to unseen scenarios during deployments. This study aims to
overview these main perspectives of trustworthy reinforcement learning
considering its intrinsic vulnerabilities on robustness, safety, and
generalizability. In particular, we give rigorous formulations, categorize
corresponding methodologies, and discuss benchmarks for each perspective.
Moreover, we provide an outlook section to spur promising future directions
with a brief discussion on extrinsic vulnerabilities considering human
feedback. We hope this survey could bring together separate threads of studies
together in a unified framework and promote the trustworthiness of
reinforcement learning.Comment: 36 pages, 5 figure
- …