"I'm sorry Dave, I'm afraid I can't do that" Deep Q-Learning From Forbidden Actions

International audienceThe use of Reinforcement Learning (RL) is still restricted to simulation or to enhance human-operated systems through recommendations. Real-world environments (e.g. industrial robots or power grids) are generally designed with safety constraints in mind implemented in the shape of valid actions masks or contingency controllers. For example, the range of motion and the angles of the motors of a robot can be limited to physical boundaries. Violating constraints thus results in rejected actions or entering in a safe mode driven by an external controller, making RL agents incapable of learning from their mistakes. In this paper, we propose a simple modification of a state-of-the-art deep RL algorithm (DQN), enabling learning from forbidden actions. To do so, the standard Q-learning update is enhanced with an extra safety loss inspired by structured classification. We empirically show that it reduces the number of hit constraints during the learning phase and accelerates convergence to near-optimal policies compared to using standard DQN. Experiments are done on a Visual Grid World Environment and Text-World domain

State Highlights 12/17/1952

This is the student newspaper from Western State High School, the high school that was on the campus of Western Michigan University, then called State Highlights, in 1952. This should be Volume 15

Volume 30, Number 1 - December 1951

Volume 30, Number 1 - December 1951. 56 pages including covers and advertisements. Editorial McGowan, Joseph, The Cheat Gough, Chester, About the Weather Manning, John C., Play Construction Gluckman, M. Howard, All Alone Trofi, Vincent C., The Return of Victor Thurber Gluckman, M. Howard, The Commuters Boudreau, Bernard, What For? Griffin, Henry, Folio Griffin, Henry, Ripe Limes Hung Crinkled Griffin, Henry, A Song of Sadness Griffin, Henry, An Impressio

University High Highlights 3/5/1958

This is the student newspaper from University High School, the high school that was on the campus of Western Michigan University, then called University High Highlights, in 1958

Spartan Daily, February 21, 1978

Volume 70, Issue 14https://scholarworks.sjsu.edu/spartandaily/6305/thumbnail.jp