Robust Losses for Learning Value Functions

Most value function learning algorithms in reinforcement learning are based on the mean squared (projected) Bellman error. However, squared errors are known to be sensitive to outliers, both skewing the solution of the objective and resulting in high-magnitude and high-variance gradients. To control these high-magnitude updates, typical strategies in RL involve clipping gradients, clipping rewards, rescaling rewards, or clipping errors. While these strategies appear to be related to robust losses -- like the Huber loss -- they are built on semi-gradient update rules which do not minimize a known loss. In this work, we build on recent insights reformulating squared Bellman errors as a saddlepoint optimization problem and propose a saddlepoint reformulation for a Huber Bellman error and Absolute Bellman error. We start from a formalization of robust losses, then derive sound gradient-based approaches to minimize these losses in both the online off-policy prediction and control settings. We characterize the solutions of the robust losses, providing insight into the problem settings where the robust losses define notably better solutions than the mean squared Bellman error. Finally, we show that the resulting gradient-based algorithms are more stable, for both prediction and control, with less sensitivity to meta-parameters.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence (2022

What You Need to Know about Bar-Code Medication Administration

Medication errors are the most common type of preventable error. Bar-code medication administration (BCMA) technology was designed to reduce medication administration errors. Poor system design, implementation and workarounds remain a cause of errors. This paper reviews the literature on BCMA, identifies a gap in the findings and identifies three evidence based practices that could be used to improve system implementation and reduce error. The literature review identified that Bar-code medication administration and system workarounds are well documented and affect patient safety. Based on the critical analysis of 10 studies, we identified gaps in the standardization of BCMA planning, implementation, and sustainability. The themes that emerged from the literature were poor BCMA design and implementation that resulted in workarounds.The three evidence based strategies proposed to address this gap are, evidence based standardization in planning and implementation, the identification and elimination of workarounds and hard wiring. An evidence based checklist evaluates compliance with standard procedures. The LEAN model of Jodoka is used to assure adaptation of the machine to human workflow. Direct observation provides valuable workflow assessment. An effective BCMA implementation involves careful system design, identification of workflow issues which cause workarounds, and adapting the machine to nursing needs

Reviews

1977 Tolkien Calendar. Greg and Tim Hildebrandt. Reviewed by Nancy-Lou Patterson. The Lord of the Rings 1977 Calendar. Illustrations by J. R. R. Tolkien, notes by Christopher Tolkien. Reviewed by Nancy-Lou Patterson. Adventure, Mystery, and Romance: Formula Stories as Art and Popular Culture. John G. Caweiti. Reviewed by Joe R. Christopher. Encyclopedia of Mystery and Detection. Chris Steinbrunner and Otto Penzler (eds.). Reviewed by Joe R. Christopher. The Father Christmas Letters. John Ronald Reuel Tolkien. Reviewed by Martha and Laurence Krieg. The Middle-earth Song- book. Ruth Berman and Ken Nahigian (eds.). Reviewed by George Colvin. From Elfland to Poughkeepsie. Ursula K. Le Guin. Reviewed by George Colvin. Camber of Culdi. Katherine Kurtz. Reviewed by George Colvin

Simple and inexpensive method for the detection of carbon monoxide released from thermal cheletropic decarbonylation reactions

This work presents a simple protocol that demonstrates the use of an inexpensive household carbon monoxide detector as a useful tool to detect the release of carbon monoxide gas from thermal cheletropic decarbonylation reactions. The carbon monoxide detection method described has been employed in a short series of reactions used in a university teaching laboratory setting, and full procedures for these are outlined. In each case, the procedure to isolate and identify the decarbonylation products has also been provided.PostprintPeer reviewe