Hybrid Reinforcement Learning with Expert State Sequences

Existing imitation learning approaches often require that the complete demonstration data, including sequences of actions and states, are available. In this paper, we consider a more realistic and difficult scenario where a reinforcement learning agent only has access to the state sequences of an expert, while the expert actions are unobserved. We propose a novel tensor-based model to infer the unobserved actions of the expert state sequences. The policy of the agent is then optimized via a hybrid objective combining reinforcement learning and imitation learning. We evaluated our hybrid approach on an illustrative domain and Atari games. The empirical results show that (1) the agents are able to leverage state expert sequences to learn faster than pure reinforcement learning baselines, (2) our tensor-based action inference model is advantageous compared to standard deep neural networks in inferring expert actions, and (3) the hybrid policy optimization objective is robust against noise in expert state sequences.Comment: AAAI 2019; https://github.com/XiaoxiaoGuo/tensor4r

Conflict-driven Hybrid Observer-based Anomaly Detection

This paper presents an anomaly detection method using a hybrid observer -- which consists of a discrete state observer and a continuous state observer. We focus our attention on anomalies caused by intelligent attacks, which may bypass existing anomaly detection methods because neither the event sequence nor the observed residuals appear to be anomalous. Based on the relation between the continuous and discrete variables, we define three conflict types and give the conditions under which the detection of the anomalies is guaranteed. We call this method conflict-driven anomaly detection. The effectiveness of this method is demonstrated mathematically and illustrated on a Train-Gate (TG) system

Drivers of overall satisfaction with primary care: Evidence from the English General Practice Patient Survey

This is the final version. Available from Wiley via the DOI in this record.Background/objectives: To determine which aspects of primary care matter most to patients, we aim to identify those aspects of patient experience that show the strongest relationship with overall satisfaction and examine the extent to which these relationships vary by socio-demographic and health characteristics. Design/setting: Data from the 2009/10 English General Practice Patient Survey including 2 169 718 respondents registered with 8362 primary care practices. Measures/analyses: Linear mixed-effects regression models (fixed effects adjusting for age, gender, ethnicity, deprivation, self-reported health, self-reported mental health condition and random practice effect) predicting overall satisfaction from six items covering four domains of care: access, helpfulness of receptionists, doctor communication and nurse communication. Additional models using interactions tested whether associations between patient experience and satisfaction varied by socio-demographic group. Results: Doctor communication showed the strongest relationship with overall satisfaction (standardized coefficient 0.48, 95% CI = 0.48, 0.48), followed by the helpfulness of reception staff (standardized coefficient 0.22, 95% CI = 0.22, 0.22). Among six measures of patient experience, obtaining appointments in advance showed the weakest relationship with overall satisfaction (standardized coefficient 0.06, 95% CI = 0.05, 0.06). Interactions showed statistically significant but small variation in the importance of drivers across different patient groups. Conclusions: For all patient groups, communication with the doctor is the most important driver of overall satisfaction with primary care in England, along with the helpfulness of receptionists. In contrast, and despite being a policy priority for government, measures of access, including the ability to obtain appointments, were poorly related to overall satisfaction.UK Department of HealthNational Institute for Health Research (NIHR

Using Faculty Learning Communities to Link FYE and High-Risk Core Courses: A Pilot Study

Can success rates in a gateway course be improved by linking it to a college success course? This article describes the results of a pilot study that linked a first-year biology course that had a high drop-out and failure rate to a college success course that included study skills. The proposal to link courses came from the work of a faculty learning community aimed at sharing strategies for increasing engagement in first year courses. Faculty involved in the link worked closely together. The college success course used biology content to provide hands-on study skills applications for students. The results illustrate that students in the pilot program did significantly better in the biology course as well as in their overall fall GPA than students in the same biology course who were not in the learning community

JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions

Commonsense reasoning simulates the human ability to make presumptions about our physical world, and it is an essential cornerstone in building general AI systems. We propose a new commonsense reasoning dataset based on human's Interactive Fiction (IF) gameplay walkthroughs as human players demonstrate plentiful and diverse commonsense reasoning. The new dataset provides a natural mixture of various reasoning types and requires multi-hop reasoning. Moreover, the IF game-based construction procedure requires much less human interventions than previous ones. Experiments show that the introduced dataset is challenging to previous machine reading models with a significant 20% performance gap compared to human experts.Comment: arXiv admin note: text overlap with arXiv:2010.0978

Improved radiative corrections for (e,e'p) experiments: Beyond the peaking approximation and implications of the soft-photon approximation

Analysing (e,e'p) experimental data involves corrections for radiative effects which change the interaction kinematics and which have to be carefully considered in order to obtain the desired accuracy. Missing momentum and energy due to bremsstrahlung have so far always been calculated using the peaking approximation which assumes that all bremsstrahlung is emitted in the direction of the radiating particle. In this article we introduce a full angular Monte Carlo simulation method which overcomes this approximation. The angular distribution of the bremsstrahlung photons is reconstructed from H(e,e'p) data. Its width is found to be underestimated by the peaking approximation and described much better by the approach developed in this work.Comment: 11 pages, 13 figure