Learning to Teach Reinforcement Learning Agents

In this article we study the transfer learning model of action advice under a budget. We focus on reinforcement learning teachers providing action advice to heterogeneous students playing the game of Pac-Man under a limited advice budget. First, we examine several critical factors affecting advice quality in this setting, such as the average performance of the teacher, its variance and the importance of reward discounting in advising. The experiments show the non-trivial importance of the coefficient of variation (CV) as a statistic for choosing policies that generate advice. The CV statistic relates variance to the corresponding mean. Second, the article studies policy learning for distributing advice under a budget. Whereas most methods in the relevant literature rely on heuristics for advice distribution we formulate the problem as a learning one and propose a novel RL algorithm capable of learning when to advise, adapting to the student and the task at hand. Furthermore, we argue that learning to advise under a budget is an instance of a more generic learning problem: Constrained Exploitation Reinforcement Learning

Neurologists' lived experiences of communicating the diagnosis of a motor neurodegenerative condition: an interpretative phenomenological analysis

BACKGROUND Receiving the diagnosis of a motor neurodegenerative condition (MNDC) can be a life-changing experience. Although several studies of individuals' experiences have indicated dissatisfaction with aspects of how an MNDC diagnosis was communicated, few studies have addressed doctors' experiences of breaking bad news for these conditions, especially from a qualitative perspective. This study explored UK neurologists' lived experience of delivering an MNDC diagnosis. METHODS Interpretative phenomenological analysis was used as the overarching method. Eight consultant neurologists working with patients with MNDCs took part in individual, semi-structured interviews. RESULTS Two themes were constructed from the data: 'Meeting patients' emotional and information needs at diagnosis: a balancing act between disease, patient and organization-related factors', and 'Empathy makes the job harder: the emotional impact and uncovered vulnerabilities associated with breaking bad news'. Breaking the news of an MNDC diagnosis was challenging for participants, both in terms of achieving a patient-centred approach and in terms of dealing with their own emotions during the process. CONCLUSIONS Based on the study's findings an attempt to explain sub-optimal diagnostic experiences documented in patient studies was made and how organizational changes can support neurologists with this demanding clinical task was discussed

Neurologists' current practice and perspectives on communicating the diagnosis of a motor neurodegenerative condition: a UK survey

Background The communication of a life-changing diagnosis can be a difficult task for doctors with potential long-term effects on patient outcomes. Although several studies have addressed the experiences of individuals with motor neurodegenerative diseases in receiving this diagnosis, a significant research gap exists regarding professionals’ perspectives, especially in the UK. This study aimed to assess UK neurologists’ current practice and perspectives on delivering the diagnosis of a motor neurodegenerative disease, explore different aspects of the process and detail the potential challenges professionals might face. Methods We conducted an anonymised online survey with 44 questions, grouped into four sections; basic demographic information, current practice, the experience of breaking bad news and education and training needs. Results Forty-nine professionals completed the survey. Overall, participants seemed to meet the setting-related standards of good practice; however, they also acknowledged the difficulty of this aspect of their clinical work, with about half of participants (46.5%) reporting moderate levels of stress while breaking bad news. Patients’ relatives were not always included in diagnostic consultations and participants were more reluctant to promote a sense of optimism to patients with poorer prognosis. Although professionals reported spending a mean of around 30–40 min for the communication of these diagnoses, a significant proportion of participants (21–39%) reported significantly shorter consultation times, highlighting organisational issues related to lack of capacity. Finally, the majority of participants (75.5%) reported not following any specific guidelines or protocols but indicated their interest in receiving further training in breaking bad news (78.5%). Conclusions This was the first UK survey to address neurologists’ practice and experiences in communicating these diagnoses. Although meeting basic standards of good practice was reported by most professionals, we identified several areas of improvement. These included spending enough time to deliver the diagnosis appropriately, including patients’ relatives as a standard, promoting a sense of hope and responding to professionals’ training needs regarding breaking bad news

Exploring socioeconomic inequities in access to palliative and end-of-life care in the UK: a narrative synthesis

Background: Efforts inequities in access to palliative and end-of-life care require comprehensive understanding about the extent of and reasons for inequities. Most research on this topic examines differences in receipt of care. There is a need, particularly in the UK, for theoretically driven research that considers both receipt of care and the wider factors influencing the relationship between socioeconomic position and access to palliative and end-of-life care. Methods: This is a mixed studies narrative synthesis on socioeconomic position and access to palliative and end-of-life care in the UK. Study searches were conducted in databases AMED, Medline, Embase, CINAHL, SocIndex, and Academic Literature Search, as well as grey literature sources, in July 2020. The candidacy model of access, which describes access as a seven-stage negotiation between patients and providers, guided study searches and provided a theoretical lens through which data were synthesised. Results: Searches retrieved 5303 studies (after de-duplication), 29 of which were included. The synthesis generated four overarching themes, within which concepts of candidacy were evident: identifying needs; taking action; local conditions; and receiving care. Conclusion: There is not a consistent or clear narrative regarding the relationship between socioeconomic position and receipt of palliative and end-of-life care in the UK. Attempts to address any inequities in access will require knowledge and action across many different areas. Key evidence gaps in the UK literature concern the relationship between socioeconomic position, organisational context, and assessing need for care

Healthcare professionals’ involvement in breaking bad news to newly diagnosed patients with motor neurodegenerative conditions: a qualitative study

Purpose Research on breaking bad news (BBN) in healthcare has mostly focused on the doctor-patient interaction during a single consultation. However, it has been increasingly recognised that BBN is a wider process that also involves other healthcare professionals. This qualitative study explored non-medical1 healthcare professionals’ involvement in BBN to newly diagnosed patients with motor neurodegenerative conditions in the UK. Materials and methods 19 healthcare professionals working with people with motor neurone disease, multiple sclerosis, Parkinson’s disease or Huntington’s disease took part in individual, semi-structured interviews which were analysed using thematic analysis. Results Four themes were constructed: dealing with the diagnostic aftermath, unpacking the diagnosis, breaking bad news as a balancing act and empowering patients to regain control over their health and lives. Participants reported being broadly involved in BBN by supporting patients with negative diagnostic experiences, re-iterating diagnostic information and helping patients understand the impact of their condition. The challenges of effectively breaking bad news and how these difficult conversations could help empower patients were also emphasised. Conclusions BBN was a critical and challenging aspect of healthcare professionals’ clinical work with newly diagnosed patients with motor neurodegenerative conditions. Besides providing information, BBN was perceived as a way to educate patients, encourage them to make decisions and prepare for the future

‘It's a double whammy’ A qualitative study of illness uncertainty in individuals with Parkinson's disease in the context of COVID-19

Objectives: The purpose of this study was to explore the experiences of individuals with Parkinson's through the theoretical lens of illness uncertainty during the first UK full lockdown period (March–June 2020) put in place due outbreak of the COVID-19 pandemic. Methods: Individual semi-structured interviews were carried out via telephone in May 2020 with 10 individuals with Parkinson's (six men and four women) recruited from Parkinson's UK. Interviews were recorded and transcribed verbatim, and thematic analysis was adopted to analyse the resulting data. Results: Four overarching themes emerged from the interview data: (1) COVID-19 amplifying existing fears and difficulties around the uncertainty of Parkinson's; (2) practical and psychological efforts to manage uncertainty; (3) benefit-finding as a way of acknowledging the positives of lockdown; (4) risk and future management in the context of uncertainty. Discussion: Participants reported a range of implicit and explicit strategies to cope with the ‘double whammy’ of uncertainty caused by having Parkinson's during a global pandemic. While these were generally successful in maintaining well-being, it is important that such successful accounts are used to help inform novel strategies and interventions targeting individuals who might need additional support

Manifestations of Overarousal Account For the Association Between Cognitive Anxiety Sensitivity and Suicidal Ideation

Background: Recent evidence suggests an association between cognitive anxiety sensitivity and suicidal ideation. Cognitive anxiety sensitivity has also been implicated as a precursor to various forms of overarousal. These manifestations of overarousal (i.e., agitation, insomnia, nightmares, and anger) may account for the association between cognitive anxiety sensitivity and suicidal ideation. Methods: In Study 1, undergraduate students selectively sampled for recent suicidal ideation completed all measures online. In Study 2, clinical outpatients completed all measures prior to their initial intake appointments at a psychology clinic. Results: Study 1 demonstrated that agitation and insomnia individually and jointly accounted for the association between cognitive anxiety sensitivity and suicidal ideation, controlling for general anxiety and demographic variables. Study 2 replicated and extended these findings, such that, controlling for demographics, general anxiety, and physical and social anxiety sensitivity, agitation and anger each independently and together accounted for the association between cognitive anxiety sensitivity and suicidal ideation, whereas insomnia and nightmares did not. Limitations: This study utilized a cross-sectional design and self-report measures in both samples as well as a sample of undergraduate students in Study 1. Conclusions: Together, these findings suggest that agitation and anger may explain the previously established relationship between cognitive anxiety sensitivity and suicidal ideation. Targeting cognitive anxiety sensitivity in treatment may in turn reduce these forms of overarousal and thereby suicide risk

Methodological considerations in the analysis of fecal glucocorticoid metabolites in tufted capuchins (Cebus apella)

Analysis of fecal glucocorticoid (GC) metabolites has recently become the standard method to monitor adrenocortical activity in primates noninvasively. However, given variation in the production, metabolism, and excretion of GCs across species and even between sexes, there are no standard methods that are universally applicable. In particular, it is important to validate assays intended to measure GC production, test extraction and storage procedures, and consider the time course of GC metabolite excretion relative to the production and circulation of the native hormones. This study examines these four methodological aspects of fecal GC metabolite analysis in tufted capuchins (Cebus apella). Specifically, we conducted an adrenocorticotrophic hormone (ACTH) challenge on one male and one female capuchin to test the validity of four GC enzyme immunoassays (EIAs) and document the time course characterizing GC me- tabolite excretion in this species. In addition, we compare a common field-friendly technique for extracting fecal GC metabolites to an established laboratory extraction methodology and test for effects of storing “field extracts” for up to 1 yr. Results suggest that a corticosterone EIA is most sensitive to changes in GC production, provides reliable measures when extracted according to the field method, and measures GC metabolites which remain highly stable after even 12 mo of storage. Further, the time course of GC metabolite excretion is shorter than that described yet for any primate taxa. These results provide guidelines for studies of GCs in tufted capuchins, and underscore the importance of validating methods for fecal hormone analysis for each species of interest