Are PPO-ed Language Models Hackable?

Numerous algorithms have been proposed to $\textit{align}$ language models to remove undesirable behaviors. However, the challenges associated with a very large state space and creating a proper reward function often result in various jailbreaks. Our paper aims to examine this effect of reward in the controlled setting of positive sentiment language generation. Instead of online training of a reward model based on human feedback, we employ a statically learned sentiment classifier. We also consider a setting where our model's weights and activations are exposed to an end-user after training. We examine a pretrained GPT-2 through the lens of mechanistic interpretability before and after proximal policy optimization (PPO) has been applied to promote positive sentiment responses. Using these insights, we (1) attempt to "hack" the PPO-ed model to generate negative sentiment responses and (2) add a term to the reward function to try and alter `negative' weights.Comment: 8 pages, 4 figure

Patterns of management of patients with dual disorder (psychosis) in Italy: a survey of psychiatrists and other physicians focusing on clinical practice

© 2018 Clerici, de Bartolomeis, De Filippis, Ducci, Maremmani, Martinotti and Schifano. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY).Patients with severe psychotic disorders such as schizophrenia, schizoaffective and bipolar disorders frequently suffer from concomitant substance use disorders (SUDs) – Dual Disorder (DD) patients. In order to better understand current practices for management of patients with psychotic episodes and concomitant SUD in Italy, we carried out a survey of psychiatrists on current routine practice among prescribers. These aspects can help to identify at-risk patients, improve current prescribing practices, and favor early intervention. An ad hoc survey of 17 questions was administered to psychiatrists via electronic polling and on-line distribution; 448 completed questionnaires were collected. Comorbid substance abuse was most frequently diagnosed within the context of anxiety disorder (46%), followed by bipolar disorder (25%), and schizophrenia/schizoaffective disorder (12%). The vast majority of respondents felt that patient management was becoming more complex due to substance abuse. The areas reported to be most affected in patients with SUD were functioning, interpersonal relations, and impulsivity, while sensory perception disorders, ideation, agitation, and impulsivity were the most frequently reported symptoms. In the acute setting, haloperidol was used as the first-line agent of choice followed by aripiprazole and olanzapine. In the maintenance phase, aripiprazole was the dominantly used first-line agent, followed by olanzapine. Almost half of respondents used long-acting agents, while about one-third did not. Among those prescribing long-acting agents, efficacy, control of impulsivity, and control of specific symptoms were cited as motivators, while in the maintenance phase, better adherence and tolerability were mainly cited. From the responses to the present survey, it is clear that the respondents are aware of the problem of SUD in psychotic patients. While treatment be optimized in terms of the choice and formulation of antipsychotics, greater emphasis should be placed on efficacy, tolerability and the negative metabolic consequences of some antipsychotics. When considering the ideal antipsychotic, long-acting agents were considered to be superior in reducing relapse, even if current treatment guidelines often give preference to oral formulations.Peer reviewe

How Business Cycles Affect the Healthcare Sector: A Cross-country Investigation

The long-term relationship between the general economy and healthcare expenditures has been extensively researched, to explain differences in healthcare spending between countries, but the midterm (i.e., business cycle) perspective has been overlooked. This study explores business cycle sensitivity in both public and private parts of the healthcare sector across 32 countries. Responses to the business cycle vary notably, both across spending sources and across countries. Whereas in some countries, consumers and/or governments cut back, in others, private and/or public healthcare buyers tend to spend more. We also assess long-term consequences of business cycle sensitivity and show that public cost cutting during economic downturns deflates the mortality rates, whereas private cut backs increase the long-term growth in total healthcare expenditures. Finally, multiple factors help explain variability in cyclical sensitivity. Private cost cuts during economic downturns are smaller in countries with a predominantly publicly funded healthcare system and more preventive public activities. Public cut backs during contractions are smaller in countries that rely more on tax-based resources rather than social health insurances

Identification of Bacterial Operons Required During The Plant Immune Response using a RB-TnSeq Approach

Today, the majority of agrochemicals used to increase agricultural output have adverse effects on the environment, and the developments of more sustainable approaches that utilize plant-growth promoting bacteria are needed 1. In order to benefit the plants, these microbes must be able to colonize the plant root in mixed environmental microbial communities that induce immune responses from the plant. Toward understanding the genetic basis of root colonization by plant-growth promoting microbes, a randomly barcoded transposon insertion library sequencing method (Bar-Seq) was utilized to identify genes that protect bacteria from the plant-immune response. Arabidopsis thaliana ecotype Col-0 and two mutants deficient in immune responses were grown for 5 weeks in liquid media and then an immune response was induced for 1 day using flg22 flagellin peptide. Exudates containing anti-microbial chemicals produced by the plants were collected. Bar-Seq libraries of four different plant-associated bacteria were grown in each exudate and the transposon barcodes corresponding to interrupted genes were sequenced. First, the genes are assigned fitness scores by comparing barcode abundance to input. Then, the fitness scores are compared (mutant vs. wild-type, + flg22 vs. – flg22) to identify genes required for growth during the immune response. These results will assist in the efforts to achieve higher crop yields in a more sustainable way, and provide insight into the genetic interactions between bacteria and their hosts.Bachelor of Scienc

Country-level cost-effectiveness thresholds : initial estimates and the need for further research

Objectives: Cost-effectiveness analysis (CEA) can guide policymakers in resource allocation decisions. CEA assesses whether the health gains offered by an intervention are large enough relative to any additional costs to warrant adoption. Where there are constraints on the healthcare system’s budget or ability to increase expenditures, additional costs imposed by interventions have an ‘opportunity cost’ in terms of the health foregone as other interventions cannot be provided. Cost-effectiveness thresholds (CETs) are typically used to assess whether an intervention is worthwhile and should reflect health opportunity cost. However, CETs used by some decision makers - such as the World Health Organization (WHO) suggested CETs of 1-3 times gross domestic product per capita (GDP pc) - do not. This study estimates CETs based on opportunity cost for a wide range of countries. Methods: We estimate CETs based upon recent empirical estimates of opportunity cost (from the English NHS), estimates of the relationship between country GDP pc and the value of a statistical life, and a series of explicit assumptions. Results: CETs for Malawi (the lowest income country in the world), Cambodia (borderline low/low-middle income), El Salvador (borderline low-middle/upper-middle) and Kazakhstan (borderline high-middle/high) are estimated to be $3-116 (1-51$44-518 (4-51%), $422-1,967 (11-51$4,485-8,018 (32-59%); respectively. Conclusions: To date opportunity cost-based CETs for low/middle income countries have not been available. Although uncertainty exists in the underlying assumptions, these estimates can provide a useful input to inform resource allocation decisions and suggest that routinely used CETs have been too high

Mining for Equitable Health: Assessing the Impact of Missing Data in Electronic Health Records

Electronic health records (EHR) are collected as a routine part of healthcare delivery, and have great potential to be utilized to improve patient health outcomes. They contain multiple years of health information to be leveraged for risk prediction, disease detection, and treatment evaluation. However, they do not have a consistent, standardized format across institutions, particularly in the United States, and can present significant analytical challenges- they contain multi-scale data from heterogeneous domains and include both structured and unstructured data. Data for individual patients are collected at irregular time intervals and with varying frequencies. In addition to the analytical challenges, EHR can reflect inequity- patients belonging to different groups will have differing amounts of data in their health records. Many of these issues can contribute to biased data collection. The consequence is that the data for under-served groups may be less informative partly due to more fragmented care, which can be viewed as a type of missing data problem. For EHR data in this complex form, there is currently no framework for introducing realistic missing values. There has also been little to no work in assessing the impact of missing data in EHR. In this work, we first introduce a terminology to define three levels of EHR data and then propose a novel framework for simulating realistic missing data scenarios in EHR to adequately assess their impact on predictive modeling. We incorporate the use of a medical knowledge graph to capture dependencies between medical events to create a more realistic missing data framework. In an intensive care unit setting, we found that missing data have greater negative impact on the performance of disease prediction models in groups that tend to have less access to healthcare, or seek less healthcare. We also found that the impact of missing data on disease prediction models is stronger when using the knowledge graph framework to introduce realistic missing values as opposed to random event removal

The Oregon Health Insurance Experiment: Evidence from the First Year

In 2008, a group of uninsured low-income adults in Oregon was selected by lottery to be given the chance to apply for Medicaid. This lottery provides an opportunity to gauge the effects of expanding access to public health insurance on the health care use, financial strain, and health of low-income adults using a randomized controlled design. In the year after random assignment, the treatment group selected by the lottery was about 25 percentage points more likely to have insurance than the control group that was not selected. We find that in this first year, the treatment group had substantively and statistically significantly higher health care utilization (including primary and preventive care as well as hospitalizations), lower out-of-pocket medical expenditures and medical debt (including fewer bills sent to collection), and better self-reported physical and mental health than the control group.National Institutes of Health. Department of Health and Human ServicesCalifornia HealthCare FoundationJohn D. and Catherine T. MacArthur FoundationNational Institute on Aging (P30AG012810)National Institute on Aging (RC2AGO36631)National Institute on Aging (R01AG0345151)Robert Wood Johnson FoundationAlfred P. Sloan FoundationSmith Richardson FoundationUnited States. Social Security Administration (grant 5 RRC 08098400-03-00 to the National Bureau of Economic Research as part of the SSA Retirement Research Consortium)Centers for Medicare & Medicaid Services (U.S.

Demographic and indication-specific characteristics have limited association with social network engagement: evidence from 24,954 members of four health care support groups

Background: Digital health social networks (DHSNs) are widespread, and the consensus is that they contribute to wellness by offering social support and knowledge sharing. The success of a DHSN is based on the number of participants and their consistent creation of externalities through the generation of new content. To promote network growth, it would be helpful to identify characteristics of superusers or actors who create value by generating positive network externalities. Objective: The aim of the study was to investigate the feasibility of developing predictive models that identify potential superusers in real time. This study examined associations between posting behavior, 4 demographic variables, and 20 indication-specific variables. Methods: Data were extracted from the custom structured query language (SQL) databases of 4 digital health behavior change interventions with DHSNs. Of these, 2 were designed to assist in the treatment of addictions (problem drinking and smoking cessation), and 2 for mental health (depressive disorder, panic disorder). To analyze posting behavior, 10 models were developed, and negative binomial regressions were conducted to examine associations between number of posts, and demographic and indication-specific variables. Results: The DHSNs varied in number of days active (3658-5210), number of registrants (5049-52,396), number of actors (1085-8452), and number of posts (16,231-521,997). In the sample, all 10 models had low R2 values (.013-.086) with limited statistically significant demographic and indication-specific variables. Conclusions: Very few variables were associated with social network engagement. Although some variables were statistically significant, they did not appear to be practically significant. Based on the large number of study participants, variation in DHSN theme, and extensive time-period, we did not find strong evidence that demographic characteristics or indication severity sufficiently explain the variability in number of posts per actor. Researchers should investigate alternative models that identify superusers or other individuals who create social network externalities

Evolving health expenditure landscape of the BRICS nations and projections to 2025

Global health spending share of low/middle income countries continues its long-term growth. BRICS nations remain to be major drivers of such change since 1990s. Governmental, private and out-of-pocket health expenditures were analyzed based on WHO sources. Medium-term projections of national health spending to 2025 were provided based on macroeconomic budgetary excess growth model. In terms of per capita spending Russia was highest in 2013. India's health expenditure did not match overall economic growth and fell to slightly less than 4% of GDP. Up to 2025 China will achieve highest excess growth rate of 2% and increase its GDP% spent on health care from 5.4% in 2012 to 6.6% in 2025. Russia's spending will remain highest among BRICS in absolute per capita terms reaching net gain from $1523 PPP in 2012 to$2214 PPP in 2025. In spite of BRICS' diversity, all countries were able to significantly increase their investments in health care. The major setback was bold rise in out-of-pocket spending. Most of BRICS' growing share of global medical spending was heavily attributable to the overachievement of People's Republic of China. Such trend is highly likely to continue beyond 2025