Activation Addition: Steering Language Models Without Optimization

Reliably controlling the behavior of large language models (LLMs) is a pressing open problem. Existing methods include supervised finetuning, reinforcement learning from human feedback (RLHF), prompt engineering and guided decoding. We instead investigate activation engineering: modifying activations at inference time to predictably alter model behavior. In particular, we bias the forward pass with an added 'steering vector' implicitly specified through natural language. Unlike past work which learned these steering vectors (Subramani, Suresh, and Peters 2022; Hernandez, Li, and Andreas 2023), our Activation Addition (ActAdd) method computes them by taking the activation differences that result from pairs of prompts. We demonstrate ActAdd on GPT-2 on OpenWebText and ConceptNet. Our inference-time approach yields control over high-level properties of output and preserves off-target model performance. It involves far less compute and implementation effort compared to finetuning or RLHF, allows users to provide natural language specifications, and its overhead scales naturally with model size

Local Data Spaces: Leveraging trusted research environments for secure location-based policy research in the age of coronavirus disease-2019

This work explores the use of Trusted Research Environments for the secure analysis of sensitive, record-level data on local coronavirus disease-2019 (COVID-19) inequalities and economic vulnerabilities. The Local Data Spaces (LDS) project was a targeted rapid response and cross-disciplinary collaborative initiative using the Office for National Statistics’ Secure Research Service for localized comparison and analysis of health and economic outcomes over the course of the COVID-19 pandemic. Embedded researchers worked on co-producing a range of locally focused insights and reports built on secure secondary data and made appropriately open and available to the public and all local stakeholders for wider use. With secure infrastructure and overall data governance practices in place, accredited researchers were able to access a wealth of detailed data and resources to facilitate more targeted local policy analysis. Working with data within such infrastructure as part of a larger research project involved advanced planning and coordination to be efficient. As new and novel granular data resources become securely available (e.g., record-level administrative digital health records or consumer data), a range of local policy insights can be gained across issues of public health or local economic vitality. Many of these new forms of data however often come with a large degree of sensitivity around issues of personal identifiability and how the data is used for public-facing research and require secure and responsible use. Learning to work appropriately with secure data and research environments can open up many avenues for collaboration and analysis

Interpretable brain age prediction using linear latent variable models of functional connectivity

Neuroimaging-driven prediction of brain age, defined as the predicted biological age of a subject using only brain imaging data, is an exciting avenue of research. In this work we seek to build models of brain age based on functional connectivity while prioritizing model interpretability and understanding. This way, the models serve to both provide accurate estimates of brain age as well as allow us to investigate changes in functional connectivity which occur during the ageing process. The methods proposed in this work consist of a two-step procedure: first, linear latent variable models, such as PCA and its extensions, are employed to learn reproducible functional connectivity networks present across a cohort of subjects. The activity within each network is subsequently employed as a feature in a linear regression model to predict brain age. The proposed framework is employed on the data from the CamCAN repository and the inferred brain age models are further demonstrated to generalize using data from two open-access repositories: the Human Connectome Project and the ATR Wide-Age-Range.Peer reviewe

Deep Sequencing of B Cell Receptor Repertoires From COVID-19 Patients Reveals Strong Convergent Immune Signatures.

Deep sequencing of B cell receptor (BCR) heavy chains from a cohort of 31 COVID-19 patients from the UK reveals a stereotypical naive immune response to SARS-CoV-2 which is consistent across patients. Clonal expansion of the B cell population is also observed and may be the result of memory bystander effects. There was a strong convergent sequence signature across patients, and we identified 1,254 clonotypes convergent between at least four of the COVID-19 patients, but not present in healthy controls or individuals following seasonal influenza vaccination. A subset of the convergent clonotypes were homologous to known SARS and SARS-CoV-2 spike protein neutralizing antibodies. Convergence was also demonstrated across wide geographies by comparison of data sets between patients from UK, USA, and China, further validating the disease association and consistency of the stereotypical immune response even at the sequence level. These convergent clonotypes provide a resource to identify potential therapeutic and prophylactic antibodies and demonstrate the potential of BCR profiling as a tool to help understand patient responses

Age-Related Neuronal Degeneration: Complementary Roles of Nucleotide Excision Repair and Transcription-Coupled Repair in Preventing Neuropathology

Neuronal degeneration is a hallmark of many DNA repair syndromes. Yet, how DNA damage causes neuronal degeneration and whether defects in different repair systems affect the brain differently is largely unknown. Here, we performed a systematic detailed analysis of neurodegenerative changes in mouse models deficient in nucleotide excision repair (NER) and transcription-coupled repair (TCR), two partially overlapping DNA repair systems that remove helix-distorting and transcription-blocking lesions, respectively, and that are associated with the UV-sensitive syndromes xeroderma pigmentosum (XP) and Cockayne syndrome (CS). TCR–deficient Csa−/− and Csb−/− CS mice showed activated microglia cells surrounding oligodendrocytes in regions with myelinated axons throughout the nervous system. This white matter microglia activation was not observed in NER–deficient Xpa−/− and Xpc−/− XP mice, but also occurred in XpdXPCS mice carrying a point mutation (G602D) in the Xpd gene that is associated with a combined XPCS disorder and causes a partial NER and TCR defect. The white matter abnormalities in TCR–deficient mice are compatible with focal dysmyelination in CS patients. Both TCR–deficient and NER–deficient mice showed no evidence for neuronal degeneration apart from p53 activation in sporadic (Csa−/−, Csb−/−) or highly sporadic (Xpa−/−, Xpc−/−) neurons and astrocytes. To examine to what extent overlap occurs between both repair systems, we generated TCR–deficient mice with selective inactivation of NER in postnatal neurons. These mice develop dramatic age-related cumulative neuronal loss indicating DNA damage substrate overlap and synergism between TCR and NER pathways in neurons, and they uncover the occurrence of spontaneous DNA injury that may trigger neuronal degeneration. We propose that, while Csa−/− and Csb−/− TCR–deficient mice represent powerful animal models to study the mechanisms underlying myelin abnormalities in CS, neuron-specific inactivation of NER in TCR–deficient mice represents a valuable model for the role of NER in neuronal maintenance and survival

Need for recovery amongst emergency physicians in the UK and Ireland: A cross-sectional survey

OBJECTIVES: To determine the need for recovery (NFR) among emergency physicians and to identify demographic and occupational characteristics associated with higher NFR scores. DESIGN: Cross-sectional electronic survey. SETTING: Emergency departments (EDs) (n=112) in the UK and Ireland. PARTICIPANTS: Emergency physicians, defined as any registered physician working principally within the ED, responding between June and July 2019. MAIN OUTCOME MEASURE: NFR Scale, an 11-item self-administered questionnaire that assesses how work demands affect intershift recovery. RESULTS: The median NFR Score for all 4247 eligible, consented participants with a valid NFR Score was 70.0 (95% CI: 65.5 to 74.5), with an IQR of 45.5-90.0. A linear regression model indicated statistically significant associations between gender, health conditions, type of ED, clinical grade, access to annual and study leave, and time spent working out-of-hours. Groups including male physicians, consultants, general practitioners (GPs) within the ED, those working in paediatric EDs and those with no long-term health condition or disability had a lower NFR Score. After adjusting for these characteristics, the NFR Score increased by 3.7 (95% CI: 0.3 to 7.1) and 6.43 (95% CI: 2.0 to 10.8) for those with difficulty accessing annual and study leave, respectively. Increased percentage of out-of-hours work increased NFR Score almost linearly: 26%-50% out-of-hours work=5.7 (95% CI: 3.1 to 8.4); 51%-75% out-of-hours work=10.3 (95% CI: 7.6 to 13.0); 76%-100% out-of-hours work=14.5 (95% CI: 11.0 to 17.9). CONCLUSION: Higher NFR scores were observed among emergency physicians than reported in any other profession or population to date. While out-of-hours working is unavoidable, the linear relationship observed suggests that any reduction may result in NFR improvement. Evidence-based strategies to improve well-being such as proportional out-of-hours working and improved access to annual and study leave should be carefully considered and implemented where feasible

Metabolic and immune effects of immunotherapy with proinsulin peptide in human new-onset type 1 diabetes*

Immunotherapy using short immunogenic peptides of disease-related autoantigens restores immune tolerance in preclinical disease models. We studied safety and mechanistic effects of injecting human leukocyte antigen–DR4(DRB1*0401)–restricted immunodominant proinsulin peptide intradermally every 2 or 4 weeks for 6 months in newly diagnosed type 1 diabetes patients. Treatment was well tolerated with no systemic or local hypersensitivity. Placebo subjects showed a significant decline in stimulated C-peptide (measuring insulin reserve) at 3, 6, 9, and 12 months versus baseline, whereas no significant change was seen in the 4-weekly peptide group at these time points or the 2-weekly group at 3, 6, and 9 months. The placebo group’s daily insulin use increased by 50% over 12 months but remained unchanged in the intervention groups. C-peptide retention in treated subjects was associated with proinsulin-stimulated interleukin-10 production, increased FoxP3 expression by regulatory T cells, low baseline levels of activated β cell–specific CD8 T cells, and favorable β cell stress markers (proinsulin/C-peptide ratio). Thus, proinsulin peptide immunotherapy is safe, does not accelerate decline in β cell function, and is associated with antigen-specific and nonspecific immune modulation

Mortality from gastrointestinal congenital anomalies at 264 hospitals in 74 low-income, middle-income, and high-income countries: a multicentre, international, prospective cohort study

Summary Background Congenital anomalies are the fifth leading cause of mortality in children younger than 5 years globally. Many gastrointestinal congenital anomalies are fatal without timely access to neonatal surgical care, but few studies have been done on these conditions in low-income and middle-income countries (LMICs). We compared outcomes of the seven most common gastrointestinal congenital anomalies in low-income, middle-income, and high-income countries globally, and identified factors associated with mortality. Methods We did a multicentre, international prospective cohort study of patients younger than 16 years, presenting to hospital for the first time with oesophageal atresia, congenital diaphragmatic hernia, intestinal atresia, gastroschisis, exomphalos, anorectal malformation, and Hirschsprung’s disease. Recruitment was of consecutive patients for a minimum of 1 month between October, 2018, and April, 2019. We collected data on patient demographics, clinical status, interventions, and outcomes using the REDCap platform. Patients were followed up for 30 days after primary intervention, or 30 days after admission if they did not receive an intervention. The primary outcome was all-cause, in-hospital mortality for all conditions combined and each condition individually, stratified by country income status. We did a complete case analysis. Findings We included 3849 patients with 3975 study conditions (560 with oesophageal atresia, 448 with congenital diaphragmatic hernia, 681 with intestinal atresia, 453 with gastroschisis, 325 with exomphalos, 991 with anorectal malformation, and 517 with Hirschsprung’s disease) from 264 hospitals (89 in high-income countries, 166 in middleincome countries, and nine in low-income countries) in 74 countries. Of the 3849 patients, 2231 (58·0%) were male. Median gestational age at birth was 38 weeks (IQR 36–39) and median bodyweight at presentation was 2·8 kg (2·3–3·3). Mortality among all patients was 37 (39·8%) of 93 in low-income countries, 583 (20·4%) of 2860 in middle-income countries, and 50 (5·6%) of 896 in high-income countries (p<0·0001 between all country income groups). Gastroschisis had the greatest difference in mortality between country income strata (nine [90·0%] of ten in lowincome countries, 97 [31·9%] of 304 in middle-income countries, and two [1·4%] of 139 in high-income countries; p≤0·0001 between all country income groups). Factors significantly associated with higher mortality for all patients combined included country income status (low-income vs high-income countries, risk ratio 2·78 [95% CI 1·88–4·11], p<0·0001; middle-income vs high-income countries, 2·11 [1·59–2·79], p<0·0001), sepsis at presentation (1·20 [1·04–1·40], p=0·016), higher American Society of Anesthesiologists (ASA) score at primary intervention (ASA 4–5 vs ASA 1–2, 1·82 [1·40–2·35], p<0·0001; ASA 3 vs ASA 1–2, 1·58, [1·30–1·92], p<0·0001]), surgical safety checklist not used (1·39 [1·02–1·90], p=0·035), and ventilation or parenteral nutrition unavailable when needed (ventilation 1·96, [1·41–2·71], p=0·0001; parenteral nutrition 1·35, [1·05–1·74], p=0·018). Administration of parenteral nutrition (0·61, [0·47–0·79], p=0·0002) and use of a peripherally inserted central catheter (0·65 [0·50–0·86], p=0·0024) or percutaneous central line (0·69 [0·48–1·00], p=0·049) were associated with lower mortality. Interpretation Unacceptable differences in mortality exist for gastrointestinal congenital anomalies between lowincome, middle-income, and high-income countries. Improving access to quality neonatal surgical care in LMICs will be vital to achieve Sustainable Development Goal 3.2 of ending preventable deaths in neonates and children younger than 5 years by 2030