Reinforced Self-Training (ReST) for Language Modeling

Reinforcement learning from human feedback (RLHF) can improve the quality of large language model's (LLM) outputs by aligning them with human preferences. We propose a simple algorithm for aligning LLMs with human preferences inspired by growing batch reinforcement learning (RL), which we call Reinforced Self-Training (ReST). Given an initial LLM policy, ReST produces a dataset by generating samples from the policy, which are then used to improve the LLM policy using offline RL algorithms. ReST is more efficient than typical online RLHF methods because the training dataset is produced offline, which allows data reuse. While ReST is a general approach applicable to all generative learning settings, we focus on its application to machine translation. Our results show that ReST can substantially improve translation quality, as measured by automated metrics and human evaluation on machine translation benchmarks in a compute and sample-efficient manner.Comment: 23 pages, 16 figure

Strategic News Releases in Equity Vesting Months

This is a pre-copyedited, author-produced version of an article accepted for publication in Review of Financial Studies following peer review. The version of record Alex Edmans, Luis Goncalves-Pinto, Moqi Groen-Xu, Yanbo Wang, Strategic News Releases in Equity Vesting Months, The Review of Financial Studies, Volume 31, Issue 11, November 2018, Pages 4099–4141, https://doi.org/10.1093/rfs/hhy070 is available online at: https://doi.org/10.1093/rfs/hhy070

Countermeasures for Preventing and Treating Opioid Overdose

The only medication available currently to prevent and treat opioid overdose (naloxone) was approved by the US Food and Drug Administration (FDA) nearly 50 years ago. Because of its pharmacokinetic and pharmacodynamic properties, naloxone has limited utility under some conditions and would not be effective to counteract mass casualties involving large-scale deployment of weaponized synthetic opioids. To address shortcomings of current medical countermeasures for opioid toxicity, a trans-agency scientific meeting was convened by the US National Institute of Allergy and Infectious Diseases/National Institutes of Health (NIAID/NIH) on August 6 and 7, 2019, to explore emerging alternative approaches for treating opioid overdose in the event of weaponization of synthetic opioids. The meeting was initiated by the Chemical Countermeasures Research Program (CCRP), was organized by NIAID, and was a collaboration with the National Institute on Drug Abuse/NIH (NIDA/NIH), the FDA, the Defense Threat Reduction Agency (DTRA), and the Biomedical Advanced Research and Development Authority (BARDA). This paper provides an overview of several presentations at that meeting that discussed emerging new approaches for treating opioid overdose, including the following: (1) intranasal nalmefene, a competitive, reversible opioid receptor antagonist with a longer duration of action than naloxone; (2) methocinnamox, a novel opioid receptor antagonist; (3) covalent naloxone nanoparticles; (4) serotonin (5-HT)1A receptor agonists; (5) fentanyl-binding cyclodextrin scaffolds; (6) detoxifying biomimetic “nanosponge” decoy receptors; and (7) antibody-based strategies. These approaches could also be applied to treat opioid use disorder.</p

Effect of a Perioperative, Cardiac Output-Guided Hemodynamic Therapy Algorithm on Outcomes Following Major Gastrointestinal Surgery A Randomized Clinical Trial and Systematic Review

Importance: small trials suggest that postoperative outcomes may be improved by the use of cardiac output monitoring to guide administration of intravenous fluid and inotropic drugs as part of a hemodynamic therapy algorithm.Objective: to evaluate the clinical effectiveness of a perioperative, cardiac output–guided hemodynamic therapy algorithm.Design, setting, and participants: OPTIMISE was a pragmatic, multicenter, randomized, observer-blinded trial of 734 high-risk patients aged 50 years or older undergoing major gastrointestinal surgery at 17 acute care hospitals in the United Kingdom. An updated systematic review and meta-analysis were also conducted including randomized trials published from 1966 to February 2014.Interventions: patients were randomly assigned to a cardiac output–guided hemodynamic therapy algorithm for intravenous fluid and inotrope (dopexamine) infusion during and 6 hours following surgery (n=368) or to usual care (n=366).Main outcomes and measures: the primary outcome was a composite of predefined 30-day moderate or major complications and mortality. Secondary outcomes were morbidity on day 7; infection, critical care–free days, and all-cause mortality at 30 days; all-cause mortality at 180 days; and length of hospital stay.Results: baseline patient characteristics, clinical care, and volumes of intravenous fluid were similar between groups. Care was nonadherent to the allocated treatment for less than 10% of patients in each group. The primary outcome occurred in 36.6% of intervention and 43.4% of usual care participants (relative risk [RR], 0.84 [95% CI, 0.71-1.01]; absolute risk reduction, 6.8% [95% CI, ?0.3% to 13.9%]; P?=?.07). There was no significant difference between groups for any secondary outcomes. Five intervention patients (1.4%) experienced cardiovascular serious adverse events within 24 hours compared with none in the usual care group. Findings of the meta-analysis of 38 trials, including data from this study, suggest that the intervention is associated with fewer complications (intervention, 488/1548 [31.5%] vs control, 614/1476 [41.6%]; RR, 0.77 [95% CI, 0.71-0.83]) and a nonsignificant reduction in hospital, 28-day, or 30-day mortality (intervention, 159/3215 deaths [4.9%] vs control, 206/3160 deaths [6.5%]; RR, 0.82 [95% CI, 0.67-1.01]) and mortality at longest follow-up (intervention, 267/3215 deaths [8.3%] vs control, 327/3160 deaths [10.3%]; RR, 0.86 [95% CI, 0.74-1.00]).Conclusions and relevance: in a randomized trial of high-risk patients undergoing major gastrointestinal surgery, use of a cardiac output–guided hemodynamic therapy algorithm compared with usual care did not reduce a composite outcome of complications and 30-day mortality. However, inclusion of these data in an updated meta-analysis indicates that the intervention was associated with a reduction in complication rate

A blood atlas of COVID-19 defines hallmarks of disease severity and specificity.

Treatment of severe COVID-19 is currently limited by clinical heterogeneity and incomplete description of specific immune biomarkers. We present here a comprehensive multi-omic blood atlas for patients with varying COVID-19 severity in an integrated comparison with influenza and sepsis patients versus healthy volunteers. We identify immune signatures and correlates of host response. Hallmarks of disease severity involved cells, their inflammatory mediators and networks, including progenitor cells and specific myeloid and lymphocyte subsets, features of the immune repertoire, acute phase response, metabolism, and coagulation. Persisting immune activation involving AP-1/p38MAPK was a specific feature of COVID-19. The plasma proteome enabled sub-phenotyping into patient clusters, predictive of severity and outcome. Systems-based integrative analyses including tensor and matrix decomposition of all modalities revealed feature groupings linked with severity and specificity compared to influenza and sepsis. Our approach and blood atlas will support future drug development, clinical trial design, and personalized medicine approaches for COVID-19

Gut microbiome diversity, variability, and latent community types compared with shifts in body weight during the freshman year of college in dormitory-housed adolescents.

Significant human gut microbiome changes during adolescence suggest that microbial community evolution occurs throughout important developmental periods including the transition to college, a typical life phase of weight gain. In this observational longitudinal study of 139 college freshmen living in on-campus dormitories, we tracked changes in the gut microbiome via 16S amplicon sequencing and body weight across a single academic year. Participants were grouped by weight change categories of gain (WG), loss (WL), and maintenance (WM). Upon assessment of the community structure, unweighted and weighted UniFrac metrics revealed significant shifts with substantial variation explained by individual effects within weight change categories. Genera that positively contributed to these associations with weight change included Bacteroides, Blautia, and Bifidobacterium in WG participants and Prevotella and Faecalibacterium in WL and WM participants. Moreover, the Prevotella/Bacteroides ratio was significantly different by weight change category, with WL participants displaying an increased ratio. Importantly, these genera did not display co-dominance nor ease of transition between Prevotella- and Bacteroides-dominated states. We further assessed the overall taxonomic variation, noting the increased stability of the WL compared to the WG microbiome. Finally, we found 30 latent community structures within the microbiome with significant associations with waist circumference, sleep, and dietary factors, with alcohol consumption chief among them. Our findings highlight the high level of individual variation and the importance of initial gut microbiome community structure in college students during a period of major lifestyle changes. Further work is needed to confirm these findings and explore mechanistic relationships between gut microbes and weight change in free-living individuals

Does the Low-Field MRI Appearance of Intraosseous STIR Hyperintensity in Equine Cadaver Limbs Change when Subjected to a Freeze-Thaw Process?

Equine advanced imaging research involving racehorse fetlock pathology commonly uses cadaver limbs and a freeze-thaw process. The presence of short tau inversion recovery (STIR) signal intensity in the distal third metacarpal/metatarsal bone is of particular interest and may be clinically relevant in the diagnosis of horses at risk of fracture. However, little is known about the effect of the freeze-thaw process on the MRI appearance of STIR hyperintensity in these bones. This study compares the low-field MRI appearance of the distal third metacarpal/metatarsal bone from cadaver limbs of Thoroughbreds in race training before and after a freeze-thaw protocol. Blinded and unblinded comparisons were made using objective SNR values and subjective grading. Fifteen cadaver limbs with STIR hyperintensity in the distal third metacarpal/metatarsal bone were included. No overall clinical or statistical significance was detected in STIR signal intensity and distribution after freeze-thaw. Three limbs from one horse had individual changes in STIR hyperintensity that were hypothesized to be attributable to ante-mortem haemodynamic abnormalities caused by anaesthesia. These results indicate that the distribution and intensity of STIR hyperintensity in freeze-thawed cadaver fetlocks can be considered representative of the appearance of pathology in the recently euthanized horse. However, care should be taken with horse selection and handling of the cadaver limbs to ensure reliable appearance of STIR signal after freeze-thaw

Does the Low-Field MRI Appearance of Intraosseous STIR Hyperintensity in Equine Cadaver Limbs Change when Subjected to a Freeze-Thaw Process?

Equine advanced imaging research involving racehorse fetlock pathology commonly uses cadaver limbs and a freeze-thaw process. The presence of short tau inversion recovery (STIR) signal intensity in the distal third metacarpal/metatarsal bone is of particular interest and may be clinically relevant in the diagnosis of horses at risk of fracture. However, little is known about the effect of the freeze-thaw process on the MRI appearance of STIR hyperintensity in these bones. This study compares the low-field MRI appearance of the distal third metacarpal/metatarsal bone from cadaver limbs of Thoroughbreds in race training before and after a freeze-thaw protocol. Blinded and unblinded comparisons were made using objective SNR values and subjective grading. Fifteen cadaver limbs with STIR hyperintensity in the distal third metacarpal/metatarsal bone were included. No overall clinical or statistical significance was detected in STIR signal intensity and distribution after freeze-thaw. Three limbs from one horse had individual changes in STIR hyperintensity that were hypothesized to be attributable to ante-mortem haemodynamic abnormalities caused by anaesthesia. These results indicate that the distribution and intensity of STIR hyperintensity in freeze-thawed cadaver fetlocks can be considered representative of the appearance of pathology in the recently euthanized horse. However, care should be taken with horse selection and handling of the cadaver limbs to ensure reliable appearance of STIR signal after freeze-thaw

Gut microbiome diversity, variability, and latent community types compared with shifts in body weight during the freshman year of college in dormitory-housed adolescents

ABSTRACTSignificant human gut microbiome changes during adolescence suggest that microbial community evolution occurs throughout important developmental periods including the transition to college, a typical life phase of weight gain. In this observational longitudinal study of 139 college freshmen living in on-campus dormitories, we tracked changes in the gut microbiome via 16S amplicon sequencing and body weight across a single academic year. Participants were grouped by weight change categories of gain (WG), loss (WL), and maintenance (WM). Upon assessment of the community structure, unweighted and weighted UniFrac metrics revealed significant shifts with substantial variation explained by individual effects within weight change categories. Genera that positively contributed to these associations with weight change included Bacteroides, Blautia, and Bifidobacterium in WG participants and Prevotella and Faecalibacterium in WL and WM participants. Moreover, the Prevotella/Bacteroides ratio was significantly different by weight change category, with WL participants displaying an increased ratio. Importantly, these genera did not display co-dominance nor ease of transition between Prevotella- and Bacteroides-dominated states. We further assessed the overall taxonomic variation, noting the increased stability of the WL compared to the WG microbiome. Finally, we found 30 latent community structures within the microbiome with significant associations with waist circumference, sleep, and dietary factors, with alcohol consumption chief among them. Our findings highlight the high level of individual variation and the importance of initial gut microbiome community structure in college students during a period of major lifestyle changes. Further work is needed to confirm these findings and explore mechanistic relationships between gut microbes and weight change in free-living individuals