Active teacher selection for reinforcement learning from human feedback

Reinforcement learning from human feedback (RLHF) enables machine learning systems to learn objectives from human feedback. A core limitation of these systems is their assumption that all feedback comes from a single human teacher, despite querying a range of distinct teachers. We propose the Hidden Utility Bandit (HUB) framework to model differences in teacher rationality, expertise, and costliness, formalizing the problem of learning from multiple teachers. We develop a variety of solution algorithms and apply them to two real-world domains: paper recommendation systems and COVID-19 vaccine testing. We find that the Active Teacher Selection (ATS) algorithm outperforms baseline algorithms by actively selecting when and which teacher to query. The HUB framework and ATS algorithm demonstrate the importance of leveraging differences between teachers to learn accurate reward models, facilitating future research on active teacher selection for robust reward modeling

Adapting a Kidney Exchange Algorithm to Align with Human Values

The efficient and fair allocation of limited resources is a classical problem in economics and computer science. In kidney exchanges, a central market maker allocates living kidney donors to patients in need of an organ. Patients and donors in kidney exchanges are prioritized using ad-hoc weights decided on by committee and then fed into an allocation algorithm that determines who gets what--and who does not. In this paper, we provide an end-to-end methodology for estimating weights of individual participant profiles in a kidney exchange. We first elicit from human subjects a list of patient attributes they consider acceptable for the purpose of prioritizing patients (e.g., medical characteristics, lifestyle choices, and so on). Then, we ask subjects comparison queries between patient profiles and estimate weights in a principled way from their responses. We show how to use these weights in kidney exchange market clearing algorithms. We then evaluate the impact of the weights in simulations and find that the precise numerical values of the weights we computed matter little, other than the ordering of profiles that they imply. However, compared to not prioritizing patients at all, there is a significant effect, with certain classes of patients being (de)prioritized based on the human-elicited value judgments

Transcriptional profiling of maturing tomato (Solanum lycopersicum L.) microspores reveals the involvement of heat shock proteins, ROS scavengers, hormones, and sugars in the heat stress response

Above-optimal temperatures reduce yield in tomato largely because of the high heat stress (HS) sensitivity of the developing pollen grains. The high temperature response, especially at this most HS-sensitive stage of the plant, is poorly understood. To obtain an overview of molecular mechanisms underlying the HS response (HSR) of microspores, a detailed transcriptomic analysis of heat-stressed maturing tomato microspores was carried out using a combination of Affymetrix Tomato Genome Array and cDNA-amplified fragment length polymorphism (AFLP) techniques. The results were corroborated by reverse transcription-PCR (RT-PCR) and immunoblot analyses. The data obtained reveal the involvement of specific members of the small heat shock protein (HSP) gene family, HSP70 and HSP90, in addition to the HS transcription factors A2 (HSFA2) and HSFA3, as well as factors other than the classical HS-responsive genes. The results also indicate HS regulation of reactive oxygen species (ROS) scavengers, sugars, plant hormones, and regulatory genes that were previously implicated in other types of stress. The use of cDNA-AFLP enabled the detection of genes representing pollen-specific functions that are missing from the tomato Affymetrix chip, such as those involved in vesicle-mediated transport and a pollen-specific, calcium-dependent protein kinase (CDPK2). For several genes, including LeHSFA2, LeHSP17.4-CII, as well as homologues of LeHSP90 and AtVAMP725, higher basal expression levels were detected in microspores of cv. Hazera 3042 (a heat-tolerant cultivar) compared with microspores of cv. Hazera 3017 (a heat-sensitive cultivar), marking these genes as candidates for taking part in microspore thermotolerance. This work provides a comprehensive analysis of the molecular events underlying the HSR of maturing microspores of a crop plant, tomato

Risk of acute myeloid leukemia and myelodysplastic syndrome among older women receiving anthracycline-based adjuvant chemotherapy for breast cancer on Modern Cooperative Group Trials (Alliance A151511)

We examined acute myeloid leukemia (AML) and myelodysplastic syndrome (MDS) events among 9679 women treated for breast cancer on four adjuvant Alliance for Clinical Trials in Oncology trials with >90 months of follow-up in order to better characterize the risk for AML/MDS in older patients receiving anthracyclines

A national survey of services for the prevention and management of falls in the UK

Background: The National Health Service (NHS) was tasked in 2001 with developing service provision to prevent falls in older people. We carried out a national survey to provide a description of health and social care funded UK fallers services, and to benchmark progress against current practice guidelines. Methods: Cascade approach to sampling, followed by telephone survey with senior member of the fall service. Characteristics of the service were assessed using an internationally agreed taxonomy. Reported service provision was compared against benchmarks set by the National Institute for Health and Clinical Excellence (NICE). Results: We identified 303 clinics across the UK. 231 (76%) were willing to participate. The majority of services were based in acute or community hospitals, with only a few in primary care or emergency departments. Access to services was, in the majority of cases, by health professional referral. Most services undertook a multi-factorial assessment. The content and quality of these assessments varied substantially. Services varied extensively in the way that interventions were delivered, and particular concern is raised about interventions for vision, home hazard modification, medication review and bone health. Conclusion: The most common type of service provision was a multi-factorial assessment and intervention. There were a wide range of service models, but for a substantial number of services, delivery appears to fall below recommended NICE guidance

Translating research into practice: Protocol for a community-engaged, stepped wedge randomized trial to reduce disparities in breast cancer treatment through a regional patient navigation collaborative

BACKGROUND: Racial and socioeconomic disparities in breast cancer mortality persist. In Boston, MA, Black, Non-Hispanic women and Medicaid-insured individuals are 2-3 times more likely to have delays in treatment compared to White or privately insured women. While evidence-based care coordination strategies for reducing delays exist, they are not systematically implemented across healthcare settings. METHODS: Translating Research Into Practice (TRIP) utilizes community engaged research methods to address breast cancer care delivery disparities. Four Massachusetts Clinical and Translational Science Institute (CTSI) hubs collaborated with the Boston Breast Cancer Equity Coalition (The Coalition) to implement an evidence-based care coordination intervention for Boston residents at risk for delays in breast cancer care. The Coalition used a community-driven process to define the problem of care delivery disparities, identify the target population, and develop a rigorous pragmatic approach. We chose a cluster-randomized, stepped-wedge hybrid type I effectiveness-implementation study design. The intervention implements three evidence-based strategies: patient navigation services, a shared patient registry for use across academic medical centers, and a web-based social determinants of health platform to identify and address barriers to care. Primary clinical outcomes include time to first treatment and receipt of guideline-concordant treatment, which are captured through electronic health records abstraction. We will use mixed methods to collect the secondary implementation outcomes of acceptability, adoption/penetration, fidelity, sustainability and cost. CONCLUSION: TRIP utilizes an innovative community-driven research strategy, focused on interdisciplinary collaborations, to design and implement a translational science study that aims to more efficiently integrate proven health services interventions into clinical practice

Magnetic Response in Quantized Spin Hall Phase of Correlated Electrons

We investigate the magnetic response in the quantized spin Hall (SH) phase of layered-honeycomb lattice system with intrinsic spin-orbit coupling lambda_SO and on-site Hubbard U. The response is characterized by a parameter g= 4 U a^2 d / 3, where a and d are the lattice constant and interlayer distance, respectively. When g< (sigma_{xy}^{s2} mu)^{-1}, where sigma_{xy}^{s} is the quantized spin Hall conductivity and mu is the magnetic permeability, the magnetic field inside the sample oscillates spatially. The oscillation vanishes in the non-interacting limit U -> 0. When g > (sigma_{xy}^{s2} mu)^{-1}, the system shows perfect diamagnetism, i.e., the Meissner effect occurs. We find that superlattice structure with large lattice constant is favorable to see these phenomena. We also point out that, as a result of Zeeman coupling, the topologically-protected helical edge states shows weak diamagnetism which is independent of the parameter g.Comment: 7 pages, the final version will be published in J. Phys. Soc. Jp