Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework

With the increasing integration of frontier large language models (LLMs) into society and the economy, decisions related to their training, deployment, and use have far-reaching implications. These decisions should not be left solely in the hands of frontier LLM developers. LLM users, civil society and policymakers need trustworthy sources of information to steer such decisions for the better. Involving outside actors in the evaluation of these systems - what we term 'external scrutiny' - via red-teaming, auditing, and external researcher access, offers a solution. Though there are encouraging signs of increasing external scrutiny of frontier LLMs, its success is not assured. In this paper, we survey six requirements for effective external scrutiny of frontier AI systems and organize them under the ASPIRE framework: Access, Searching attitude, Proportionality to the risks, Independence, Resources, and Expertise. We then illustrate how external scrutiny might function throughout the AI lifecycle and offer recommendations to policymakers.Comment: Accepted to Workshop on Socially Responsible Language Modelling Research (SoLaR) at the 2023 Conference on Neural Information Processing Systems (NeurIPS 2023

Orally administered oxygen nanobubbles enhance tumor response to sonodynamic therapy

Suspensions of oxygen-filled bubbles are under active investigation as potential means of relieving tissue hypoxia. Intravenous administration of large quantities of bubbles is, however, undesirable. Previous work by the authors has demonstrated that tumor oxygen levels can be increased following oral administration of phospholipid stabilized oxygen nanobubbles. The aim of this study was to determine whether this would enhance the efficacy of sonodynamic therapy (SDT), which is known to be inhibited in hypoxic tissue. Experiments were conducted in a murine model of pancreatic cancer. Animals were treated with SDT (intratumoural injection of 1 mM Rose Bengal followed by exposure to 1 MHz ultrasound, 0.1 kHz pulse repetition frequency, 30% duty cycle, 3.5 W cm−2 for 3.5 minutes) either with or without a prior gavage of oxygen bubbles. A statistically significant reduction in the rate of tumor growth was observed in the groups receiving oxygen nanobubbles either 5 or 20 minutes before SDT. Separate measurements of tumor oxygen using a fiber optic probe and expression of hypoxia inducible factor (HIF)1α following tumor excision, confirmed the change in tumor oxygen levels. These findings offer a potentially promising new approach to relieving tissue hypoxia in order to facilitate cancer therapy

Oxygen-enhanced MRI and radiotherapy in patients with oropharyngeal squamous cell carcinoma

Background and purpose: This study aimed to assess the role of T1 mapping and oxygen-enhanced MRI in patients undergoing radical dose radiotherapy for HPV positive oropharyngeal cancer, which has not yet been examined in an OE-MRI study. Materials and methods: Variable Flip Angle T1 maps were acquired on a 3T MRI scanner while patients (n = 12) breathed air and/or 100 % oxygen, before and after fraction 10 of the planned 30 fractions of chemoradiotherapy (‘visit 1’ and ‘visit 2’, respectively). The analysis aimed to assess to what extent (1) native R1 relates to patient outcome; (2) OE-MRI response relates to patient outcome; (3) changes in mean R1 before and after radiotherapy related to clinical outcome in patients with oropharyngeal squamous cell carcinoma. Results: Due to the radiotherapy being largely successful, the sample sizes of non-responder groups were small, and therefore it was not possible to properly assess the predictive nature of OE-MRI. The tumour R1 increased in some patients while decreasing in others, in a pattern that was overall consistent with the underlying OE-MRI theory and previously reported tumour OE-MRI responses. In addition, we discuss some practical challenges faced when integrating this technique into a clinical trial, with the aim that sharing this is helpful to researchers planning to use OE-MRI in future clinical studies. Conclusion: Altogether, these results suggest that further clinical OE-MRI studies to assess hypoxia and radiotherapy response are worth pursuing, and that there is important work to be done to improve the robustness of the OE-MRI technique in human applications in order for it to be useful as a widespread clinical technique

A simplified empirical model to estimate oxygen relaxivity at different magnetic fields

The change in longitudinal relaxation rate (R1) produced by oxygen has been used as a means of inferring oxygenation levels in magnetic resonance imaging in numerous applications. The relationship between oxygen partial pressure (pO2) and R1 is linear and reproducible, and the slope represents the relaxivity of oxygen (r1Ox) in that material. However, there is considerable variability in the values of r1Ox reported, and they have been shown to vary by field strength and temperature. Therefore, we have compiled 28 reported empirical values of the relaxivity of oxygen as a resource for researchers. Furthermore, we provide an empirical model for estimating the relaxivity of oxygen in water, saline, plasma, and vitreous fluids, accounting for magnetic field strength and temperature. The model agrees well (R2 = 0.93) with the data gathered from the literature for fields ranging from 0.011 to 8.45 T and temperatures of 21-40 °C. This provides a useful resource for researchers seeking to quantify pO2 in simple fluids in their studies, such as water and saline phantoms, or bodily fluids such as vitreous fluids, cerebrospinal fluids, and amniotic fluids

Modeling the effect of hyperoxia on the spin–lattice relaxation rate R1 of tissues

Purpose Inducing hyperoxia in tissues is common practice in several areas of research, including oxygen-enhanced MRI (OE-MRI), which attempts to use the resulting signal changes to detect regions of tumor hypoxia or pulmonary disease. The linear relationship between PO2 and R1 has been reproduced in phantom solutions and body fluids such as vitreous fluid; however, in tissue and blood experiments, factors such as changes in deoxyhemoglobin levels can also affect the ΔR1. Theory and Methods This manuscript proposes a three-compartment model for estimating the hyperoxia-induced changes in R1 of tissues depending on B0, SO2, blood volume, hematocrit, oxygen extraction fraction, and changes in blood and tissue PO2. The model contains two blood compartments (arterial and venous) and a tissue compartment. This model has been designed to be easy for researchers to tailor to their tissue of interest by substituting their preferred model for tissue oxygen diffusion and consumption. A specific application of the model is demonstrated by calculating the resulting ΔR1 expected in healthy, hypoxic and necrotic tumor tissues. In addition, the effect of sex-based hematocrit differences on ΔR1 is assessed. Results The ΔR1 values predicted by the model are consistent with reported literature OE-MRI results: with larger positive changes in the vascular periphery than hypoxic and necrotic regions. The observed sex-based differences in ΔR1 agree with findings by Kindvall et al. suggesting that differences in hematocrit levels may sometimes be a confounding factor in ΔR1. Conclusion This model can be used to estimate the expected tissue ΔR1 in oxygen-enhanced MRI experiments

2023 expert survey of AGI safety and governance practices

To support the creation of best practices in artificial general intelligence (AGI) safety and governance, we survey a group of leading experts from AGI labs, academia, and civil society

Determination of oxygen relaxivity in oxygen nanobubbles at 3 and 7 Tesla

Objective Oxygen-loaded nanobubbles have shown potential for reducing tumour hypoxia and improving treatment outcomes, however, it remains difficult to noninvasively measure the changes in partial pressure of oxygen (PO2) in vivo. The linear relationship between PO2 and longitudinal relaxation rate (R1) has been used to noninvasively infer PO2 in vitreous and cerebrospinal fluid, and therefore, this experiment aimed to investigate whether R1 is a suitable measurement to study oxygen delivery from such oxygen carriers. Methods T1 mapping was used to measure R1 in phantoms containing nanobubbles with varied PO2 to measure the relaxivity of oxygen (r1Ox) in the phantoms at 7 and 3 T. These measurements were used to estimate the limit of detection (LOD) in two experimental settings: preclinical 7 T and clinical 3 T MRI. Results The r1Ox in the nanobubble solution was 0.00057 and 0.000235 s−1/mmHg, corresponding to a LOD of 111 and 103 mmHg with 95% confidence at 7 and 3 T, respectively. Conclusion This suggests that T1 mapping could provide a noninvasive method of measuring a > 100 mmHg oxygen delivery from therapeutic nanobubbles

Challenges for machine learning in clinical translation of big data imaging studies

Combining deep learning image analysis methods and large-scale imaging datasets offers many opportunities to neuroscience imaging and epidemiology. However, despite these opportunities and the success of deep learning when applied to a range of neuroimaging tasks and domains, significant barriers continue to limit the impact of large-scale datasets and analysis tools. Here, we examine the main challenges and the approaches that have been explored to overcome them. We focus on issues relating to data availability, interpretability, evaluation, and logistical challenges and discuss the problems that still need to be tackled to enable the success of “big data” deep learning approaches beyond research

Oxygen-enhanced MRI MOLLI T1 mapping during chemoradiotherapy in anal squamous cell carcinoma

Background and Purpose: Oxygen-enhanced magnetic resonance imaging (MRI) and T1-mapping was used to explore its effectiveness as a prognostic imaging biomarker for chemoradiotherapy outcome in anal squamous cell carcinoma. Materials and Methods: T2-weighted, T1 mapping, and oxygen-enhanced T1 maps were acquired before and after 8–10 fractions of chemoradiotherapy and examined whether the oxygen-enhanced MRI response relates to clinical outcome. Patient response to treatment was assessed 3 months following completion of chemoradiotherapy. A mean T1 was extracted from manually segmented tumour regions of interest and a paired two-tailed t-test was used to compare changes across the patient population. Regions of subcutaneous fat and muscle tissue were examined as control ROIs. Results: There was a significant increase in T1 of the tumour ROIs across patients following the 8–10 fractions of chemoradiotherapy (paired t-test, p < 0.001, n = 7). At baseline, prior to receiving chemoradiotherapy, there were no significant changes in T1 across patients from breathing oxygen (n = 9). In the post-chemoRT scans (8–10 fractions), there was a significant decrease in T1 of the tumour ROIs across patients when breathing 100% oxygen (paired t-test, p < 0.001, n = 8). Out of the 12 patients from which we successfully acquired a visit 1 T1-map, only 1 patient did not respond to treatment, therefore, we cannot correlate these results with clinical outcome. Conclusions: These clinical data demonstrate feasibility and potential for T1-mapping and oxygen enhanced T1-mapping to indicate perfusion or treatment response in tumours of this nature. These data show promise for future work with a larger cohort containing more non-responders, which would allow us to relate these measurements to clinical outcome