A Game-Theoretic Framework for Managing Risk in Multi-Agent Systems

In order for agents in multi-agent systems (MAS) to be safe, they need to take into account the risks posed by the actions of other agents. However, the dominant paradigm in game theory (GT) assumes that agents are not affected by risk from other agents and only strive to maximise their expected utility. For example, in hybrid human-AI driving systems, it is necessary to limit large deviations in reward resulting from car crashes. Although there are equilibrium concepts in game theory that take into account risk aversion, they either assume that agents are risk-neutral with respect to the uncertainty caused by the actions of other agents, or they are not guaranteed to exist. We introduce a new GT-based Risk-Averse Equilibrium (RAE) that always produces a solution that minimises the potential variance in reward accounting for the strategy of other agents. Theoretically and empirically, we show RAE shares many properties with a Nash Equilibrium (NE), establishing convergence properties and generalising to risk-dominant NE in certain cases. To tackle large-scale problems, we extend RAE to the PSRO multi-agent reinforcement learning (MARL) framework. We empirically demonstrate the minimum reward variance benefits of RAE in matrix games with high-risk outcomes. Results on MARL experiments show RAE generalises to risk-dominant NE in a trust dilemma game and that it reduces instances of crashing by 7x in an autonomous driving setting versus the best performing baseline

Settling the Variance of Multi-Agent Policy Gradients

Policy gradient (PG) methods are popular reinforcement learning (RL) methods where a baseline is often applied to reduce the variance of gradient estimates. In multi-agent RL (MARL), although the PG theorem can be naturally extended, the effectiveness of multi-agent PG (MAPG) methods degrades as the variance of gradient estimates increases rapidly with the number of agents. In this paper, we offer a rigorous analysis of MAPG methods by, firstly, quantifying the contributions of the number of agents and agents’ explorations to the variance of MAPG estimators. Based on this analysis, we derive the optimal baseline (OB) that achieves the minimal variance. In comparison to the OB, we measure the excess variance of existing MARL algorithms such as vanilla MAPG and COMA. Considering using deep neural networks, we also propose a surrogate version of OB, which can be seamlessly plugged into any existing PG methods in MARL. On benchmarks of Multi-Agent MuJoCo and StarCraft challenges, our OB technique effectively stabilises training and improves the performance of multi-agent PPO and COMA algorithms by a significant margin. Code is released at https://github.com/morning9393/Optimal-Baseline-for-Multi-agent-Policy-Gradients

Promotion of rapid testing for HIV in primary care (RHIVA2): a cluster-randomised controlled trial

Department of Health, NHS City and Hackney; (ClinicalTrials.gov: ISRCTN63473710)

Validation of a Novel Multivariate Method of Defining HIV-Associated Cognitive Impairment

Background. The optimum method of defining cognitive impairment in virally suppressed people living with HIV is unknown. We evaluated the relationships between cognitive impairment, including using a novel multivariate method (NMM), patientreported outcome measures (PROMs), and neuroimaging markers of brain structure across 3 cohorts.Methods. Differences in the prevalence of cognitive impairment, PROMs, and neuroimaging data from the COBRA, CHARTER, and POPPY cohorts (total n = 908) were determined between HIV-positive participants with and without cognitive impairment defined using the HIV-associated neurocognitive disorders (HAND), global deficit score (GDS), and NMM criteria.Results. The prevalence of cognitive impairment varied by up to 27% between methods used to define impairment (eg, 48% for HAND vs 21% for NMM in the CHARTER study). Associations between objective cognitive impairment and subjective cognitive complaints generally were weak. Physical and mental health summary scores (SF-36) were lowest for NMM-defined impairment (P<.05). There were no differences in brain volumes or cortical thickness between participants with and without cognitive impairment defined using the HAND and GDS measures. In contrast, those identified with cognitive impairment by the NMM had reduced mean cortical thickness in both hemispheres (P<.05), as well as smaller brain volumes (P<.01). The associations with measures of white matter microstructure and brain-predicted age generally were weaker.Conclusion. Different methods of defining cognitive impairment identify different people with varying symptomatology and measures of brain injury. Overall, NMM-defined impairment was associated with most neuroimaging abnormalities and poorer selfreported health status. This may be due to the statistical advantage of using a multivariate approach

Level of agreement between frequently used cardiovascular risk calculators in people living with HIV

Objectives The aim of the study was to describe agreement between the QRISK2, Framingham and Data Collection on Adverse Events of Anti‐HIV Drugs (D:A:D) cardiovascular disease (CVD) risk calculators in a large UK study of people living with HIV (PLWH). Methods PLWH enrolled in the Pharmacokinetic and Clinical Observations in People over Fifty (POPPY) study without a prior CVD event were included in this study. QRISK2, Framingham CVD and the full and reduced D:A:D CVD scores were calculated; participants were stratified into ‘low’ ( 20%) categories for each. Agreement between scores was assessed using weighted kappas and Bland–Altman plots. Results The 730 included participants were predominantly male (636; 87.1%) and of white ethnicity (645; 88.5%), with a median age of 53 [interquartile range (IQR) 49–59] years. The median calculated 10‐year CVD risk was 11.9% (IQR 6.8–18.4%), 8.9% (IQR 4.6–15.0%), 8.5% (IQR 4.8–14.6%) and 6.9% (IQR 4.1–11.1%) when using the Framingham, QRISK2, and full and reduced D:A:D scores, respectively. Agreement between the different scores was generally moderate, with the highest level of agreement being between the Framingham and QRISK2 scores (weighted kappa = 0.65) but with most other kappa coefficients in the 0.50–0.60 range. Conclusions Estimates of predicted 10‐year CVD risk obtained with commonly used CVD risk prediction tools demonstrate, in general, only moderate agreement among PLWH in the UK. While further validation with clinical endpoints is required, our findings suggest that care should be taken when interpreting any score alone

Global patient outcomes after elective surgery: prospective cohort study in 27 low-, middle- and high-income countries.

BACKGROUND: As global initiatives increase patient access to surgical treatments, there remains a need to understand the adverse effects of surgery and define appropriate levels of perioperative care. METHODS: We designed a prospective international 7-day cohort study of outcomes following elective adult inpatient surgery in 27 countries. The primary outcome was in-hospital complications. Secondary outcomes were death following a complication (failure to rescue) and death in hospital. Process measures were admission to critical care immediately after surgery or to treat a complication and duration of hospital stay. A single definition of critical care was used for all countries. RESULTS: A total of 474 hospitals in 19 high-, 7 middle- and 1 low-income country were included in the primary analysis. Data included 44 814 patients with a median hospital stay of 4 (range 2-7) days. A total of 7508 patients (16.8%) developed one or more postoperative complication and 207 died (0.5%). The overall mortality among patients who developed complications was 2.8%. Mortality following complications ranged from 2.4% for pulmonary embolism to 43.9% for cardiac arrest. A total of 4360 (9.7%) patients were admitted to a critical care unit as routine immediately after surgery, of whom 2198 (50.4%) developed a complication, with 105 (2.4%) deaths. A total of 1233 patients (16.4%) were admitted to a critical care unit to treat complications, with 119 (9.7%) deaths. Despite lower baseline risk, outcomes were similar in low- and middle-income compared with high-income countries. CONCLUSIONS: Poor patient outcomes are common after inpatient surgery. Global initiatives to increase access to surgical treatments should also address the need for safe perioperative care. STUDY REGISTRATION: ISRCTN5181700

Clinical outcomes and response to treatment of patients receiving topical treatments for pyoderma gangrenosum: a prospective cohort study

Background: pyoderma gangrenosum (PG) is an uncommon dermatosis with a limited evidence base for treatment. Objective: to estimate the effectiveness of topical therapies in the treatment of PG. Methods: prospective cohort study of UK secondary care patients with a clinical diagnosis of PG suitable for topical treatment (recruited July 2009 to June 2012). Participants received topical therapy following normal clinical practice (mainly Class I-III topical corticosteroids, tacrolimus 0.03% or 0.1%). Primary outcome: speed of healing at 6 weeks. Secondary outcomes: proportion healed by 6 months; time to healing; global assessment; inflammation; pain; quality-of-life; treatment failure and recurrence. Results: Sixty-six patients (22 to 85 years) were enrolled. Clobetasol propionate 0.05% was the most commonly prescribed therapy. Overall, 28/66 (43.8%) of ulcers healed by 6 months. Median time-to-healing was 145 days (95% CI: 96 days, ∞). Initial ulcer size was a significant predictor of time-to-healing (hazard ratio 0.94 (0.88;80 1.00); p = 0.043). Four patients (15%) had a recurrence. Limitations: No randomised comparator Conclusion: Topical therapy is potentially an effective first-line treatment for PG that avoids possible side effects associated with systemic therapy. It remains unclear whether more severe disease will respond adequately to topical therapy alone

Dimethyl fumarate in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial

Dimethyl fumarate (DMF) inhibits inflammasome-mediated inflammation and has been proposed as a treatment for patients hospitalised with COVID-19. This randomised, controlled, open-label platform trial (Randomised Evaluation of COVID-19 Therapy [RECOVERY]), is assessing multiple treatments in patients hospitalised for COVID-19 (NCT04381936, ISRCTN50189673). In this assessment of DMF performed at 27 UK hospitals, adults were randomly allocated (1:1) to either usual standard of care alone or usual standard of care plus DMF. The primary outcome was clinical status on day 5 measured on a seven-point ordinal scale. Secondary outcomes were time to sustained improvement in clinical status, time to discharge, day 5 peripheral blood oxygenation, day 5 C-reactive protein, and improvement in day 10 clinical status. Between 2 March 2021 and 18 November 2021, 713 patients were enroled in the DMF evaluation, of whom 356 were randomly allocated to receive usual care plus DMF, and 357 to usual care alone. 95% of patients received corticosteroids as part of routine care. There was no evidence of a beneficial effect of DMF on clinical status at day 5 (common odds ratio of unfavourable outcome 1.12; 95% CI 0.86-1.47; p = 0.40). There was no significant effect of DMF on any secondary outcome