LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models

Large language models (LLMs) provide excellent text-generation capabilities, but standard prompting and generation methods generally do not lead to intentional or goal-directed agents and might necessitate considerable prompt tuning. This becomes particularly apparent in multi-turn conversations: even the best current LLMs rarely ask clarifying questions, engage in explicit information gathering, or take actions now that lead to better decisions after multiple turns. Reinforcement learning has the potential to leverage the powerful modeling capabilities of LLMs, as well as their internal representation of textual interactions, to create capable goal-directed language agents. This can enable intentional and temporally extended interactions, such as with humans, through coordinated persuasion and carefully crafted questions, or in goal-directed play through text games to bring about desired final outcomes. However, enabling this requires the community to develop stable and reliable reinforcement learning algorithms that can effectively train LLMs. Developing such algorithms requires tasks that can gauge progress on algorithm design, provide accessible and reproducible evaluations for multi-turn interactions, and cover a range of task properties and challenges in improving reinforcement learning algorithms. Our paper introduces the LMRL-Gym benchmark for evaluating multi-turn RL for LLMs, together with an open-source research framework containing a basic toolkit for getting started on multi-turn RL with offline value-based and policy-based RL methods. Our benchmark consists of 8 different language tasks, which require multiple rounds of language interaction and cover a range of tasks in open-ended dialogue and text games

Advances in the application and utility of subseasonal-to-seasonal predictions

The joint WWRP–WCRP Subseasonal to Seasonal Prediction Project (e.g., Robertson et al. 2014) created a global repository of experimental or operational near-real-time S2S forecasts and reforecasts (hindcasts) from 11 international meteorological institutions, cohosted by ECMWF and CMA (Vitart et al. 2017). These data are publicly accessible by researchers and users (https://apps.ecmwf.int/datasets/data/s2s and http://s2s.cma.cn/index). With the exception of the fourth case study, which uses GloSea5 forecasts (MacLachlan et al. 2015), all case studies use selected S2S forecasts and reforecasts that are available from this repository, providing a consistent basis for S2S forecast skill assessment and evaluation of their utility.The subseasonal-to-seasonal (S2S) predictive time scale, encompassing lead times ranging from 2 weeks to a season, is at the frontier of forecasting science. Forecasts on this time scale provide opportunities for enhanced application-focused capabilities to complement existing weather and climate services and products. There is, however, a “knowledge–value” gap, where a lack of evidence and awareness of the potential socioeconomic benefits of S2S forecasts limits their wider uptake. To address this gap, here we present the first global community effort at summarizing relevant applications of S2S forecasts to guide further decision-making and support the continued development of S2S forecasts and related services. Focusing on 12 sectoral case studies spanning public health, agriculture, water resource management, renewable energy and utilities, and emergency management and response, we draw on recent advancements to explore their application and utility. These case studies mark a significant step forward in moving from potential to actual S2S forecasting applications. We show that by placing user needs at the forefront of S2S forecast development—demonstrating both skill and utility across sectors—this dialogue can be used to help promote and accelerate the awareness, value, and cogeneration of S2S forecasts. We also highlight that while S2S forecasts are increasingly gaining interest among users, incorporating probabilistic S2S forecasts into existing decision-making operations is not trivial. Nevertheless, S2S forecasting represents a significant opportunity to generate useful, usable, and actionable forecast applications for and with users that will increasingly unlock the potential of this forecasting time scale.DD gratefully acknowledges support from the Swiss National Science Foundation through project PP00P2_170523. For case study 1, ACP and WTKH were funded by the U.K. Climate Resilience Programme, supported by the UKRI Strategic Priorities Fund. RWL was funded by NERC Grant NE/P00678/1 and by the BER DOE Office of Science Federal Award DE-SC0020324. TS was funded by NERC Independent Research Fellowship (NE/P018637/1). CMG and DB were funded by the Helmholtz Young Investigator Group “SPREADOUT” Grant VH-NG-1243. Case study 2 was supported by the U.K. Global Challenges Research Fund NE/P021077/1 (GCRF African SWIFT) and the Tertiary Education Trust Fund (TETFUND) of Nigeria TETFund/DR&D/CE/NRF/STI/73/VOL.1. EO thanks Adrian Tomkins of ICTP, Italy, for his contribution. Case study 3 was undertaken as part of the Columbia World Project, ACToday, Columbia University (https://iri.columbia.edu/actoday/). Case study 4 was supported by the ForPAc (Towards Forecast-based Preparedness Action) project within the NERC/FCDO SHEAR Programme NE/P000428/1, NE/P000673/1, and NE/P000568/1. Case study 5 was undertaken as part of the International Research Applications Project, funded by the U.S. National Oceanic and Atmospheric Administration. EO thanks IRAP project colleagues at The University of Arizona, Indian Meteorological Department, Regional Integrated Multi-Hazard Early Warning System for Africa and Asia, and two of Bihar’s State Agricultural Universities for their contributions. For case study 6, CASC thanks Conselho Nacional de Desenvolvimento Científico e Tecnológico Process 305206/2019-2 and Fundação de Amparo à Pesquisa do Estado de São Paulo Process 2015/50687-8 (CLIMAX Project) for their support. For case study 7, DW’s contributions were carried out under contract with the National Aeronautics and Space Administration. Case study 8 was funded by the EU Horizon 2020 Research and Innovation Programme Grant 7767874 (S2S4E). We also acknowledge the Subseasonal-to-Seasonal Project’s Real-Time Pilot Initiative for providing access to real-time forecasts. For case study 9, TIC-LCPE Hydro-04 was funded by the University of Strathclyde’s Low Carbon Power and Energy program. JB was supported by EPSRC Innovation Fellowship EP/R023484/1. We thank Andrew Low and Richard Hearnden from SSE Renewables for their input. Case study 10 was supported by the Earth Systems and Climate Change Hub under the Australian Government’s National Environmental Science Program, and the Decadal Climate Forecasting Project (CSIRO). Case study 11 was funded by the Technologies for Sustainable Built Environments Centre, Reading University, in conjunction with the EPSRC Grant EP/G037787/1 and BT PLC. Case study 12 was funded through the framework service contract for operating the EFAS Computational Center Contract 198702 and the Copernicus Fire Danger Computations Contract 389730 295 in support of the Copernicus Emergency Management Service and Early Warning Systems between the Joint Research Centre and ECMWF.Peer Reviewed"Article signat per 60 autors/es: Christopher J. White, Daniela I. V. Domeisen, Nachiketa Acharya, Elijah A. Adefisan, Michael L. Anderson, Stella Aura, Ahmed A. Balogun, Douglas Bertram, Sonia Bluhm, David J. Brayshaw, Jethro Browell, Dominik Büeler, Andrew Charlton-Perez, Xandre Chourio, Isadora Christel, Caio A. S. Coelho, Michael J. DeFlorio, Luca Delle Monache, Francesca Di Giuseppe, Ana María García-Solórzano, Peter B. Gibson, Lisa Goddard, Carmen González Romero, Richard J. Graham, Robert M. Graham, Christian M. Grams, Alan Halford, W. T. Katty Huang, Kjeld Jensen, Mary Kilavi, Kamoru A. Lawal, Robert W. Lee, David MacLeod, Andrea Manrique-Suñén, Eduardo S. P. R. Martins, Carolyn J. Maxwell, William J. Merryfield, Ángel G. Muñoz, Eniola Olaniyan, George Otieno, John A. Oyedepo, Lluís Palma, Ilias G. Pechlivanidis, Diego Pons, F. Martin Ralph, Dirceu S. Reis Jr., Tomas A. Remenyi, James S. Risbey, Donald J. C. Robertson, Andrew W. Robertson, Stefan Smith, Albert Soret, Ting Sun, Martin C. Todd, Carly R. Tozer, Francisco C. Vasconcelos Jr., Ilaria Vigo, Duane E. Waliser, Fredrik Wetterhall, and Robert G. Wilson"Postprint (author's final draft

Chromobacterium violaceum in Siblings, Brazil

Chromobacterium violaceum, a saprophyte bacterium found commonly in soil and water in tropical and subtropical climates, is a rare cause of severe, often fatal, human disease. We report 1 confirmed and 2 suspected cases of C. violaceum septicemia, with 2 fatalities, in siblings after recreational exposure in northeastern Brazil

Call for emergency action to restore dietary diversity and protect global food systems in times of COVID-19 and beyond: Results from a cross-sectional study in 38 countries

Background: The COVID-19 pandemic has revealed the fragility of the global food system, sending shockwaves across countries' societies and economy. This has presented formidable challenges to sustaining a healthy and resilient lifestyle. The objective of this study is to examine the food consumption patterns and assess diet diversity indicators, primarily focusing on the food consumption score (FCS), among households in 38 countries both before and during the first wave of the COVID-19 pandemic. Methods: A cross-sectional study with 37 207 participants (mean age: 36.70 ± 14.79, with 77 % women) was conducted in 38 countries through an online survey administered between April and June 2020. The study utilized a pre-tested food frequency questionnaire to explore food consumption patterns both before and during the COVID-19 periods. Additionally, the study computed Food Consumption Score (FCS) as a proxy indicator for assessing the dietary diversity of households. Findings: This quantification of global, regional and national dietary diversity across 38 countries showed an increment in the consumption of all food groups but a drop in the intake of vegetables and in the dietary diversity. The household's food consumption scores indicating dietary diversity varied across regions. It decreased in the Middle East and North Africa (MENA) countries, including Lebanon (p < 0.001) and increased in the Gulf Cooperation Council countries including Bahrain (p = 0.003), Egypt (p < 0.001) and United Arab Emirates (p = 0.013). A decline in the household's dietary diversity was observed in Australia (p < 0.001), in South Africa including Uganda (p < 0.001), in Europe including Belgium (p < 0.001), Denmark (p = 0.002), Finland (p < 0.001) and Netherland (p = 0.027) and in South America including Ecuador (p < 0.001), Brazil (p < 0.001), Mexico (p < 0.0001) and Peru (p < 0.001). Middle and older ages [OR = 1.2; 95 % CI = [1.125–1.426] [OR = 2.5; 95 % CI = [1.951–3.064], being a woman [OR = 1.2; 95 % CI = [1.117–1.367], having a high education (p < 0.001), and showing amelioration in food-related behaviors [OR = 1.4; 95 % CI = [1.292–1.709] were all linked to having a higher dietary diversity. Conclusion: The minor to moderate changes in food consumption patterns observed across the 38 countries within relatively short time frames could become lasting, leading to a significant and prolonged reduction in dietary diversity, as demonstrated by our findings.Revisión por pare

Call for emergency action to restore dietary diversity and protect global food systems in times of COVID-19 and beyond: Results from a cross-sectional study in 38 countries

Background: The COVID-19 pandemic has revealed the fragility of the global food system, sending shockwaves across countries\u27 societies and economy. This has presented formidable challenges to sustaining a healthy and resilient lifestyle. The objective of this study is to examine the food consumption patterns and assess diet diversity indicators, primarily focusing on the food consumption score (FCS), among households in 38 countries both before and during the first wave of the COVID-19 pandemic. Methods: A cross-sectional study with 37 207 participants (mean age: 36.70 ± 14.79, with 77 % women) was conducted in 38 countries through an online survey administered between April and June 2020. The study utilized a pre-tested food frequency questionnaire to explore food consumption patterns both before and during the COVID-19 periods. Additionally, the study computed Food Consumption Score (FCS) as a proxy indicator for assessing the dietary diversity of households. Findings: This quantification of global, regional and national dietary diversity across 38 countries showed an increment in the consumption of all food groups but a drop in the intake of vegetables and in the dietary diversity. The household\u27s food consumption scores indicating dietary diversity varied across regions. It decreased in the Middle East and North Africa (MENA) countries, including Lebanon (p \u3c 0.001) and increased in the Gulf Cooperation Council countries including Bahrain (p = 0.003), Egypt (p \u3c 0.001) and United Arab Emirates (p = 0.013). A decline in the household\u27s dietary diversity was observed in Australia (p \u3c 0.001), in South Africa including Uganda (p \u3c 0.001), in Europe including Belgium (p \u3c 0.001), Denmark (p = 0.002), Finland (p \u3c 0.001) and Netherland (p = 0.027) and in South America including Ecuador (p \u3c 0.001), Brazil (p \u3c 0.001), Mexico (p \u3c 0.0001) and Peru (p \u3c 0.001). Middle and older ages [OR = 1.2; 95 % CI = [1.125–1.426] [OR = 2.5; 95 % CI = [1.951–3.064], being a woman [OR = 1.2; 95 % CI = [1.117–1.367], having a high education (p \u3c 0.001), and showing amelioration in food-related behaviors [OR = 1.4; 95 % CI = [1.292–1.709] were all linked to having a higher dietary diversity. Conclusion: The minor to moderate changes in food consumption patterns observed across the 38 countries within relatively short time frames could become lasting, leading to a significant and prolonged reduction in dietary diversity, as demonstrated by our findings

Toxoplasma gondii myelitis in a patient with adult T-cell leukemia-lymphoma Mielite por Toxoplasma gondii em um paciente com leucemia-linfoma de células T do adulto

Adult T cell leukemia-lymphoma (ATL) caused by HTLV-I may be associated with severe immunosupression and several opportunistic infections. Toxoplasmic encephalitis is a common central nervous system opportunistic infection in severely immunosupressed patients, however spinal cord involvement by this parasite is rare. In this paper, we report a case of toxoplasmic myelitis in a patient with ATL.<br>Leucemia de células T do adulto (ATL), causada pelo HTLV-I, pode estar associada com imunossupressão severa e muitas infecções oportunistas. Encefalite por toxoplasmose é uma infecção oportunista do sistema nervoso central em pacientes imunossuprimidos, no entanto o envolvimento da medula espinal por este parasita é raro. Neste artigo, apresentamos um caso de mielite em um paciente com ATL

Advances in the Subseasonal Prediction of Extreme Events: Relevant Case Studies across the Globe

Extreme weather events have devastating impacts on human health, economic activities, ecosystems, and infrastructure. It is therefore crucial to anticipate extremes and their impacts to allow for preparedness and emergency measures. There is indeed potential for probabilistic subseasonal prediction on time scales of several weeks for many extreme events. Here we provide an overview of subseasonal predictability for case studies of some of the most prominent extreme events across the globe using the ECMWF S2S prediction system: heatwaves, cold spells, heavy precipitation events, and tropical and extratropical cyclones. The considered heatwaves exhibit predictability on time scales of 3–4 weeks, while this time scale is 2–3 weeks for cold spells. Precipitation extremes are the least predictable among the considered case studies. ­Tropical cyclones, on the other hand, can exhibit probabilistic predictability on time scales of up to 3 weeks, which in the presented cases was aided by remote precursors such as the Madden–Julian oscillation. For extratropical cyclones, lead times are found to be shorter. These case studies clearly illustrate the potential for event-dependent advance warnings for a wide range of extreme events. The subseasonal predictability of extreme events demonstrated here allows for an extension of warning horizons, provides advance information to impact modelers, and informs communities and stakeholders affected by the impacts of extreme weather events

Advances in the application and utility of subseasonal-to-seasonal predictions

The subseasonal-to-seasonal (S2S) predictive timescale, encompassing lead times ranging from 2 weeks to a season, is at the frontier of forecasting science. Forecasts on this timescale provide opportunities for enhanced application-focused capabilities to complement existing weather and climate services and products. There is, however, a ‘knowledge-value’ gap, where a lack of evidence and awareness of the potential socio-economic benefits of S2S forecasts limits their wider uptake. To address this gap, here we present the first global community effort at summarizing relevant applications of S2S forecasts to guide further decision-making and support the continued development of S2S forecasts and related services. Focusing on 12 sectoral case studies spanning public health, agriculture, water resource management, renewable energy and utilities, and emergency management and response, we draw on recent advancements to explore their application and utility. These case studies mark a significant step forward in moving from potential to actual S2S forecasting applications. We show that by placing user needs at the forefront of S2S forecast development – demonstrating both skill and utility across sectors – this dialogue can be used to help promote and accelerate the awareness, value and co-generation of S2S forecasts. We also highlight that while S2S forecasts are increasingly gaining interest among users, incorporating probabilistic S2S forecasts into existing decision-making operations is not trivial. Nevertheless, S2S forecasting represents a significant opportunity to generate useful, usable and actionable forecast applications for and with users that will increasingly unlock the potential of this forecasting timescale.ISSN:0003-0007ISSN:1520-047

Impact of the COVID-19 pandemic on patients with paediatric cancer in low-income, middle-income and high-income countries: a multicentre, international, observational cohort study

OBJECTIVES: Paediatric cancer is a leading cause of death for children. Children in low-income and middle-income countries (LMICs) were four times more likely to die than children in high-income countries (HICs). This study aimed to test the hypothesis that the COVID-19 pandemic had affected the delivery of healthcare services worldwide, and exacerbated the disparity in paediatric cancer outcomes between LMICs and HICs. DESIGN: A multicentre, international, collaborative cohort study. SETTING: 91 hospitals and cancer centres in 39 countries providing cancer treatment to paediatric patients between March and December 2020. PARTICIPANTS: Patients were included if they were under the age of 18 years, and newly diagnosed with or undergoing active cancer treatment for Acute lymphoblastic leukaemia, non-Hodgkin's lymphoma, Hodgkin lymphoma, Wilms' tumour, sarcoma, retinoblastoma, gliomas, medulloblastomas or neuroblastomas, in keeping with the WHO Global Initiative for Childhood Cancer. MAIN OUTCOME MEASURE: All-cause mortality at 30 days and 90 days. RESULTS: 1660 patients were recruited. 219 children had changes to their treatment due to the pandemic. Patients in LMICs were primarily affected (n=182/219, 83.1%). Relative to patients with paediatric cancer in HICs, patients with paediatric cancer in LMICs had 12.1 (95% CI 2.93 to 50.3) and 7.9 (95% CI 3.2 to 19.7) times the odds of death at 30 days and 90 days, respectively, after presentation during the COVID-19 pandemic (p<0.001). After adjusting for confounders, patients with paediatric cancer in LMICs had 15.6 (95% CI 3.7 to 65.8) times the odds of death at 30 days (p<0.001). CONCLUSIONS: The COVID-19 pandemic has affected paediatric oncology service provision. It has disproportionately affected patients in LMICs, highlighting and compounding existing disparities in healthcare systems globally that need addressing urgently. However, many patients with paediatric cancer continued to receive their normal standard of care. This speaks to the adaptability and resilience of healthcare systems and healthcare workers globally