Semi-Centralised Multi-Agent Reinforcement Learning with Policy-Embedded Training

Centralised training (CT) is the basis for many popular multi-agent reinforcement learning (MARL) methods because it allows agents to quickly learn high-performing policies. However, CT relies on agents learning from one-off observations of other agents' actions at a given state. Because MARL agents explore and update their policies during training, these observations often provide poor predictions about other agents' behaviour and the expected return for a given action. CT methods therefore suffer from high variance and error-prone estimates, harming learning. CT methods also suffer from explosive growth in complexity due to the reliance on global observations, unless strong factorisation restrictions are imposed (e.g., monotonic reward functions for QMIX). We address these challenges with a new semi-centralised MARL framework that performs policy-embedded training and decentralised execution. Our method, policy embedded reinforcement learning algorithm (PERLA), is an enhancement tool for Actor-Critic MARL algorithms that leverages a novel parameter sharing protocol and policy embedding method to maintain estimates that account for other agents' behaviour. Our theory proves PERLA dramatically reduces the variance in value estimates. Unlike various CT methods, PERLA, which seamlessly adopts MARL algorithms, scales easily with the number of agents without the need for restrictive factorisation assumptions. We demonstrate PERLA's superior empirical performance and efficient scaling in benchmark environments including StarCraft Micromanagement II and Multi-agent Mujoc

Integrating multi-influencing factor techniques and fuzzy methods to identify recommendation domains for out-scaling conservation agriculture in China

Climate-smart agriculture (CSA) is a global development strategy aimed to address the interlinked challenges of food security and climate change. Expanding the implementation of conservation agriculture (CA), a vital component of CSA, is essential for enhancing agricultural and food security resilience while sustainably managing arable land. However, the extensive heterogeneity of biophysical and socioeconomic conditions presents significant complexities in promoting CA adoption. Addressing these challenges, this study carried out a comprehensive theoretical investigation of biophysical and socioeconomic factors influencing CA adoption and performance, integrating stakeholder feedback to create a systematic and robust evaluation index system for assessing CA suitability. By integrating multi-influencing factor techniques and fuzzy logic methods, we spatially identified suitable areas for CA implementation in China, providing valuable insights for land use policy. The reliability of the models was verified through a sensitivity analysis using the map removal sensitivity analysis method and the extended Fourier amplitude sensitivity test. The results indicated that 29.78% of the cultivated land was unsuitable or marginally suitable for CA, while 29.30 and 40.92% were determined to be moderately suitable and suitable zones, respectively. Suitable cultivated land was primarily distributed in the northern arid and semi-arid regions, the Loess Plateau, the Huang-Huai-Hai Plain, and the Northeast China Plain. Conversely, unsuitable, and marginally suitable cultivated land was predominantly located in the Qinghai Tibet Plateau, Middle-lower Yangtze Plain, Sichuan Basin and surrounding areas, the Yunnan-Guizhou Plateau, and Southern China. The topographical index, annual mean precipitation, humidity index, and population density were identified as the most significant factors influencing CA suitability. The CA suitability maps generated in this study will guide the development and extension agents targeting CA to suitable locations with a high potential impact, thereby maximizing the likelihood of adoption and minimizing the risk of failure

Accumbal Adenosine A2A Receptors Enhance Cognitive Flexibility by Facilitating Strategy Shifting

The deficits of cognitive flexibility (including attentional set-shifting and reversal learning) concomitant with dysfunction of the striatum are observed in several neuropsychiatric disorders. Rodent and human studies have identified the striatum [particularly the dorsomedial striatum (DMS) and nucleus accumbens (NAc)] as the critical locus for control of cognitive flexibility, but the effective neuromodulator and pharmacological control of cognitive flexibility remains to be determined. The adenosine A2A receptors (A2ARs) are highly enriched in the striatopallidal neurons where they integrate dopamine and glutamate signals to modulate several cognitive behaviors, but their contribution to cognitive flexibility control is unclear. In this study, by coupling an automated operant cognitive flexibility task with striatal subregional knockdown (KD) of the A2AR via the Cre-loxP strategy, we demonstrated that NAc A2AR KD improved cognitive flexibility with enhanced attentional set-shifting and reversal learning by decreasing regressive and perseverative errors, respectively. This facilitation was not attributed to mnemonic process or motor activity as NAc A2AR KD did not affect the visual discrimination, lever-pressing acquisition, and locomotor activity, but was associated with increased attention and motivation as evident by the progressive ratio test (PRT). In contrast to NAc A2ARs, DMS A2ARs KD neither affected visual discrimination nor improved set-shifting nor reversal learning, but promoted the effort-related motivation. Thus, NAc and DMS A2ARs exert dissociable controls of cognitive flexibility with NAc A2ARs KD selectively enhancing cognitive flexibility by facilitating strategy shifting with increased motivation/attention

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Prevalence, associated factors and outcomes of pressure injuries in adult intensive care unit patients: the DecubICUs study

Funder: European Society of Intensive Care Medicine; doi: http://dx.doi.org/10.13039/501100013347Funder: Flemish Society for Critical Care NursesAbstract: Purpose: Intensive care unit (ICU) patients are particularly susceptible to developing pressure injuries. Epidemiologic data is however unavailable. We aimed to provide an international picture of the extent of pressure injuries and factors associated with ICU-acquired pressure injuries in adult ICU patients. Methods: International 1-day point-prevalence study; follow-up for outcome assessment until hospital discharge (maximum 12 weeks). Factors associated with ICU-acquired pressure injury and hospital mortality were assessed by generalised linear mixed-effects regression analysis. Results: Data from 13,254 patients in 1117 ICUs (90 countries) revealed 6747 pressure injuries; 3997 (59.2%) were ICU-acquired. Overall prevalence was 26.6% (95% confidence interval [CI] 25.9–27.3). ICU-acquired prevalence was 16.2% (95% CI 15.6–16.8). Sacrum (37%) and heels (19.5%) were most affected. Factors independently associated with ICU-acquired pressure injuries were older age, male sex, being underweight, emergency surgery, higher Simplified Acute Physiology Score II, Braden score 3 days, comorbidities (chronic obstructive pulmonary disease, immunodeficiency), organ support (renal replacement, mechanical ventilation on ICU admission), and being in a low or lower-middle income-economy. Gradually increasing associations with mortality were identified for increasing severity of pressure injury: stage I (odds ratio [OR] 1.5; 95% CI 1.2–1.8), stage II (OR 1.6; 95% CI 1.4–1.9), and stage III or worse (OR 2.8; 95% CI 2.3–3.3). Conclusion: Pressure injuries are common in adult ICU patients. ICU-acquired pressure injuries are associated with mainly intrinsic factors and mortality. Optimal care standards, increased awareness, appropriate resource allocation, and further research into optimal prevention are pivotal to tackle this important patient safety threat

Correction to: Prevalence, associated factors and outcomes of pressure injuries in adult intensive care unit patients: the DecubICUs study

The original version of this article unfortunately contained a mistake

Prevalence, associated factors and outcomes of pressure injuries in adult intensive care unit patients: the DecubICUs study

Funder: European Society of Intensive Care Medicine; doi: http://dx.doi.org/10.13039/501100013347Funder: Flemish Society for Critical Care NursesAbstract: Purpose: Intensive care unit (ICU) patients are particularly susceptible to developing pressure injuries. Epidemiologic data is however unavailable. We aimed to provide an international picture of the extent of pressure injuries and factors associated with ICU-acquired pressure injuries in adult ICU patients. Methods: International 1-day point-prevalence study; follow-up for outcome assessment until hospital discharge (maximum 12 weeks). Factors associated with ICU-acquired pressure injury and hospital mortality were assessed by generalised linear mixed-effects regression analysis. Results: Data from 13,254 patients in 1117 ICUs (90 countries) revealed 6747 pressure injuries; 3997 (59.2%) were ICU-acquired. Overall prevalence was 26.6% (95% confidence interval [CI] 25.9–27.3). ICU-acquired prevalence was 16.2% (95% CI 15.6–16.8). Sacrum (37%) and heels (19.5%) were most affected. Factors independently associated with ICU-acquired pressure injuries were older age, male sex, being underweight, emergency surgery, higher Simplified Acute Physiology Score II, Braden score 3 days, comorbidities (chronic obstructive pulmonary disease, immunodeficiency), organ support (renal replacement, mechanical ventilation on ICU admission), and being in a low or lower-middle income-economy. Gradually increasing associations with mortality were identified for increasing severity of pressure injury: stage I (odds ratio [OR] 1.5; 95% CI 1.2–1.8), stage II (OR 1.6; 95% CI 1.4–1.9), and stage III or worse (OR 2.8; 95% CI 2.3–3.3). Conclusion: Pressure injuries are common in adult ICU patients. ICU-acquired pressure injuries are associated with mainly intrinsic factors and mortality. Optimal care standards, increased awareness, appropriate resource allocation, and further research into optimal prevention are pivotal to tackle this important patient safety threat