FACMAC: Factored Multi-Agent Centralised Policy Gradients

We propose FACtored Multi-Agent Centralised policy gradients (FACMAC), a new method for cooperative multi-agent reinforcement learning in both discrete and continuous action spaces. Like MADDPG, a popular multi-agent actor-critic method, our approach uses deep deterministic policy gradients to learn policies. However, FACMAC learns a centralised but factored critic, which combines per-agent utilities into the joint action-value function via a non-linear monotonic function, as in QMIX, a popular multi-agent Q-learning algorithm. However, unlike QMIX, there are no inherent constraints on factoring the critic. We thus also employ a nonmonotonic factorisation and empirically demonstrate that its increased representational capacity allows it to solve some tasks that cannot be solved with monolithic, or monotonically factored critics. In addition, FACMAC uses a centralised policy gradient estimator that optimises over the entire joint action space, rather than optimising over each agent's action space separately as in MADDPG. This allows for more coordinated policy changes and fully reaps the benefits of a centralised critic. We evaluate FACMAC on variants of the multi-agent particle environments, a novel multi-agent MuJoCo benchmark, and a challenging set of StarCraft II micromanagement tasks. Empirical results demonstrate FACMAC's superior performance over MADDPG and other baselines on all three domains

Protein Networks as Logic Functions in Development and Cancer

Many biological and clinical outcomes are based not on single proteins, but on modules of proteins embedded in protein networks. A fundamental question is how the proteins within each module contribute to the overall module activity. Here, we study the modules underlying three representative biological programs related to tissue development, breast cancer metastasis, or progression of brain cancer, respectively. For each case we apply a new method, called Network-Guided Forests, to identify predictive modules together with logic functions which tie the activity of each module to the activity of its component genes. The resulting modules implement a diverse repertoire of decision logic which cannot be captured using the simple approximations suggested in previous work such as gene summation or subtraction. We show that in cancer, certain combinations of oncogenes and tumor suppressors exert competing forces on the system, suggesting that medical genetics should move beyond cataloguing individual cancer genes to cataloguing their combinatorial logic

Recommended sleep duration is associated with higher consumption of fruits and vegetables; cross-sectional and prospective analyses from the UK Women’s Cohort Study

Background: High intakes of fruit and vegetable has been shown to protect against diseases and all-cause mortality however, the associations between sleep and fruit and vegetable consumption are not well characterized. This study aims to explore both cross-sectional and prospective associations between sleep duration and fruit and vegetable intakes in UK women. This is the first study to demonstrate the prospective association between sleep duration and fruit and vegetable consumption. Methods: Cross–sectional and prospective data were obtained from the UK Women’s Cohort Study. Sleep duration was assessed by self-report of average hours slept on weekdays and weekends and diet was assessed by a 4-day food diary at baseline and follow-up (~ 4 years later). Sleep duration was categorized as short (≤6 h/d), recommended (7–9 h/d) and long (≥9 h/d). Regression analyses adjusting for age, socio-economic status, smoking, ethnicity and total energy intake were used and restricted cubic spline models were developed to explore potential non-linear associations between sleep duration and fruit and vegetable intakes. Results: In adjusted cross-sectional analyses, short sleepers had on average 17 g/d (95% CI -30 to-4, p = 0.01) and long sleepers had 25 g/d (95% CI -39 to − 12, p < 0.001) less total fruits and vegetables compared to Recommended Sleepers (RS). In adjusted prospective analyses, short sleepers had on average 85 g/d (95% CI -144 to − 26, p = 0.005) less total fruits and vegetables in comparison to RS. Restricted cubic spline models showed that the cross-sectional (p < 0.001) and prospective (p = 0.001) associations between sleep duration and fruit and vegetable intakes were non-linear with women sleeping 7–9 h/d having the highest intakes. Conclusions: Fruit and vegetable consumption differed between sleep duration categories with UK women sleeping the recommended 7–9 h/day having the highest intake of fruits and vegetables in cross-sectional and prospective analyses. These findings suggest that sleeping the recommended duration is associated with higher consumption of fruits and vegetables. Sleep is an overlooked lifestyle factor in relation to fruit and vegetable consumption and more notice is vital. Further studies are required to clarify the underlying mechanisms for these associations

Guidelines for the use and interpretation of assays for monitoring autophagy (4th edition)1.

In 2008, we published the first set of guidelines for standardizing research in autophagy. Since then, this topic has received increasing attention, and many scientists have entered the field. Our knowledge base and relevant new technologies have also been expanding. Thus, it is important to formulate on a regular basis updated guidelines for monitoring autophagy in different organisms. Despite numerous reviews, there continues to be confusion regarding acceptable methods to evaluate autophagy, especially in multicellular eukaryotes. Here, we present a set of guidelines for investigators to select and interpret methods to examine autophagy and related processes, and for reviewers to provide realistic and reasonable critiques of reports that are focused on these processes. These guidelines are not meant to be a dogmatic set of rules, because the appropriateness of any assay largely depends on the question being asked and the system being used. Moreover, no individual assay is perfect for every situation, calling for the use of multiple techniques to properly monitor autophagy in each experimental setting. Finally, several core components of the autophagy machinery have been implicated in distinct autophagic processes (canonical and noncanonical autophagy), implying that genetic approaches to block autophagy should rely on targeting two or more autophagy-related genes that ideally participate in distinct steps of the pathway. Along similar lines, because multiple proteins involved in autophagy also regulate other cellular pathways including apoptosis, not all of them can be used as a specific marker for bona fide autophagic responses. Here, we critically discuss current methods of assessing autophagy and the information they can, or cannot, provide. Our ultimate goal is to encourage intellectual and technical innovation in the field

Regional response of winter snow cover over the Northern Eurasia to late autumn Arctic sea ice and associated mechanism

Variations of sea ice can exert significant impacts on the atmospheric general circulation, temperature and precipitation, resulting in anomalous land surface conditions such as snow cover. In this study, we explored the possible impacts of the abnormal late autumn Arctic sea ice on winter snow depth over the Eurasia, especially the mechanism responsible for regional response of the snow cover to the sea ice anomalies through both observational analysis based on multi-source snow depth data and numerical sensitivity experiments. Results show that with decreasing sea ice in the Barents and Kara Seas (BKS) region, the snow depth exhibits evident regional responses, which are featured by decreased snow depth in the North Eurasia but increased in central Europe. Further analysis suggests that the Arctic Oscillation (AO) is an important connection between the BKS sea ice and anomalous winter snow depth, which affect both the temperature and the precipitation in winter over the Eurasia. The regional difference mainly comes from significant differences in regional atmospheric general circulation anomalies as well as the various controlling factors affecting the formation of the snowfall. In North Eurasia, water vapor plays a decisive role in the formation of the winter snowfall and further the winter snow depth. Positive (negative) BKS sea ice anomaly generally induces positive (negative) AO pattern, which can enhance (weaken) the zonal wind and increase (decrease) water vapor transportation to the North Eurasia, causing increased (decreased) winter snow depth. In contrast, in Europe, temperature is the dominant factor affecting the formation of the snowfall and the snow depth. Positive (negative) AO pattern, enhances (weakens) the warm moist advection and thus leads to increased (decreased) temperatures in the Europe, consequently resulting in reduced (increased) winter snow depth