What model does MuZero learn?

Model-based reinforcement learning has drawn considerable interest in recent years, given its promise to improve sample efficiency. Moreover, when using deep-learned models, it is potentially possible to learn compact models from complex sensor data. However, the effectiveness of these learned models, particularly their capacity to plan, i.e., to improve the current policy, remains unclear. In this work, we study MuZero, a well-known deep model-based reinforcement learning algorithm, and explore how far it achieves its learning objective of a value-equivalent model and how useful the learned models are for policy improvement. Amongst various other insights, we conclude that the model learned by MuZero cannot effectively generalize to evaluate unseen policies, which limits the extent to which we can additionally improve the current policy by planning with the model

Protection effect of gut microbiota composition and acetate absorption against hypertension-induced damages on the longevity population in Guangxi, China

IntroductionRecent evidence supports a role for the gut microbe-metabolites in longevity. However, the phenomenon of hypertension is more common in the longevity area and whether hypertension is associated with longevity remains unclear. Here, we hypothesize that the levels of gut microbiota, SCFAs, and urine metabolites were different between hypertension elderly and hypertension longevity.MethodsWe recruited 46 elderly volunteers from Donglan County, Guangxi, and 32 were selected and included in the experiment. The subjects with hypertension were divided into two groups according to age, Hypertension Elderly (HTE, aged 70.5 ± 8.59, n = 19) and Hypertension Longevity (HTL, aged 100 ± 5.72, n = 13). The gut microbiota, SCFAs, and urine metabolites were determined by three-generation 16S rRNA full-length sequencing, GC-MS, and 1H-NMR, respectively.ResultsCompared with the HTL group, the HTE group had higher levels of hypertension-related genera Klebsiella and Streptococcus, while having lower levels of the SCFA-producing genera Bacteroides, Faecalibacterium, and Alistipes. Based on LEFse analysis, Klebsiella pneumoniae, Lactobacillus gasseri, Streptococcus salivarius, Ruminococcus, Actinomyces, Rikenellaceae, f_Saccharimonadaceae, Clostridium perfringens, and Bacteroids, Faecalibacterium prausnitzii, Parabacteroides, Alistipes were biomarkers that showed significant differences between the groups. In addition, the microbial pathways associated with K. pneumoniae and E. coli may promote hypertension, while A. muciniphila may play a role in reversing the development of hypertension in long-lived elderly. Metabolomics revealed that HTL contained a lower concentration of fecal acetate and propionate than HTE, while it contained a higher concentration of serum acetate and urine acetate. Furthermore, their immune cells exhibited no significant changes in SCFAs receptors.ConclusionAlthough long-lived elderly have extremely high systolic blood pressure, their unique gut microbiota composition and efficient acetate absorption in the colon may offset the damages caused by hypertension and maintain healthy homeostasis