Agents Explore the Environment Beyond Good Actions to Improve Their Model for Better Decisions

Improving the decision-making capabilities of agents is a key challenge on the road to artificial intelligence. To improve the planning skills needed to make good decisions, MuZero's agent combines prediction by a network model and planning by a tree search using the predictions. MuZero's learning process can fail when predictions are poor but planning requires them. We use this as an impetus to get the agent to explore parts of the decision tree in the environment that it otherwise would not explore. The agent achieves this, first by normal planning to come up with an improved policy. Second, it randomly deviates from this policy at the beginning of each training episode. And third, it switches back to the improved policy at a random time step to experience the rewards from the environment associated with the improved policy, which is the basis for learning the correct value expectation. The simple board game Tic-Tac-Toe is used to illustrate how this approach can improve the agent's decision-making ability. The source code, written entirely in Java, is available at https://github.com/enpasos/muzero.Comment: Submitted to NeurIPS 202

From structural to functional glycomics: core substitutions as molecular switches for shape and lectin affinity of N-glycans

Glycan epitopes of cellular glycoconjugates act as versatile biochemical signals (sugar coding). Here, we test the hypothesis that the common N-glycan modifications by core fucosylation and introduction of the bisecting N-acetylglucosamine moiety have long-range effects with functional consequences. Molecular dynamics simulations indicate a shift in conformational equilibria between linear extension or backfolding of the glycan antennae upon substitution. We also present a new fingerprint-like mode of presentation for this multi-parameter system. In order to delineate definite structure-function relationships, we strategically combined chemoenzymatic synthesis with bioassaying cell binding and the distribution of radioiodinated neoglycoproteins in vivo. Of clinical relevance, tailoring the core region affects serum clearance markedly, e. g., prolonging circulation time for the neoglycoprotein presenting the N-glycan with both substitutions. alpha 2,3-Sialylation is another means toward this end, similarly seen for type II branching in triantennary N-glycans. This discovery signifies that rational glycoengineering along the given lines is an attractive perspective to optimize pharmacokinetic behavior of glycosylated pharmaproteins. Of general importance for the concept of the sugar code, the presented results teach the fundamental lesson that N-glycan core substitutions convey distinct characteristics to the concerned oligosaccharide relevant for cis and trans biorecognition processes. These modifications are thus molecular switches

Hemostasis biomarkers and incident cognitive impairment: the REGARDS study

Essentials Cognitive disorders are increasing and vascular risk factors play a role in this. We performed a nested case control study of hemostasis biomarkers and cognitive impairment (CI). Higher baseline fibrinogen, factor VIII and D-dimer were related to incident CI over 3.5 years. Adjusted for other risk factors, 2+ abnormal markers (but not single ones) led to higher risk. SUMMARY: Background Vascular risk factors are associated with cognitive impairment, a condition that imposes a substantial public health burden. We hypothesized that hemostasis biomarkers related to vascular disease would be associated with the risk of incident cognitive impairment. Methods We performed a nested case-control study including 1082 participants with 3.5 years of follow-up in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study, a longitudinal cohort study of 30 239 black and white Americans aged ≥ 45 years. Participants were free of stroke or cognitive impairment at baseline. Baseline D-dimer, fibrinogen, factor VIII and protein C levels were measured in 495 cases who developed cognitive impairment during follow-up (based on abnormal scores on two or more of three cognitive tests) and 587 controls. Results Unadjusted odds ratios (ORs) for incident cognitive impairment were 1.32 (95% confidence interval [CI] 1.02-1.70) for D-dimer > 0.50 μg mL-1 , 1.83 (95% CI 1.24-2.71) for fibrinogen > 90th percentile, 1.63 (95% CI 1.11-2.38) for FVIII > 90th percentile, and 1.10 (95% CI 0.73-1.65) for protein C < 10th percentile. There were no differences in associations by race or region. Adjustment for demographic, vascular and health behavior risk factors attenuated these associations. However, having at least two elevated biomarkers was associated with incident cognitive impairment, with an adjusted OR of 1.73 (95% CI 1.10-2.69). Conclusion Elevated D-dimer, fibrinogen and FVIII levels were not associated with the occurrence of cognitive impairment after multivariable adjustment; however, having at least two abnormal biomarkers was associated with the occurrence of cognitive impairment, suggesting that the burden of these biomarkers is relevant

N-glycans of human amniotic fluid transferrin stimulate progesterone production in human first trimester trophoblast cells in vitro

Aims: During pregnancy, the placenta produces a variety of steroid hormones and proteins. Several of these substances have been shown to exert immunomodulatory effects. Progesterone is thought to mediate some of these effects by regulating uterine responsiveness. The aim of this study was to clarify the effect of amniotic fluid transferrin and its N-glycans on the release of progesterone by first trimester trophoblast cells in vitro. Methods: Cytotrophoblast cells were prepared from human first trimester placentae by trypsin-DNAse dispersion of villous tissue followed by a percoll gradient centrifugation and depletion of CD45 positive cells by magnetic cell sorting. Trophoblasts were incubated with varying concentrations (50-300 mug/ml) of transferrin from human amniotic fluid and serum as well as with N-glycans obtained from amniotic fluid transferrin. Culture supernatants were assayed for progesterone by enzyme-immunometric methods. Results: The release of progesterone increased in amniotic fluid transferrin- and N-glycan-treated trophoblast cell cultures compared to untreated trophoblast cells. There was no stimulating effect of serum transferrin on the progesterone production of trophoblast cells. Conclusions: The results suggest that amnion-transferrin and especially its N-glycans modulate the endocrine function of trophoblasts in culture by up regulating progesterone secretion

Dry Needling for Myofascial Trigger Point Pain: A Clinical Commentary

Sports and orthopaedic physical therapists have long used a multitude of techniques in order to address pain and dysfunction associated with myofascial trigger points. One technique in particular has recently received overwhelming attention: trigger point dry needling (DN). Despite its efficacy and low risk, questions remain as to its effectiveness, safety, and whether the technique is within the scope of practice of physical therapists. Therefore, the purpose of this clinical commentary is to summarize the current literature related to the associated mechanisms of action of DN, the safety of DN, as well as to discuss relevant scope of practice concerns

Does Targeted Cognitive Training Reduce Educational Disparities in Cognitive Function among Cognitively Normal Older Adults?

Objective The aim of this study was to investigate educational differences in treatment responses to memory, reasoning, and speed of processing cognitive training relative to no-contact control. Methods Secondary analyses of the Advanced Cognitive Training for Independent and Vital Elderly trial were conducted. Two thousand eight hundred older adults were randomized to memory, reasoning, or speed of processing training or no-contact control. A repeated-measures mixed-effects model was used to investigate immediate post-training and 1-year outcomes with sensitivity analyses out to 10 years. Outcomes were as follows: (1) memory composite of Hopkins Verbal Learning Test, Rey Auditory Verbal Learning Test, and Rivermead Behavioral Memory Test; (2) reasoning composite of letter series, letter sets, and word series; and (3) speed of processing measured using three trials of useful field of view and the digit symbol substitution test. Results The effects of reasoning and memory training did not differ by educational attainment. The effect of speed of processing training did. Those with fewer than 12 years of education experienced a 50% greater effect on the useful field of view test compared with those with 16 or more years of education. The training advantage for those with fewer than 12 years of education was maintained to 3 years post-training. Conclusion Older adults with less than a secondary education are at elevated risk of dementia, including Alzheimer's disease. The analyses here indicate that speed of processing training is effective in older adults with low educational attainment