Goal-Conditioned Predictive Coding as an Implicit Planner for Offline Reinforcement Learning

Recent work has demonstrated the effectiveness of formulating decision making as a supervised learning problem on offline-collected trajectories. However, the benefits of performing sequence modeling on trajectory data is not yet clear. In this work we investigate if sequence modeling has the capability to condense trajectories into useful representations that can contribute to policy learning. To achieve this, we adopt a two-stage framework that first summarizes trajectories with sequence modeling techniques, and then employs these representations to learn a policy along with a desired goal. This design allows many existing supervised offline RL methods to be considered as specific instances of our framework. Within this framework, we introduce Goal-Conditioned Predicitve Coding (GCPC), an approach that brings powerful trajectory representations and leads to performant policies. We conduct extensive empirical evaluations on AntMaze, FrankaKitchen and Locomotion environments, and observe that sequence modeling has a significant impact on some decision making tasks. In addition, we demonstrate that GCPC learns a goal-conditioned latent representation about the future, which serves as an "implicit planner", and enables competitive performance on all three benchmarks

Federated Learning over a Wireless Network: Distributed User Selection through Random Access

User selection has become crucial for decreasing the communication costs of federated learning (FL) over wireless networks. However, centralized user selection causes additional system complexity. This study proposes a network intrinsic approach of distributed user selection that leverages the radio resource competition mechanism in random access. Taking the carrier sensing multiple access (CSMA) mechanism as an example of random access, we manipulate the contention window (CW) size to prioritize certain users for obtaining radio resources in each round of training. Training data bias is used as a target scenario for FL with user selection. Prioritization is based on the distance between the newly trained local model and the global model of the previous round. To avoid excessive contribution by certain users, a counting mechanism is used to ensure fairness. Simulations with various datasets demonstrate that this method can rapidly achieve convergence similar to that of the centralized user selection approach

Outracing Human Racers with Model-based Planning and Control for Time-trial Racing

Autonomous racing has become a popular sub-topic of autonomous driving in recent years. The goal of autonomous racing research is to develop software to control the vehicle at its limit of handling and achieve human-level racing performance. In this work, we investigate how to approach human expert-level racing performance with model-based planning and control methods using the high-fidelity racing simulator Gran Turismo Sport (GTS). GTS enables a unique opportunity for autonomous racing research, as many recordings of racing from highly skilled human players can served as expert emonstrations. By comparing the performance of the autonomous racing software with human experts, we better understand the performance gap of existing software and explore new methodologies in a principled manner. In particular, we focus on the commonly adopted model-based racing framework, consisting of an offline trajectory planner and an online Model Predictive Control-based (MPC) tracking controller. We thoroughly investigate the design challenges from three perspective, namely vehicle model, planning algorithm, and controller design, and propose novel solutions to improve the baseline approach toward human expert-level performance. We showed that the proposed control framework can achieve top 0.95% lap time among human-expert players in GTS. Furthermore, we conducted comprehensive ablation studies to validate the necessity of proposed modules, and pointed out potential future directions to reach human-best performance.Comment: 16 pages, 13 figures, 3 table

AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?

Can we better anticipate an actor's future actions (e.g. mix eggs) by knowing what commonly happens after his/her current action (e.g. crack eggs)? What if we also know the longer-term goal of the actor (e.g. making egg fried rice)? The long-term action anticipation (LTA) task aims to predict an actor's future behavior from video observations in the form of verb and noun sequences, and it is crucial for human-machine interaction. We propose to formulate the LTA task from two perspectives: a bottom-up approach that predicts the next actions autoregressively by modeling temporal dynamics; and a top-down approach that infers the goal of the actor and plans the needed procedure to accomplish the goal. We hypothesize that large language models (LLMs), which have been pretrained on procedure text data (e.g. recipes, how-tos), have the potential to help LTA from both perspectives. It can help provide the prior knowledge on the possible next actions, and infer the goal given the observed part of a procedure, respectively. To leverage the LLMs, we propose a two-stage framework, AntGPT. It first recognizes the actions already performed in the observed videos and then asks an LLM to predict the future actions via conditioned generation, or to infer the goal and plan the whole procedure by chain-of-thought prompting. Empirical results on the Ego4D LTA v1 and v2 benchmarks, EPIC-Kitchens-55, as well as EGTEA GAZE+ demonstrate the effectiveness of our proposed approach. AntGPT achieves state-of-the-art performance on all above benchmarks, and can successfully infer the goal and thus perform goal-conditioned "counterfactual" prediction via qualitative analysis. Code and model will be released at https://brown-palm.github.io/AntGP

IgA-Targeted Lactobacillus jensenii Modulated Gut Barrier and Microbiota in High-Fat Diet-Fed Mice

IgA-coated Lactobacillus live in the mucous layer of the human or mammalian intestine in close proximity to epithelial cells. They act as potential probiotics for functional food development, but their physiological regulation has not yet been studied. We isolated IgA-targeted (Lactobacillus jensenii IgA21) and lumen lactic acid bacterial strains (Pediococcus acidilactici FS1) from the fecal microbiota of a healthy woman. C57BL/6 mice were fed a normal (CON) or high fat diet (HFD) for 6 weeks and then treated with IgA21 or FS1 for 4 weeks. HFD caused dyslipidemia, mucosal barrier damage, and intestinal microbiota abnormalities. Only IgA21 significantly inhibited dyslipidemia and gut barrier damage. This was related to significant up-regulation of mucin-2, PIgR mRNA expression, and colonic butyrate production (P < 0.05 vs. HFD). Unlike IgA21, FS1 caused a more pronounced gut dybiosis than did HFD, and, in particular, it induced a significant decrease in the Bacteroidales S24-7 group and an increase in Desulfovibrionaceae (P < 0.05 vs. CON). In conclusion, IgA-coated and non-coated lactic acid bacteria of gut have been demonstrated to differentially affect the intestinal barrier and serum lipids. This indicates that IgA-bound bacteria possess the potential to more easily interact with the host gut to regulate homeostasis

Direct synthesis of ultrafine tetragonal BaTiO3 nanoparticles at room temperature

A large quantity of ultrafine tetragonal barium titanate (BaTiO3) nanoparticles is directly synthesized at room temperature. The crystalline form and grain size are checked by both X-ray diffraction and transmission electron microscopy. The results revealed that the perovskite nanoparticles as fine as 7 nm have been synthesized. The phase transition of the as-prepared nanoparticles is investigated by the temperature-dependent Raman spectrum and shows the similar tendency to that of bulk BaTiO3 materials. It is confirmed that the nanoparticles have tetragonal phase at room temperature

The Predictive Potentiality of Salivary Microbiome for the Recurrence of Early Childhood Caries

The aim of this study was to investigate the variation of the salivary microbiota in the recurrence of early childhood caries (ECC), and to explore and verify the potential microbial indicators of ECC recurrence. Saliva samples from kindergarten children were tracked every 6 months for 1 year. Finally, in total 28 children and 84 samples were placed on the analysis phase: 7 children with ECC recurrence made up the ECC-recurrence (ER) group, 6 children without ECC recurrence constituted the non-ECC-recurrence (NER) group, and 15 children who kept ECC-free were set as the ECC-free (EF) group. DNA amplicons of the V3-V4 hypervariable region of the bacterial 16S rDNA were generated and sequencing was performed using Illumina MiSeq PE250 platform. No statistically significant differences of the Shannon indices were found in both cross-sectional and longitudinal comparisons. Furthermore, both principal coordinates analysis (PCoA) and heatmap plots demonstrated that the salivary microbial community structure might have potentiality to predict ECC recurrence at an early phase. The relative abundance of Fusobacterium, Prevotella, Leptotrichia, and Capnocytophaga differed significantly between the ER and NER groups at baseline. The values of area under the curve (AUC) of the four genera and their combined synthesis in the prediction for ECC recurrence were 0.857, 0.833, 0.786, 0.833, and 0.952, respectively. The relative abundance of Fusobacterium, Prevotella, Leptotrichia, and Capnocytophaga and their combination showed satisfactory accuracy in the prediction for ECC recurrence, indicating that salivary microbiome had predictive potentiality for recurrence of this disease. These findings might facilitate more effective strategy to be taken in the management of the recurrence of ECC

Inverse Association between trans Isomeric and Long-Chain Polyunsaturated Fatty Acids in Pregnant Women and Their Newborns: Data from Three European Countries

Background: trans unsaturated fatty acids are thought to interfere with essential fatty acid metabolism. To extend our knowledge of this phenomenon, we investigated the relationship between trans isomeric and long-chain polyunsaturated fatty acids (LCPUFA) in mothers during pregnancy and in their infants at birth. Methods: Fatty acid composition of erythrocyte phosphatidylcholine (PC) and phosphatidylethanolamine (PE) was determined in Spanish (n = 120), German (n = 78) and Hungarian (n = 43) women at the 20th and 30th week of gestation, at delivery and in their newborns. Results: At the 20th week of gestation, the sum of trans fatty acids in PE was significantly (p < 0.01) lower in Hungarian [0.73 (0.51), % wt/wt, median (IQR)] than in Spanish [1.42 (1.36)] and German [1.30 (1.21)] women. Docosahexaenoic acid (DHA) values in PE were significantly (p < 0.01) higher in Hungarian {[}5.65 (2.09)] than in Spanish [4.37 (2.60)] or German [4.39 (3.3.2)] women. The sum of trans fatty acids significantly inversely correlated to DHA in PCs in Spanish (r = -0.37, p < 0.001), German (n = -0.77, p < 0.001) and Hungarian (r = -0.35, p < 0.05) women, and in PEs in Spanish (r = -0.67, p < 0.001) and German (r = -0.71, p < 0.001), but not in Hungarian (r = -0.02) women. Significant inverse correlations were seen between trans fatty acids and DHA in PEs at the 30th week of gestation (n = 241, r = -0.52, p < 0.001), at delivery (n = 241, r = -0.40, p < 0.001) and in cord lipids (n = 218, r = -0.28, p < 0.001). Conclusion: Because humans cannot synthesize trans isomeric fatty acids, the data obtained in the present study support the concept that high maternal trans isomeric fatty acid intake may interfere with the availability of LCPUFA both for the mother and the fetus. Copyright (C) 2011 S. Karger AG, Base

Long decay length of magnon-polarons in BiFeO3/La0.67Sr0.33MnO3 heterostructures

Long-distance magnon transport is highly desired for magnonics. Here, the authors demonstrate a millimetre-long magnon decay length in multiferroic heterostructures, which is attributed to magnon-polarons induced by the magnetoelastic coupling. Magnons can transfer information in metals and insulators without Joule heating, and therefore are promising for low-power computation. The on-chip magnonics however suffers from high losses due to limited magnon decay length. In metallic thin films, it is typically on the tens of micrometre length scale. Here, we demonstrate an ultra-long magnon decay length of up to one millimetre in multiferroic/ferromagnetic BiFeO3(BFO)/La0.67Sr0.33MnO3(LSMO) heterostructures at room temperature. This decay length is attributed to a magnon-phonon hybridization and is more than two orders of magnitude longer than that of bare metallic LSMO. The long-distance modes have high group velocities of 2.5 km s(-1) as detected by time-resolved Brillouin light scattering. Numerical simulations suggest that magnetoelastic coupling via the BFO/LSMO interface hybridizes phonons in BFO with magnons in LSMO to form magnon-polarons. Our results provide a solution to the long-standing issue on magnon decay lengths in metallic magnets and advance the bourgeoning field of hybrid magnonics

Electroacupuncture for relieving itching in atopic eczema: study protocol for a multicenter, randomized, sham-controlled trial

BackgroundAtopic eczema (AE) is a common atopic inflammatory skin disease affecting 2.1–4.9% of the population in different countries. Pruritus, one of the most burdensome symptoms, is often underestimated for the problems it can cause, creating a vicious loop of itching, scratching, and lichenification. Therefore, further research into practical and safe treatments that relieve itchy symptoms and enhance skin protection is key to overcoming AE. Acupuncture, with or without electrical stimulation, is one of the most commonly used therapeutic measures to treat AE. This trial aimed to objectively evaluate the efficacy and safety of the electroacupuncture (EA) antipruritic technique in AE pruritus and obtain high-level clinical evidence for the popularization and application of EA for AE.Methods and analysisThis multicenter, single-blinded, randomized controlled trial is planned to transpire from April 15, 2023, to June 30, 2025. We will recruit 132 participants with AE (44 per group). Participants will be assigned randomly to three equal-sized groups: EA, sham electroacupuncture, and sham acupuncture. Treatment will be administered three times a week during the 2-week intervention phase. The primary outcome measure is the Visual Analog Scale, with a numeric rating scale to evaluate pruritus. Secondary outcome measures include the Eczema Area and Severity Index and Dermatology Life Quality Index. Other outcome measures include physical examination, serum IgE, and safety evaluation. The number, nature, and severity of adverse events will be carefully recorded.Trial registrationClinicalTrials.gov, 22Y11922200. Registered 3 September 2022, https://register.clinicaltrials.gov