Live in the Moment: Learning Dynamics Model Adapted to Evolving Policy

Model-based reinforcement learning (RL) often achieves higher sample efficiency in practice than model-free RL by learning a dynamics model to generate samples for policy learning. Previous works learn a dynamics model that fits under the empirical state-action visitation distribution for all historical policies, i.e., the sample replay buffer. However, in this paper, we observe that fitting the dynamics model under the distribution for \emph{all historical policies} does not necessarily benefit model prediction for the \emph{current policy} since the policy in use is constantly evolving over time. The evolving policy during training will cause state-action visitation distribution shifts. We theoretically analyze how this distribution shift over historical policies affects the model learning and model rollouts. We then propose a novel dynamics model learning method, named \textit{Policy-adapted Dynamics Model Learning (PDML)}. PDML dynamically adjusts the historical policy mixture distribution to ensure the learned model can continually adapt to the state-action visitation distribution of the evolving policy. Experiments on a range of continuous control environments in MuJoCo show that PDML achieves significant improvement in sample efficiency and higher asymptotic performance combined with the state-of-the-art model-based RL methods.Comment: 16 pages, 5 figure

Trojan Horse nanotheranostics with dual transformability and multifunctionality for highly effective cancer treatment.

Nanotheranostics with integrated diagnostic and therapeutic functions show exciting potentials towards precision nanomedicine. However, targeted delivery of nanotheranostics is hindered by several biological barriers. Here, we report the development of a dual size/charge- transformable, Trojan-Horse nanoparticle (pPhD NP) for delivery of ultra-small, full active pharmaceutical ingredients (API) nanotheranostics with integrated dual-modal imaging and trimodal therapeutic functions. pPhD NPs exhibit ideal size and charge for drug transportation. In tumour microenvironment, pPhD NPs responsively transform to full API nanotheranostics with ultra-small size and higher surface charge, which dramatically facilitate the tumour penetration and cell internalisation. pPhD NPs enable visualisation of biodistribution by near-infrared fluorescence imaging, tumour accumulation and therapeutic effect by magnetic resonance imaging. Moreover, the synergistic photothermal-, photodynamic- and chemo-therapies achieve a 100% complete cure rate on both subcutaneous and orthotopic oral cancer models. This nanoplatform with powerful delivery efficiency and versatile theranostic functions shows enormous potentials to improve cancer treatment

Using protection motivation theory to explain the intention to initiate human papillomavirus vaccination among men who have sex with men in China

Human papillomavirus (HPV) infection and related diseases are common among men who have sex with men (MSM). The most effective prevention is HPV vaccination. In China, however, men are not included in the HPV vaccination plan. We investigated the intention to initiate HPV vaccination and associated factors among MSM in China. Methods We surveyed 563 unvaccinated MSM aged 18 or older from six cities in China. Participants completed an electronic questionnaire about demographics, knowledge of and attitude towards HPV and HPV vaccine, intention to initiate HPV vaccination, willingness to recommend HPV vaccine to peers, feeling about government policy about HPV vaccination. We used the structural equation modeling (SEM) to analyze factors associated with HPV vaccine intention. Results The knowledge of HPV and HPV vaccine among participants was low. The mean score of knowledge about HPV and HPV vaccine was only 1.59 (range 0–11). The intention to initiate HPV vaccination within 6 months among participants was moderate (43.3% in total, 18.1% for ‘very high' and 25.2% for ‘above average')

High circulating CD39+ regulatory T cells predict poor survival for sepsis patients

SummaryBackgroundSepsis encompasses two phases, the ‘hyper’-reactive phase and the ‘hypo’-reactive phase. The initial inflammatory stage is quickly counterbalanced by an anti-inflammatory response, which compromises the immune system, leading to immune suppression. Regulatory T cells (Tregs) have been implicated in the pathogenesis of sepsis by inducing immunosuppression; however, the role of CD39+ Tregs in the process of sepsis is uncertain. This study investigated the dynamic levels of CD39+ Tregs and their phenotypic change in sepsis.MethodsFourteen patients with systemic inflammatory response syndrome (SIRS), 42 patients with sepsis, and 14 healthy controls were enrolled. Sequential blood samples were used to analyze the numbers of CD39+ Tregs and their phenotypic changes. Survival at 28 days was used to evaluate the capacity of CD39+ Treg levels to predict mortality in sepsis patients.ResultsSepsis patients displayed a high percentage (3.13%, 1.46%, and 0.35%, respectively) and mean fluorescence intensity (MFI) (59.65, 29.7, and 24.3, respectively) of CD39+ Tregs compared with SIRS patients and healthy subjects. High-level expression of CD39+ Tregs was correlated with the severity of sepsis, which was reflected by the sepsis-related organ failure assessment score (r=0.322 and r=0.31, respectively). In addition, the expression of CD39+ Tregs was associated with survival of sepsis patients (p<0.01). By receiver-operating characteristic (ROC) curve analysis, the percentage and MFI of CD39+ Tregs showed similar sensitivities and specificities to predict mortality (74.2% and 85.1%, and 73.9% and 84.1%, respectively). Using Kaplan–Meier curves to assess the impact of CD39+ Tregs percentage and MFI on overall survival, we found that a high CD39+ Tregs percentage (p<0.001; >4.1%) and MFI (p<0.001; >49.2) were significantly associated with mortality. Phenotypically, CD39+ Tregs from sepsis patients showed high expression of CD38 and PD-1 (p<0.01 and p<0.01 respectively).ConclusionsIncreased expression of CD39+ Tregs was associated with a poor prognosis for sepsis patients, which suggests that CD39+ Treg levels could be used as a biomarker to predict the outcome of sepsis patients

Landscape composition and configuration relatively affect invasive pest and its associator across multiple spatial scales

Landscape structures affect pests, depending on compositional heterogeneity (the number and proportions of different habitats), configurational heterogeneity (spatial arrangement of habitats), and spatial scales. However, there is limited information on the relative effects of compositional and configurational heterogeneity on invasive pests and their associates (species that can benefit from invasive pests), and how they vary across spatial scales. In this study, we assayed the invasive pest Bactrocera dorsalis (Hendel) and its associated fly Drosophila melanogaster in 15 landscapes centered on mango orchards. We calculated landscape composition (forest percentage, mango percentage, and Shannon's diversity) and configuration (edge density) using two methods: spatial distance scales and combined scales. Spatial distance scales included buffer rings with radii of 0.5, 1.0, and 1.5 km, and combined scales referred to cutting or not cutting a smaller ring from larger ones. Our results shown that compositional heterogeneity positively affected B. dorsalis and D. melanogaster due to forest cover percentage, whereas configurational heterogeneity with high edge density negative effect on B. dorsalis. Forest cover had less of an effect on B. dorsalis than configurational heterogeneity, but the opposite effect was observed for D. melanogaster. Importantly, the direction and strength of forest cover and configurational heterogeneity to species did not vary with spatial distance scales or spatial combined scales. Thus, compositional and configurational heterogeneity exhibit differential effects on this invasive pest and its associator, and revealed that the relative effects of landscape structures are consistent across multiple scales. These results provide new insights into landscape effects on interconnected species using a diverse spatial-scale approach

Contextualizing the Revised Patient Perception of Patient-Centeredness (Pppc-R) Scale in Primary Healthcare Settings: a Validity and Reliability Evaluation Study

BACKGROUND: An English version of the Patient Perception of Patient-Centeredness (PPPC) scale was recently revised, and it is necessary to test this instrument in different primary care populations. AIM: This study aimed to assess the validity and reliability of a Chinese version of the PPPC scale. DESIGN: A mixed method was used in this study. The Delphi method was used to collect qualitative and quantitative data to address the content validity of the PPPC scale by calculating the Content Validity Index, Content Validity Ratio, the adjusted Kappa, and the Item Impact Score. Confirmatory factor analysis (CFA) and exploratory factor analysis (EFA) were used to assess the construct validity of the PPPC scale through a cross-sectional survey. The internal consistency was also assessed. SETTING/PARTICIPANTS: In the Delphi consultation, seven experts were consulted through a questionnaire sent by email. The cross-sectional survey interviewed 188 outpatients in Guangzhou city and 108 outpatients in Hohhot City from community health service centers or stations face-to-face. RESULTS: The 21 items in the scale were relevant to their component. The Item-level Content Validity Index for each item was higher than 0.79, and the average Scale-level content validity index was 0.97 in each evaluation round. The initial proposed 4-factor CFA model did not fit adequately. Still, we found a 3-factor solution based on our EFA model and the validation via the CFA model (model fit: [Formula: see text], P \u3c 0.001, RMSEA = 0.044, CFI = 0.981; factor loadings: 0.553 to 0.888). Cronbach\u27s α also indicated good internal consistency reliability: The overall Cronbach\u27s α was 0.922, and the Cronbach\u27s α for each factor was 0.851, 0.872, and 0.717, respectively. CONCLUSIONS: The Chinese version of the PPPC scale provides a valuable tool for evaluating patient-centered medical service quality

Using Integrative Analysis of DNA Methylation and Gene Expression Data in Multiple Tissue Types to Prioritize Candidate Genes for Drug Development in Obesity

Obesity has become a major public health issue which is caused by a combination of genetic and environmental factors. Genome-wide DNA methylation studies have identified that DNA methylation at Cytosine-phosphate-Guanine (CpG) sites are associated with obesity. However, subsequent functional validation of the results from these studies has been challenging given the high number of reported associations. In this study, we applied an integrative analysis approach, aiming to prioritize the drug development candidate genes from many associated CpGs. Association data was collected from previous genome-wide DNA methylation studies and combined using a sample-size-weighted strategy. Gene expression data in adipose tissues and enriched pathways of the affiliated genes were overlapped, to shortlist the associated CpGs. The CpGs with the most overlapping evidence were indicated as the most appropriate CpGs for future studies. Our results revealed that 119 CpGs were associated with obesity (p ≤ 1.03 × 10−7). Of the affiliated genes, SOCS3 was the only gene involved in all enriched pathways and was differentially expressed in both visceral adipose tissue (VAT) and subcutaneous adipose tissue (SAT). In conclusion, our integrative analysis is an effective approach in highlighting the DNA methylation with the highest drug development relevance. SOCS3 may serve as a target for drug development of obesity and its complications

RSL3 Drives Ferroptosis Through GPX4 Inactivation and ROS Production in Colorectal Cancer

Ferroptosis is an iron-dependent, oxidative cell death, and is characterized by iron-dependent accumulation of reactive oxygen species (ROS) within the cell. It has been implicated in various human diseases, including cancer. Recently, ferroptosis, as a non-apoptotic form of cell death, is emerging in specific cancer types; however, its relevance in colorectal cancer (CRC) is unexplored and remains unclear. Here, we showed that ferroptosis inducer RSL3 initiated cell death and ROS accumulation in HCT116, LoVo, and HT29 CRC cells over a 24 h time course. Furthermore, we found that ROS levels and transferrin expression were elevated in CRC cells treated with RSL3 accompanied by a decrease in the expression of glutathione peroxidase 4 (GPX4), indicating an iron-dependent cell death, ferroptosis. Overexpression GPX4 resulted in decreased cell death after RSL3 treatment. Therefore, RSL3 was able to induce ferroptosis on three different CRC cell lines in vitro in a dose- and time-dependent manner, which was due to increased ROS and an increase in the cellular labile iron pool. Moreover, this effect was able to be reversed by overexpression of GPX4. Taken together, our results suggest that the induction of ferroptosis contributed to RSL3-induced cell death in CRC cells and ferroptosis may be a pervasive and dynamic form of cell death for cancer treatment

Glutamic Acid Decarboxylase-Derived Epitopes with Specific Domains Expand CD4+CD25+ Regulatory T Cells

BACKGROUND:CD4(+)CD25(+) regulatory T cell (Treg)-based immunotherapy is considered a promising regimen for controlling the progression of autoimmune diabetes. In this study, we tested the hypothesis that the therapeutic effects of Tregs in response to the antigenic epitope stimulation depend on the structural properties of the epitopes used. METHODOLOGY/PRINCIPAL FINDINGS:Splenic lymphocytes from nonobese diabetic (NOD) mice were stimulated with different glutamic acid decarboxylase (GAD)-derived epitopes for 7-10 days and the frequency and function of Tregs was analyzed. We found that, although all expanded Tregs showed suppressive functions in vitro, only p524 (GAD524-538)-expanded CD4(+)CD25(+) T cells inhibited diabetes development in the co-transfer models, while p509 (GAD509-528)- or p530 (GAD530-543)-expanded CD4(+)CD25(+) T cells had no such effects. Using computer-guided molecular modeling and docking methods, the differences in structural characteristics of these epitopes and the interaction mode (including binding energy and identified domains in the epitopes) between the above-mentioned epitopes and MHC class II I-A(g7) were analyzed. The theoretical results showed that the epitope p524, which induced protective Tregs, possessed negative surface-electrostatic potential and bound two chains of MHC class II I-A(g7), while the epitopes p509 and p530 which had no such ability exhibited positive surface-electrostatic potential and bound one chain of I-A(g7). Furthermore, p524 bound to I-A(g7) more stably than p509 and p530. Of importance, we hypothesized and subsequently confirmed experimentally that the epitope (GAD570-585, p570), which displayed similar characteristics to p524, was a protective epitope by showing that p570-expanded CD4(+)CD25(+) T cells suppressed the onset of diabetes in NOD mice. CONCLUSIONS/SIGNIFICANCE:These data suggest that molecular modeling-based structural analysis of epitopes may be an instrumental tool for prediction of protective epitopes to expand functional Tregs