Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learning

Offline multi-agent reinforcement learning is challenging due to the coupling effect of both distribution shift issue common in offline setting and the high dimension issue common in multi-agent setting, making the action out-of-distribution (OOD) and value overestimation phenomenon excessively severe. Tomitigate this problem, we propose a novel multi-agent offline RL algorithm, named CounterFactual Conservative Q-Learning (CFCQL) to conduct conservative value estimation. Rather than regarding all the agents as a high dimensional single one and directly applying single agent methods to it, CFCQL calculates conservative regularization for each agent separately in a counterfactual way and then linearly combines them to realize an overall conservative value estimation. We prove that it still enjoys the underestimation property and the performance guarantee as those single agent conservative methods do, but the induced regularization and safe policy improvement bound are independent of the agent number, which is therefore theoretically superior to the direct treatment referred to above, especially when the agent number is large. We further conduct experiments on four environments including both discrete and continuous action settings on both existing and our man-made datasets, demonstrating that CFCQL outperforms existing methods on most datasets and even with a remarkable margin on some of them.Comment: 37th Conference on Neural Information Processing Systems (NeurIPS 2023

A Complement Receptor C5a Antagonist Regulates Epithelial to Mesenchymal Transition and Crystallin Expression After Lens Cataract Surgery in Mice

Purpose: To evaluate the effects of complement employing a mouse model for secondary cataract. Methods: The role of complement receptor C5a (CD88) was evaluated after cataract surgery in mice. An antagonist specific to C5a receptor was administered intraperitoneally to mice. Epithelial to mesenchymal transition (EMT) was evaluated by alpha-smooth muscle actin (α-SMA) staining and proliferation by bromodeoxyuridine (5-bromo-2\u27- deoxyuridine, BrdU) incorporation. Gene expression patterns was examined by microarray analysis and quantitative polymerase chain reaction (QPCR). Results: We found that administration of a C5aR antagonist in C57BL/6J mice decreases EMT, as evidenced by α-SMA expression, and cell proliferation. Gene expression by microarray analysis reveals discreet steps of gene regulation in the two major stages that of EMT and lens fiber differentiation in vivo. A hallmark of the microarray analysis is that the antagonist seems to be a novel stage-specific regulator of crystallin genes. At week two, which is marked by lens fiber differentiation genes encoding 12 crystallins and 3 lens-specific structural proteins were severely down-regulated. Conclusions: These results suggest a possible therapeutic role of an antagonist to C5aR in preventing secondary cataracts after surgery. Also these results suggest that crystallin gene expression can be regulated by pro-inflammatory events in the eye

Combined glucose-dependent insulinotropic polypeptide receptor and glucagon-like peptide-1 receptor agonism attenuates atherosclerosis severity in APOE*3-Leiden.CETP mice

Background and aims: Combined agonism of the glucose-dependent insulinotropic polypeptide receptor (GIPR) and the glucagon-like peptide-1 receptor (GLP1R) is superior to single GLP1R agonism in terms of glycemic control and lowering body weight in individuals with obesity and with or without type 2 diabetes mellitus. As both GIPR and GLP1R signaling have also been implicated in improving inflammatory responses and lipid handling, two crucial players in atherosclerosis development, here we aimed to investigate the effects of combined GIPR/GLP1R agonism in APOE*3-Leiden.CETP mice, a well-established mouse model for human-like lipoprotein metabolism and atherosclerosis development. Methods: Female APOE*3-Leiden.CETP mice were fed a Western-type diet (containing 16% fat and 0.15% cholesterol) to induce dyslipidemia, and received subcutaneous injections with either vehicle, a GIPR agonist (GIPFA-085), a GLP1R agonist (GLP-140) or both agonists. In the aortic root area, atherosclerosis development was assessed. Results: Combined GIPR/GLP1R agonism attenuated the development of severe atherosclerotic lesions, while single treatments only showed non-significant improvements. Mechanistically, combined GIPR/GLP1R agonism decreased markers of systemic low-grade inflammation. In addition, combined GIPR/GLP1R agonism markedly lowered plasma triglyceride (TG) levels as explained by reduced hepatic very-low-density lipoprotein (VLDL)-TG production as well as increased TG-derived fatty acid uptake by brown and white adipose tissue which was coupled to enhanced hepatic uptake of core VLDL remnants. Conclusions: Combined GIPR/GLP1R agonism attenuates atherosclerosis severity by diminishing inflammation and increasing VLDL turnover. We anticipate that combined GIPR/GLP1R agonism is a promising strategy to lower cardiometabolic risk in humans.</p

Synthesis of Novel Amino Acids and Use of Peptides & Peptidomimetics Containing Unnatural Amino Acids for the Development of Selective Melanocortin Peptide Antagonists and for the Study of Melanocortin Receptor Signaling

Unnatural amino acids are indispensible tools, not only for the elucidation of molecular mechanisms during the study of the complicated biological system, but also for the development of novel peptide and protein drugs with better efficacy and lower toxicity. Beta-substituted gamma,delta-unsaturated amino acids have been shown to be an important type of novel amino acid because of the terminal double bond which can be converted to many other functionalities. The methodology for the synthesis of syn-beta-substituted gamma,delta-unsaturated amino acids has been developed. However, there is no satisfactory general method for the synthesis of anti-beta-substituted gamma,delta-unsaturated amino acids. Therefore, a general methodology was developed by using the Eschenmoser-Claisen rearrangement for the synthesis of both racemic and optically active anti-beta-substituted gamma,delta-unsaturated amino acids. This rearrangement is highly diastereoselective and good asymmetric induction was obtained with a relatively small C2-symmetric chiral auxiliary (2R,5R)-dimethylpyrrolidine. In an effort to design peptide antagonists that are selective for human melanocortin 4 receptor, highly constrained trans and cis 4-guanidinium proline derivatives were synthesized and incorporated in various melanotropin analogues designed to mimic the endogenous hMC1,4R selective antagonist hASIP (Agouti Signaling Protein) central loop. Biological assays show that some of these analogues are highly selective for hMC1R and/or hMC4R with partial agonist or antagonist activities due to a new beta-turn structure induced by the presence of the constrained amino acids. According to molecular modeling studies, the lowest energy conformations of these selective analogues resemble the NMR solution structure of the hASIP central loop. Therefore, a new template was developed for the rational design of novel selective melanotropin analogues that may have therapeutic potential. To further understand the molecular mechanisms of hMC4R signaling upon agonist activation, an hMC4R selective nonpeptide agonist THIQ and its fluorescent dye labeled derivatives were needed to compare to peptide agonist MTII with regard to receptor phosphorylation, internalization, etc. An improved synthetic method was developed for the efficient synthesis of THIQ. A method for the synthesis of TRITC labeled THIQ derivatives was also developed

An inflammation-related nomogram for predicting the survival of patients with non-small cell lung cancer after pulmonary lobectomy

Abstract Background Emerging inflammatory response biomarkers are developed to predict the survival of patients with cancer, the aim of our study is to establish an inflammation-related nomogram based on the classical predictive biomarkers to predict the survivals of patients with non-small cell lung cancer (NSCLC). Methods Nine hundred and fifty-two NSCLC patients with lung cancer surgery performed were enrolled into this study. The cutoffs of inflammatory response biomarkers were determined by Receiver operating curve (ROC). Univariate and multivariate analysis were conducted to select independent prognostic factors to develop the nomogram. Results The median follow-up time was 40.0 months (range, 1 to 92 months). The neutrophil to lymphocyte ratio (cut-off: 3.10, HR:1.648, P = 0.045) was selected to establish the nomogram which could predict the 5-year OS probability. The C-index of nomogram was 0.72 and the 5-year OS calibration curve displayed an optimal agreement between the actual observed outcomes and the predictive results. Conclusions Neutrophil to lymphocyte ratio was shown to be a valuable biomarker for predicting survival of patients with NSCLC. The addition of neutrophil to lymphocyte ratio could improve the accuracy and predictability of the nomogram in order to provide reference for clinicians to assess patient outcomes

Evaluation of Preoperative Hematologic Markers as Prognostic Factors and Establishment of Novel Risk Stratification in Resected pN0 Non-Small-Cell Lung Cancer

<div><p>Background</p><p>The aims of this study were to investigate whether the preoperative hematologic markers, the neutrophil-lymphocyte ratio (NLR) or the platelet-lymphocyte ratio (PLR) were prognostic indicators and to develop a novel risk stratification model in pN0 non-small-cell lung cancer (NSCLC).</p><p>Methods</p><p>We performed a retrospective analysis of 400 consecutive pN0 NSCLC patients. Prognostic values were evaluated by Cox proportional hazard model analyses and patients were stratified according to relative risks for patients’ survival.</p><p>Results</p><p>During the follow-up, 117 patients had cancer recurrence, and 86 patients died. In univariate analysis, age, gender, smoke status and tumor size as well as WBC, NEU, LYM, PLR and NLR were significantly associated with patients’ prognosis. In multivariate analysis, age, tumor size and NLR were independent predictors for patients’ overall survival (P = 0.024, 0.001, and 0.002 respectively). PLR didn’t associated with patients’ survival in multivariate analysis. Patients were stratified into 3 risk groups and the differences among the groups were significant according to disease free survival and overall survival (P = 0.000 and 0.000 respectively).</p><p>Conclusions</p><p>We confirmed that NLR other than PLR was an independent prognostic factor. Combination of NLR, age and tumor size could stratify pN0 NSCLC patients into 3 risk groups and enabled us to develop a novel risk stratification model.</p></div

Kaplan-Meier estimates according to low, intermediate and high risk groups on DFS (a) and OS (b).

<p>Kaplan-Meier estimates according to low, intermediate and high risk groups on DFS (a) and OS (b).</p

Clinical characteristic of all 400 lung cancer patients.

<p>Clinical characteristic of all 400 lung cancer patients.</p

Inheritance patterns of the transcriptome in hybrid chickens and their parents revealed by expression analysis

Although many phenotypic traits of chickens have been well documented, the genetic patterns of gene expression levels in chickens remain to be determined. In the present study, we crossed two chicken breeds, White Leghorn (WL) and Cornish (Cor), which have been selected for egg and meat production, respectively, for a few hundred years. We evaluated transcriptome abundance in the brain, muscle, and liver from the day-old progenies of pure-bred WL and Cor, and the hybrids of these two breeds, by RNA-Seq in order to determine the inheritance patterns of gene expression. Comparison among expression levels in the different groups revealed that most of the genes showed conserved expression patterns in all three examined tissues and that brain had the highest number of conserved genes, which indicates that conserved genes are predominantly important compared to others. On the basis of allelic expression analysis, in addition to the conserved genes, we identified the extensive presence of additive, dominant (Cor dominant and WL dominant), over-dominant, and under-dominant genes in all three tissues in hybrids. Our study is the first to provide an overview of inheritance patterns of the transcriptome in layers and broilers, and we also provide insights into the genetics of chickens at the gene expression level