DOMAIN: MilDly COnservative Model-BAsed OfflINe Reinforcement Learning

Model-based reinforcement learning (RL), which learns environment model from offline dataset and generates more out-of-distribution model data, has become an effective approach to the problem of distribution shift in offline RL. Due to the gap between the learned and actual environment, conservatism should be incorporated into the algorithm to balance accurate offline data and imprecise model data. The conservatism of current algorithms mostly relies on model uncertainty estimation. However, uncertainty estimation is unreliable and leads to poor performance in certain scenarios, and the previous methods ignore differences between the model data, which brings great conservatism. Therefore, this paper proposes a milDly cOnservative Model-bAsed offlINe RL algorithm (DOMAIN) without estimating model uncertainty to address the above issues. DOMAIN introduces adaptive sampling distribution of model samples, which can adaptively adjust the model data penalty. In this paper, we theoretically demonstrate that the Q value learned by the DOMAIN outside the region is a lower bound of the true Q value, the DOMAIN is less conservative than previous model-based offline RL algorithms and has the guarantee of security policy improvement. The results of extensive experiments show that DOMAIN outperforms prior RL algorithms on the D4RL dataset benchmark, and achieves better performance than other RL algorithms on tasks that require generalization.Comment: 13 pages, 6 figure

CROP: Conservative Reward for Model-based Offline Policy Optimization

Offline reinforcement learning (RL) aims to optimize policy using collected data without online interactions. Model-based approaches are particularly appealing for addressing offline RL challenges due to their capability to mitigate the limitations of offline data through data generation using models. Prior research has demonstrated that introducing conservatism into the model or Q-function during policy optimization can effectively alleviate the prevalent distribution drift problem in offline RL. However, the investigation into the impacts of conservatism in reward estimation is still lacking. This paper proposes a novel model-based offline RL algorithm, Conservative Reward for model-based Offline Policy optimization (CROP), which conservatively estimates the reward in model training. To achieve a conservative reward estimation, CROP simultaneously minimizes the estimation error and the reward of random actions. Theoretical analysis shows that this conservative reward mechanism leads to a conservative policy evaluation and helps mitigate distribution drift. Experiments on D4RL benchmarks showcase that the performance of CROP is comparable to the state-of-the-art baselines. Notably, CROP establishes an innovative connection between offline and online RL, highlighting that offline RL problems can be tackled by adopting online RL techniques to the empirical Markov decision process trained with a conservative reward. The source code is available with https://github.com/G0K0URURI/CROP.git

Salmonella Outer Protein B Suppresses Colitis Development via Protecting Cell From Necroptosis

Salmonella effectors translocated into epithelial cells contribute to the pathogenesis of infection. They mediate epithelial cell invasion and subsequent intracellular replication. However, their functions in vivo have not been well-identified. In this study, we uncovered a role for Salmonella outer protein B (SopB) in modulating necroptosis to facilitate bacteria escape epithelial cell and spread to systemic sites through a Salmonella-induced colitis model. Mice infected with SopB deleted strain ΔsopB displayed increased severity to colitis, reduced mucin expression and increased bacterial translocation. In vitro study, we found there was an increased goblet cell necroptosis following ΔsopB infection. Consistently, mice infected with ΔsopB had a strong upregulation of mixed lineage kinase domain-like (MLKL) phosphorylation. Deletion of MLKL rescued severity of tissue inflammatory, improved mucin2 expression and abolished the increased bacterial translocation in mice infected with ΔsopB. Intriguingly, the expression of sopB in LS174T cells was downregulated. The temporally regulated SopB expression potentially switched the role from epithelial cell invasion to bacterial transmission. Collectively, these results indicated a role for SopB in modulating the onset of necroptosis to increased bacteria pathogenesis and translocated to systemic sites

Atrioventricular block of intraoperative device closure perimembranous ventricular septal defects; a serious complication

<p>Abstract</p> <p>Background</p> <p>Atrioventricular block (AVB) is a well-reported complication after closure of perimembranous ventricular septal defects (VSDs). To report the occurrence of AVB either during or following closure of perimembranous VSDs using a novel "hybrid" method involving a minimal inferior median incision and of intraoperative device closure of the perimembranous VSDs.</p> <p>Methods</p> <p>Between January 2009 and January 2011, patients diagnosed with perimembranous VSDs eligible for intraoperative device closure with a domestic occluder were identified. All patients were assessed by real-time transesophageal echocardiography (TEE) and electrocardiography.</p> <p>Results</p> <p>Of the 97 included patients, 94 were successfully occluded using this approach. Complete AVB occurred in only one case and one case of Mobitz type II AVB was diagnosed intraoperatively. In both patients, the procedure was aborted and the AVBs quickly resolved. Glucocorticosteroids were administered to another two patients who developed Mobitz type II AVB intraoperatively. Those two patients converted to Mobitz type I AVB 3 days and 5 days postsurgically. During the follow-up period (range, 6-24 months), one patient developed complete AVB 1 week following device insertion. Surgical device removal was followed by a rapid and complete recovery of atrioventricular conduction.</p> <p>Conclusions</p> <p>Intraoperative device closure of perimembranous VSDs with a domestic occluder resulted in excellent closure rates; however, AVB is a serious complication that can occur either during or any time after device closure of perimembranous VSDs. The technique described herein may reduce the incidence of perioperative AVB complications. Surgeons are encouraged to closely monitor all patients postsurgically to ensure AVB does not occur in their patients. Additional long-term data to better identify the prevalence and risk factors for AVB in treated patients are needed.</p

Asthma susceptible genes in Chinese population: A meta-analysis

<p>Abstract</p> <p>Background</p> <p>Published data regarding the associations between genetic variants and asthma risk in Chinese population were inconclusive. The aim of this study was to investigate asthma susceptible genes in Chinese population.</p> <p>Methods</p> <p>The authors conducted 18 meta-analyzes for 18 polymorphisms in 13 genes from eighty-two publications.</p> <p>Results</p> <p>Seven polymorphisms were found being associated with risk of asthma, namely: <it>A Disintegrin and Metalloprotease 33 </it>(<it>ADAM33</it>) T1-C/T (odds ratio [OR] = 6.07, 95% confidence interval [CI]: 2.69-13.73), <it>Angiotensin-Converting Enzyme </it>(<it>ACE</it>) D/I (OR = 3.85, 95%CI: 2.49-5.94), <it>High-affinity IgE receptor β chain </it>(<it>FcεRIβ</it>) -6843G/A (OR = 1.49, 95%CI: 1.01-2.22), <it>Interleukin 13</it>(<it>IL-13</it>) -1923C/T (OR = 2.99, 95%CI: 2.12-4.24), <it>IL-13 </it>-2044A/G (OR = 1.49, 95%CI: 1.07-2.08), <it>Regulated upon Activation, Normal T cell Expressed and Secreted </it>(<it>RANTES</it>) -28C/G (OR = 1.64, 95%CI: 1.09-2.46), <it>Tumor Necrosis Factor-α </it>(<it>TNF-α</it>) -308G/A(OR = 1.42, 95%CI: 1.09, 1.85). After subgroup analysis by age, the <it>ACE </it>D/I, <it>β2-Adrenergic Receptor </it>(<it>β2-AR</it>) -79G/C, <it>TNF-α </it>-308G/A, <it>Interleukin 4 receptor</it>(<it>IL-4R</it>) -1902G/A and <it>IL-13 </it>-1923C/T polymorphisms were found significantly associated with asthma risk in Chinese children. In addition, the <it>ACE </it>D/I, <it>FcεRIβ </it>-6843G/A, <it>TNF-α </it>-308G/A, <it>IL-13 </it>-1923C/T and <it>IL-13 </it>-2044A/G polymorphisms were associated with asthma risk in Chinese adults.</p> <p>Conclusion</p> <p><it>ADAM33, FcεRIβ, RANTES, TNF-α, ACE, β2-AR, IL-4R </it>and <it>IL-13 </it>genes could be proposed as asthma susceptible genes in Chinese population. Given the limited number of studies, more data are required to validate these associations.</p

New Dammarane Triterpenoids, Caffruones A–D, from the Cherries of Coffea arabica

Abstract In present study, four new dammarane-type triterpenoids, namely caffruones A–D (1–4), were isolated from the cherries of Coffea arabica. Their structures were elucidated by extensive spectroscopic analysis including 1D, 2D NMR (HSQC, HMBC, 1H–1H COSY, and ROESY), HRMS and IR spectra. This is the first time that tetracyclic triterpenes have been reported in genus Coffea. Graphical Abstrac

Lumbar Scoliosis Induction in Juvenile Dogs by Three-dimensional Modulation of Spinal Growth Using Nickel-Titanium Coil Springs

Background: Current treatments for scoliosis have some defects and complications. To study spinal deformities and test novel scoliosis treatments, many animal models of scoliosis have been developed. These models applied a single load to the spine and could not precisely modulate the spinal growth in different dimensions. In this study, we applied posterior tethering in various directions with the application of nickel-titanium (NT) coil springs in dog's spine to modulate spinal growth in the coronal, sagittal, and transverse planes and create a scoliosis model possess curves that mimic adolescent idiopathic scoliosis (AIS) three dimensionally. Methods: Scoliosis was surgically induced in eight 8-week-old female dogs (weight: 1.95–2.30 kg) using bone screws and NT coil springs. The deformity was induced through the placement of posterior NT coil springs that tethered the spine by bone screw fixation. All dogs were monitored with serial radiographs to document changes in deformities. Results: All experimental animals developed scoliotic curves convex to the left in the lumbar segment. The mean coronal Cobb angle was 18.0° immediately postoperatively and 54.5° at 22 weeks. The mean lordosis increased from 6.2° postoperatively to 35.0° at final follow-up. Apical axial rotation increased from 4.5° postoperatively to 31.2° at 22 weeks. Conclusions: With the application of NT springs in dogs that allowed posterior tethering in various directions, lumbar spinal deformity was achieved in three planes: coronal, sagittal, and transverse planes. Notably, the lumbar spine in surgically treated dogs developed lordoscoliosis with obvious rotation and the curves mimic AIS three dimensionally well. This method allows lumbar scoliosis to develop without deep dissection of muscle and maintains the essential anatomical elements along the spinal curve. Moreover, the spinal growth modulation technique could yield information that would provide a basis for developing novel early-stage treatments for children with scoliosis

Analysis and identification of oxidative stress-ferroptosis related biomarkers in ischemic stroke

Abstract Studies have shown that a series of molecular events caused by oxidative stress is associated with ferroptosis and oxidation after ischemic stroke (IS). Differential analysis was performed to identify differentially expressed mRNA (DEmRNAs) between IS and control groups. Critical module genes were identified using weighted gene co-expression network analysis (WGCNA). DEmRNAs, critical module genes, oxidative stress-related genes (ORGs), and ferroptosis-related genes (FRGs) were crossed to screen for intersection mRNAs. Candidate mRNAs were screened based on the protein–protein interaction (PPI) network and the MCODE plug-in. Biomarkers were identified based on two types of machine learning algorithms, and the intersection was obtained. Functional items and related pathways of the biomarkers were identified using gene set enrichment analysis (GSEA). Finally, single-sample GSEA (ssGSEA) and Wilcoxon tests were used to identify differential immune cells. An miRNA-mRNA-TF network was created. Quantitative real-time polymerase chain reaction (qRT-PCR) was performed to verify the expression levels of biomarkers in the IS and control groups. There were 8287 DE mRNAs between the IS and control groups. The genes in the turquoise module were selected as critical module genes for IS. Thirty intersecting mRNAs were screened for overlaps. Seventeen candidate mRNAs were also identified. Four biomarkers (CDKN1A, GPX4, PRDX1, and PRDX6) were identified using two types of machine-learning algorithms. GSEA results indicated that the biomarkers were associated with steroid biosynthesis. Nine types of immune cells (activated B cells and neutrophils) were markedly different between the IS and control groups. We identified 3747 miRNA-mRNA-TF regulatory pairs in the miRNA-mRNA-TF regulatory network, including hsa-miR-4469-CDKN1A-BACH2 and hsa-miR-188-3p-GPX4-ATF2. CDKN1A, PRDX1, and PRDX6 were upregulated in IS samples compared with control samples. This study suggests that four biomarkers (CDKN1A, GPX4, PRDX1, and PRDX6) are significantly associated with IS. This study provides a new reference for the diagnosis and treatment of IS