Robust offline reinforcement learning with heavy-tailed rewards

This paper endeavors to augment the robustness of offline reinforcement learning (RL) in scenarios laden with heavy-tailed rewards, a prevalent circumstance in real-world applications. We propose two algorithmic frameworks, ROAM and ROOM, for robust off-policy evaluation and offline policy optimization (OPO), respectively. Central to our frameworks is the strategic incorporation of the median-of-means method with offline RL, enabling straightforward uncertainty estimation for the value function estimator. This not only adheres to the principle of pessimism in OPO but also adeptly manages heavytailed rewards. Theoretical results and extensive experiments demonstrate that our two frameworks outperform existing methods on the logged dataset exhibits heavytailed reward distributions. The implementation of the proposal is available at https: //github.com/Mamba413/ROOM

Robust Offline Policy Evaluation and Optimization with Heavy-Tailed Rewards

This paper endeavors to augment the robustness of offline reinforcement learning (RL) in scenarios laden with heavy-tailed rewards, a prevalent circumstance in real-world applications. We propose two algorithmic frameworks, ROAM and ROOM, for robust off-policy evaluation (OPE) and offline policy optimization (OPO), respectively. Central to our frameworks is the strategic incorporation of the median-of-means method with offline RL, enabling straightforward uncertainty estimation for the value function estimator. This not only adheres to the principle of pessimism in OPO but also adeptly manages heavy-tailed rewards. Theoretical results and extensive experiments demonstrate that our two frameworks outperform existing methods on the logged dataset exhibits heavy-tailed reward distributions

Effects of photoperiod on body mass, thermogenesis and body composition in Eothenomys miletus during cold exposure

Many small mammals respond to seasonal changes in photoperiod by altering body mass and adiposity. These animals may provide valuable models for understanding the regulation of energy balance. In present study, we examined the effect on body mass, rest metabolic rate, food intake and body composition in cold-acclimated (5 °C) in Eothenomys miletus by transferring them from a short (SD, 8h :16h L: D) to long day photoperiod (LD, 16h: 8h L:D). During the first 4 weeks of exposure to SD, E. miletus decreased body mass. After the next 4 weeks of exposure to LD, which the average difference between body masses of LD and SD voles was 4.76 g. This 14.74% increase in body mass reflected significant increases in absolute amounts of body components, including wet carcass mass, dry carcass mass and body fat mass. After correcting body composition and organ morphology data for the differences in body mass, only livers, kidney, and small intestine were enlarged due to photoperiod treatment during cold exposure. E. miletus increased RMR and energy intake exposure to LD, but maintained a stable level to SD after 28 days. Serum leptin levels were positively correlated with body mass, body fat mass, RMR as well as energy intake. All of the results indicated that E. miletus may provide an attractive novel animal model for investigation of the regulation of body mass and energy balance at organism levels. Leptin is potentially involved in the photoperiod induced body mass regulation and thermogenesis in E. miletus during cold exposure

Detection of Candida DNA in peritoneal fluids by PCR assay optimizing the diagnosis and treatment for intra-abdominal candidiasis in high-risk ICU patients: A prospective cohort study

BackgroundIntra-abdominal candidiasis (IAC) is the predominant type of invasive candidiasis with high mortality in critically ill patients. This study aimed to investigate whether the polymerase chain reaction (PCR) assay for detecting Candida DNA in peritoneal fluids (PF) is useful in diagnosing and management of IAC in high-risk patients in intensive care unit (ICU).MethodsA prospective single-center cohort study of surgical patients at high risk for IAC was conducted in the ICU. PF was collected from the abdominal drainage tubes (within 24 h) or by percutaneous puncture. Direct PF smear microscopy, PF culture, blood culture, and serum (1–3)-β-D-glucan were performed in all patients. For Candida PCR assay, the ITS1/ITS4 primers that targeted the ITS1-5.8 s-ITS2 regions were used for PCR, and sequencing analysis was used to identify the pathogen at the species level. IAC was defined according to the 2013 European consensus criteria.ResultsAmong 83 patients at high risk for IAC, the IAC criteria were present in 17 (20.5%). The sensitivity and specificity of the Candida PCR assay were 64.7 and 89.4%, respectively, and the area under the receiver operating characteristic curve was 0.77 (95% CI: 0.63–0.91). In this cohort, the positive predictive value and negative predictive value were 90.8% (95% CI: 80.3–96.2%) and 61.1% (95% CI: 36.1–81.7%), respectively. Diagnostic consistency was moderate (kappa 0.529, p < 0.001) according to the 2013 European consensus criteria.ConclusionDetection of Candida DNA in PF using PCR can be considered an adjunct to existing routine diagnostic tools which may optimize the diagnosis and antifungal treatment of IAC in high-risk patients in the ICU

Modulating effect of adenosine deaminase on function of adenosine A 1 receptors 1

Aim : To study the modulating effect of adenosine deaminase (ADA) on yhe adenosine A 1 receptor (A 1 R) in HEK293 cells stably expressing the human A 1 R. Methods : cDNA was amplified by RT-PCR using total RNA from human embryo brain tissue as the template. The PCR products were subcloned into the plasmid pcDNA3 and cloned into the plasmid pcDNA3.1. The cloned A 1 R cDNA was sequenced and stably expressed in HEK293 cells. The modulating effect of adenosine deaminase on A 1 R was studied by using [ 3 H]DPCPX binding assay and an intracellular calcium assay. Results : HEK293 cells stably expressing human A 1 R were obtained. Saturation studies showed that the K D value and B max value of [ 3 H]DPCPX were 1.6±0.2 nmol/L and 1.819±0.215 nmol/g of protein respectively, in the absence of ecto-ADA respectively, and 1.3±0.2 nmol/L and 1.992±0.130 nmol/g of protein in the presence of ecto-ADA respectively, suggesting that the K D value and B max value of [ 3 H]DPCPX were unaffected by ecto-ADA. In the case of [ 3 H]DPCPX competition curves obtained from intact cells or membranes, A 1 R agonist CCPA/[ 3 H]DPCPX competition curve could be fitted well to a one-site model in the absence of ecto-ADA and a two-site model in the presence of ecto-ADA with a K H value of 0.74 (0.11–4.8) nmol/L (intact cells) or 1.8 (0.25–10) nmol/L (membrane) and a K L value of 0.94 (0.62–1.41) Μmol/L (intact cells) or 0.77 (0.29–0.99) Μmol/L (membrane). The K L value is not significantly different from the EC 50 value of 0.84(0.57–1.23) Μmol/L (intact cells) or 0.84 (0.63–1.12) Μmol/L (membrane) obtained in the absence of ecto-ADA. Similar results were obtained from the CPA/[ 3 H]DPCPX competition curve in the absence or presence of ecto-ADA on intact cells or membranes. Intracellular calcium assay demonstrated that the EC 50 value of CPA were 10 (5–29) nmol/L and 94 (38–229) nmol/L in the presence or absence of ecto-ADA, respectively. Conclusion : A 1 R stably expressed in the HEK293 cells display a low affinity for agonists in the absence of ADA and high and low affinities for agonists in the presence of ADA. The presence of ADA may promote the signaling through the adenosine A 1 receptor in HEK293 cells.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/73693/1/j.1745-7254.2005.00524.x.pd

Prediction of large esophageal varices in cirrhotic patients using classification and regression tree analysis

OBJECTIVES: Recent guidelines recommend that all cirrhotic patients should undergo endoscopic screening for esophageal varices. That identifying cirrhotic patients with esophageal varices by noninvasive predictors would allow for the restriction of the performance of endoscopy to patients with a high risk of having varices. This study aimed to develop a decision model based on classification and regression tree analysis for the prediction of large esophageal varices in cirrhotic patients. METHODS: 309 cirrhotic patients (training sample, 187 patients; test sample 122 patients) were included. Within the training sample, the classification and regression tree analysis was used to identify predictors and prediction model of large esophageal varices. The prediction model was then further evaluated in the test sample and different Child-Pugh classes. RESULTS: The prevalence of large esophageal varices in cirrhotic patients was 50.8%. A tree model that was consisted of spleen width, portal vein diameter and prothrombin time was developed by classification and regression tree analysis achieved a diagnostic accuracy of 84% for prediction of large esophageal varices. When reconstructed into two groups, the rate of varices was 83.2% for high-risk group and 15.2% for low-risk group. Accuracy of the tree model was maintained in the test sample and different Child-Pugh classes. CONCLUSIONS: A decision tree model that consists of spleen width, portal vein diameter and prothrombin time may be useful for prediction of large esophageal varices in cirrhotic patient