Max-Min Off-Policy Actor-Critic Method Focusing on Worst-Case Robustness to Model Misspecification

In the field of reinforcement learning, because of the high cost and risk of policy training in the real world, policies are trained in a simulation environment and transferred to the corresponding real-world environment. However, the simulation environment does not perfectly mimic the real-world environment, lead to model misspecification. Multiple studies report significant deterioration of policy performance in a real-world environment. In this study, we focus on scenarios involving a simulation environment with uncertainty parameters and the set of their possible values, called the uncertainty parameter set. The aim is to optimize the worst-case performance on the uncertainty parameter set to guarantee the performance in the corresponding real-world environment. To obtain a policy for the optimization, we propose an off-policy actor-critic approach called the Max-Min Twin Delayed Deep Deterministic Policy Gradient algorithm (M2TD3), which solves a max-min optimization problem using a simultaneous gradient ascent descent approach. Experiments in multi-joint dynamics with contact (MuJoCo) environments show that the proposed method exhibited a worst-case performance superior to several baseline approaches.Comment: Neural Information Processing Systems 2022 (NeurIPS '22

Effect of intensive granulocyte and monocyte adsorptive apheresis in patients with ulcerative colitis positive for cytomegalovirus.

[Background and aim]Cytomegalovirus (CMV) exacerbates ulcerative colitis (UC) refractory to immunosuppressive therapies. The conditions under which CMV reactivation occurs in patients with UC, however, is unclear. In addition, the diagnostic and treatment strategies for UC positive for CMV have not been established. Granulocyte and monocyte adsorptive apheresis (GMAA) is natural biological therapy for UC in which the granulocytes/macrophages producing inflammatory cytokines are removed. We investigated the rate of colonic CMV reactivation and the efficacy of GMAA in active UC patients positive for CMV without concomitant corticosteroid (CS) therapy. [Methods]Fifty-one active UC patients without concomitant CS therapy were enrolled. Colonic CMV reactivation was examined by real-time polymerase chain reaction (PCR) using biopsy specimen and/or histological examination. All patients were treated with intensive GMAA (twice per week). Rates of clinical remission and mucosal healing were compared between UC patients positive and negative for CMV. [Results]Of 51 patients, 15 (29.4%) were diagnosed as CMV positive. The clinical remission rates following intensive GMAA did not differ between UC patients positive and negative for CMV (73.3% vs 69.4%, p = 0.781). Proportion of patients achieving mucosal healing was also similar between these two groups. CMV-DNA became negative in all UC patients positive for CMV who achieved clinical remission 1 week after completion of intensive GMAA. [Conclusions]Intestinal inflammation might trigger CMV reactivation in a subpopulation of active UC patients without CS treatment. GMAA could be a promising option for active UC positive for CMV

A case of bilateral diffuse uveal melanocytic proliferation with secondary angle closure caused by ciliary body thickening

Purpose: To describe a case of bilateral diffuse uveal melanocytic proliferation (BDUMP) with secondary angle closure caused by ciliary body thickening accompanied by intraocular pressure (IOP) elevation after mydriasis. Observations: A 55-year-old woman with a history of ovarian cancer had blurred vision in both eyes. Fundus examination revealed multiple patchy lesions in both eyes and a nevus-like elevated lesion in the right eye. Anterior segment optical coherence tomography (AS-OCT) and ultrasound biomicroscopy (UBM) demonstrated angle closure resulting from ciliary body thickening. After mydriasis, the IOP was elevated in both eyes. Instillation of a miotic drug successfully reversed the IOP to normal levels. Conclusions and Importance: BDUMP caused secondary angle closure in both eyes, presumably due to thickening of the entire ciliary body. AS-OCT and UBM were advantageous for analyzing the morphology of the anterior eye segment in BDUMP. Clinicians should be aware of the possibility of angle closure during the management of patients with BDUMP

Effect and Safety of Granulocyte-Monocyte Adsorption Apheresis for Patients with Ulcerative Colitis Positive for Cytomegalovirus in Comparison with Immunosuppressants

Background: Cytomegalovirus (CMV) infection exacerbates ulcerative colitis (UC) refractory to immunosuppressive therapies (IMT). However, the underlying UC remained active in some UC patients, despite the fact that CMV-DNA in colonic mucosa became negative after antiviral therapy. Therefore, new therapeutic strategies for UC patients concomitant with CMV infection in mucosa are required. Aims: The aim of this study was to evaluate the effect and safety of granulocyte-monocyte adsorption apheresis (GMA) in UC patients positive for CMV infection after antiviral therapy. Methods: From October 2003 to December 2008, 64 patients with UC refractory to IMT, including steroids and immunomodulators, were enrolled in this retrospective, observational, multicenter study, which was reviewed and approved by the Institutional Review Board of Kyoto University. CMV infection was investigated by 3 methods (histologic examination, CMV antigenemia, and polymerase chain reaction). We investigated the clinical outcomes of GMA and IMT after 2 weeks of treatment with ganciclovir. Results: Thirty-one (48.4%) of 64 patients with UC refractory to IMT were positive for CMV. Of the 31 patients, 4 (12.9%) underwent colectomy. Twenty-seven patients (87.1%) underwent antiviral therapy. Of those 27 patients, 7 achieved remission following antiviral therapy alone. Of the remaining 20 patients who did not achieve remission despite the disappearance of CMV-DNA, 11 and 9 patients were treated with additional GMA (GMA group) and IMT (IMT group), respectively. Of 11 patients (GMA group), 9 achieved remission and 2 underwent colectomy. Out of the remaining 9 patients (IMT group), 4 achieved remission and 5 underwent colectomy. CMV-DNA was not detected in 11 patients after GMA, but it was detected again in all 5 patients of the IMT group who underwent colectomy. The total colectomy rate in UC patients positive for CMV was 35.5% (11/31). In addition, colectomy-free survival in the CMV relapse (+) group was estimated to be 12.9% at 65 months, while that in the CMV relapse (–) group was estimated to be 100% at 60 months. Conclusion: The colectomy ratio tends to be high in refractory UC patients with recurrent CMV reactivation or infection. Therefore, GMA might be a safe and effective treatment for UC patients positive for CMV because it does not induce CMV reactivation

Serum soluble triggering receptor expressed on myeloid cells‐2 was not altered by rTMS in patients with treatment‐resistant depression

Abstract Aim Repetitive transcranial magnetic stimulation (rTMS) is one of the most effective and minimally invasive treatments for treatment‐resistant depression (TRD). However, the mechanism underlying the therapeutic effects of rTMS in patients with TRD remains unclear. In recent years, the pathogenesis of depression has been closely associated with chronic inflammation and microglia are believed to play an important role in chronic inflammation. Triggering receptor expressed on myeloid cells‐2 (TREM2) plays an important role in microglial neuroinflammatory regulation. In this study, we investigated the changes in peripheral soluble TREM2 (sTREM2) before and after rTMS treatment in patients with TRD. Methods Twenty‐six patients with TRD were enrolled in this frequency (10 Hz) rTMS study. Depressive symptoms, cognitive function, and serum sTREM2 concentrations were measured at baseline and the end of the 6‐week rTMS treatment. Results This study showed that rTMS ameliorated depressive symptoms and partially improved cognitive dysfunction in TRD. However, rTMS treatment did not alter serum sTREM2 levels. Conclusions This is the first sTREM2 study in patients with TRD who underwent rTMS treatment. These results suggest that serum sTREM2 may not be relevant for the mechanism underlying the therapeutic effect of rTMS in patients with TRD. Future studies should confirm the present findings using a larger patient sample and a sham rTMS procedure, as well as CSF sTREM2. Furthermore, a longitudinal study should be conducted to clarify the effects of rTMS on sTREM2 levels