Exploiting Multiple Abstractions in Episodic RL via Reward Shaping

One major limitation to the applicability of Reinforcement Learning (RL) to many practical domains is the large number of samples required to learn an optimal policy. To address this problem and improve learning efficiency, we consider a linear hierarchy of abstraction layers of the Markov Decision Process (MDP) underlying the target domain. Each layer is an MDP representing a coarser model of the one immediately below in the hierarchy. In this work, we propose a novel form of Reward Shaping where the solution obtained at the abstract level is used to offer rewards to the more concrete MDP, in such a way that the abstract solution guides the learning in the more complex domain. In contrast with other works in Hierarchical RL, our technique has few requirements in the design of the abstract models and it is also tolerant to modeling errors, thus making the proposed approach practical. We formally analyze the relationship between the abstract models and the exploration heuristic induced in the lower-level domain. Moreover, we prove that the method guarantees optimal convergence and we demonstrate its effectiveness experimentally.Comment: This is an extended version of the paper presented at AAAI 2023, https://doi.org/10.1609/aaai.v37i6.2588

A severe case of hemobilia and biliary fistula following an open urgent cholecystectomy

<p>Abstract</p> <p>Background</p> <p>Cholecystectomy has been the treatment of choice for symptomatic gallstones, but remains the greatest source of post-operative biliary injuries. Laparoscopic approach has been recently preferred because of short hospitalisation and low morbidity but has an higher incidence of biliary leakages and bile duct injuries than open one due to a technical error or misinterpretation of the anatomy. Even open cholecystectomy presents a small number of complications especially if it was performed in urgency. Hemobilia is one of the most common cause of upper gastrointestinal bleeding from the biliary ducts into the gastrointestinal tract due to trauma, advent of invasive procedures such as percutaneous liver biopsy, transhepatic cholangiography, and biliary drainage.</p> <p>Methods</p> <p>We report here a case of massive hemobilia in a 60-year-old man who underwent an urgent open cholecystectomy and a subsequent placement of a transhepatic biliary drainage.</p> <p>Conclusion</p> <p>The management of these complications enclose endoscopic, percutaneous and surgical therapies. After a diagnosis of biliary fistula, it's most important to assess the adequacy of bile drainage to determine a controlled fistula and to avoid bile collection and peritonitis. Transarterial embolization is the first line of intervention to stop hemobilia while surgical intervention should be considered if embolization fails or is contraindicated.</p

Italian Guidelines in diagnosis and treatment of alopecia areata

Alopecia areata (AA) is an organ-specific autoimmune disorder that targets anagen phase hair follicles. The course is unpredictable and current available treatments have variable efficacy. Nowadays, there is relatively little evidence on treatment of AA from well-designed clinical trials. Moreover, none of the treatments or devices commonly used to treat AA are specifically approved by the Food and Drug Administration. The Italian Study Group for Cutaneous Annexial Disease of the Italian Society of dermatology proposes these Italian guidelines for diagnosis and treatment of Alopecia Areata deeming useful for the daily management of the disease. This article summarizes evidence-based treatment associated with expert-based recommendations

2020 WSES guidelines for the detection and management of bile duct injury during cholecystectomy.

Bile duct injury (BDI) is a dangerous complication of cholecystectomy, with significant postoperative sequelae for the patient in terms of morbidity, mortality, and long-term quality of life. BDIs have an estimated incidence of 0.4-1.5%, but considering the number of cholecystectomies performed worldwide, mostly by laparoscopy, surgeons must be prepared to manage this surgical challenge. Most BDIs are recognized either during the procedure or in the immediate postoperative period. However, some BDIs may be discovered later during the postoperative period, and this may translate to delayed or inappropriate treatments. Providing a specific diagnosis and a precise description of the BDI will expedite the decision-making process and increase the chance of treatment success. Subsequently, the choice and timing of the appropriate reconstructive strategy have a critical role in long-term prognosis. Currently, a wide spectrum of multidisciplinary interventions with different degrees of invasiveness is indicated for BDI management. These World Society of Emergency Surgery (WSES) guidelines have been produced following an exhaustive review of the current literature and an international expert panel discussion with the aim of providing evidence-based recommendations to facilitate and standardize the detection and management of BDIs during cholecystectomy. In particular, the 2020 WSES guidelines cover the following key aspects: (1) strategies to minimize the risk of BDI during cholecystectomy; (2) BDI rates in general surgery units and review of surgical practice; (3) how to classify, stage, and report BDI once detected; (4) how to manage an intraoperatively detected BDI; (5) indications for antibiotic treatment; (6) indications for clinical, biochemical, and imaging investigations for suspected BDI; and (7) how to manage a postoperatively detected BDI

Research and Development for Near Detector Systems Towards Long Term Evolution of Ultra-precise Long-baseline Neutrino Experiments

With the discovery of non-zero value of $\theta_{13}$ mixing angle, the next generation of long-baseline neutrino (LBN) experiments offers the possibility of obtaining statistically significant samples of muon and electron neutrinos and anti-neutrinos with large oscillation effects. In this document we intend to highlight the importance of Near Detector facilities in LBN experiments to both constrain the systematic uncertainties affecting oscillation analyses but also to perform, thanks to their close location, measurements of broad benefit for LBN physics goals. A strong European contribution to these efforts is possible

Computation of spectral characteristics for charged integral equations

\u3cp\u3eThe Rayleigh-Ritz and the inverse iteration methods are used in order to compute the eigenvalues of charged Fredholm-Stieltjes integral equations, i.e. Fredholm equations with respect to suitable Stieltjes-type measures. Some applications are shown, including approximation of the relevant eigenfunctions. Starting from the problem of a string charged by a finite number of cursors, a survey including the extensions to the 2D and 3D dimensional problems is presented.\u3c/p\u3

THE IMPORTANCE OF PHYSICAL EXERCISE IN ARTERIOPATHIC PATIENTS

La pratica regolare di un’attività fisica programmata fa registrare benefici clinici (clauducatio intermittens) e strumentali (riflessi posturali) nei pazienti affetti da arteriopatia obliterante cronica.Regularly scheduled physical activity leads to clinical (claudicatio intermittens) and instrumental (postural reflexes) improvement in patients suffering from chronic occluding arteriopathy

Pruritic papular eruption in HIV: A case successfully treated with NB-UVB

Abstact Pruritic papular eruption (PPE) is a skin disease characterized by the eruption of itching papules on the extremities, face, and trunk; it is quite frequent in HIV-positive patients especially during the advanced immunosuppressive stage. PPE usually improves or heals when antiretroviral therapy restores the immune system function, but in some cases, it can take several months, and a symptomatic treatment for PPE is required. Systemic antihistamines, topical steroids, topical tacrolimus, itraconazole, pentoxyphilline, and ultraviolet B phototherapy have been proposed in cases of persisting PPE, but an elective treatment has not yet been found. We describe the case of a black patient affected by PPE, nonimproving with antiretroviral treatment, and resistant to topical steroids and oral antihistamines; a satisfactory and speedy result was achieved with narrow-band ultraviolet B phototherapy. © 2013 Wiley Periodicals, Inc

PHYSICAL ACTIVITY, DIET AND PREVENTION OF BREAST CANCER

La riduzione del rischio per il cancro della mammella è legato all'insulino-resistenza e alla percentuale di massa grassa e risente più dell'attività fisica che della solo dieta. L'eliminazione completa del rischio può essere raggiunta con la dieta e l'attività fisica per la loro sinergia nel ridurre BMI, massa grassa e insulino-resistenza.Risk reduction is related to Insulin Resistance and Fat Mass percent and respond more to physical activity than to diet only. Complete deletion of risk can be achieved with diet and physical activity for their synergy in reducing BMI, Fat Mass and Insuli