Crawling in Rogue's dungeons with (partitioned) A3C

Rogue is a famous dungeon-crawling video-game of the 80ies, the ancestor of its gender. Rogue-like games are known for the necessity to explore partially observable and always different randomly-generated labyrinths, preventing any form of level replay. As such, they serve as a very natural and challenging task for reinforcement learning, requiring the acquisition of complex, non-reactive behaviors involving memory and planning. In this article we show how, exploiting a version of A3C partitioned on different situations, the agent is able to reach the stairs and descend to the next level in 98% of cases.Comment: Accepted at the Fourth International Conference on Machine Learning, Optimization, and Data Science (LOD 2018

Estimation of proteinuria as a predictor of complications of pre-eclampsia: a systematic review

Background Proteinuria is one of the essential criteria for the clinical diagnosis of pre-eclampsia. Increasing levels of proteinuria is considered to be associated with adverse maternal and fetal outcomes. We aim to determine the accuracy with which the amount of proteinuria predicts maternal and fetal complications in women with pre-eclampsia by systematic quantitative review of test accuracy studies. Methods We conducted electronic searches in MEDLINE (1951 to 2007), EMBASE (1980 to 2007), the Cochrane Library (2007) and the MEDION database to identify relevant articles and hand-search of selected specialist journals and reference lists of articles. There were no language restrictions for any of these searches. Two reviewers independently selected those articles in which the accuracy of proteinuria estimate was evaluated to predict maternal and fetal complications of pre-eclampsia. Data were extracted on study characteristics, quality and accuracy to construct 2 × 2 tables with maternal and fetal complications as reference standards. Results Sixteen primary articles with a total of 6749 women met the selection criteria with levels of proteinuria estimated by urine dipstick, 24-hour urine proteinuria or urine protein:creatinine ratio as a predictor of complications of pre-eclampsia. All 10 studies predicting maternal outcomes showed that proteinuria is a poor predictor of maternal complications in women with pre-eclampsia. Seventeen studies used laboratory analysis and eight studies bedside analysis to assess the accuracy of proteinuria in predicting fetal and neonatal complications. Summary likelihood ratios of positive and negative tests for the threshold level of 5 g/24 h were 2.0 (95% CI 1.5, 2.7) and 0.53 (95% CI 0.27, 1) for stillbirths, 1.5 (95% CI 0.94, 2.4) and 0.73 (95% CI 0.39, 1.4) for neonatal deaths and 1.5 (95% 1, 2) and 0.78 (95% 0.64, 0.95) for Neonatal Intensive Care Unit admission. Conclusion Measure of proteinuria is a poor predictor of either maternal or fetal complications in women with pre-eclampsia

Growth characteristics in individuals with osteogenesis imperfecta in North America: results from a multicenter study.

PurposeOsteogenesis imperfecta (OI) predisposes people to recurrent fractures, bone deformities, and short stature. There is a lack of large-scale systematic studies that have investigated growth parameters in OI.MethodsUsing data from the Linked Clinical Research Centers, we compared height, growth velocity, weight, and body mass index (BMI) in 552 individuals with OI. Height, weight, and BMI were plotted on Centers for Disease Control and Prevention normative curves.ResultsIn children, the median z-scores for height in OI types I, III, and IV were -0.66, -6.91, and -2.79, respectively. Growth velocity was diminished in OI types III and IV. The median z-score for weight in children with OI type III was -4.55. The median z-scores for BMI in children with OI types I, III, and IV were 0.10, 0.91, and 0.67, respectively. Generalized linear model analyses demonstrated that the height z-score was positively correlated with the severity of the OI subtype (P < 0.001), age, bisphosphonate use, and rodding (P < 0.05).ConclusionFrom the largest cohort of individuals with OI, we provide median values for height, weight, and BMI z-scores that can aid the evaluation of overall growth in the clinic setting. This study is an important first step in the generation of OI-specific growth curves

Deep Reinforcement Learning: An Overview

In recent years, a specific machine learning method called deep learning has gained huge attraction, as it has obtained astonishing results in broad applications such as pattern recognition, speech recognition, computer vision, and natural language processing. Recent research has also been shown that deep learning techniques can be combined with reinforcement learning methods to learn useful representations for the problems with high dimensional raw data input. This chapter reviews the recent advances in deep reinforcement learning with a focus on the most used deep architectures such as autoencoders, convolutional neural networks and recurrent neural networks which have successfully been come together with the reinforcement learning framework.Comment: Proceedings of SAI Intelligent Systems Conference (IntelliSys) 201

Intelligent Cooperative Control Architecture: A Framework for Performance Improvement Using Safe Learning

Planning for multi-agent systems such as task assignment for teams of limited-fuel unmanned aerial vehicles (UAVs) is challenging due to uncertainties in the assumed models and the very large size of the planning space. Researchers have developed fast cooperative planners based on simple models (e.g., linear and deterministic dynamics), yet inaccuracies in assumed models will impact the resulting performance. Learning techniques are capable of adapting the model and providing better policies asymptotically compared to cooperative planners, yet they often violate the safety conditions of the system due to their exploratory nature. Moreover they frequently require an impractically large number of interactions to perform well. This paper introduces the intelligent Cooperative Control Architecture (iCCA) as a framework for combining cooperative planners and reinforcement learning techniques. iCCA improves the policy of the cooperative planner, while reduces the risk and sample complexity of the learner. Empirical results in gridworld and task assignment for fuel-limited UAV domains with problem sizes up to 9 billion state-action pairs verify the advantage of iCCA over pure learning and planning strategies

A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks

Autonomous agents must learn to collaborate. It is not scalable to develop a new centralized agent every time a task's difficulty outpaces a single agent's abilities. While multi-agent collaboration research has flourished in gridworld-like environments, relatively little work has considered visually rich domains. Addressing this, we introduce the novel task FurnMove in which agents work together to move a piece of furniture through a living room to a goal. Unlike existing tasks, FurnMove requires agents to coordinate at every timestep. We identify two challenges when training agents to complete FurnMove: existing decentralized action sampling procedures do not permit expressive joint action policies and, in tasks requiring close coordination, the number of failed actions dominates successful actions. To confront these challenges we introduce SYNC-policies (synchronize your actions coherently) and CORDIAL (coordination loss). Using SYNC-policies and CORDIAL, our agents achieve a 58% completion rate on FurnMove, an impressive absolute gain of 25 percentage points over competitive decentralized baselines. Our dataset, code, and pretrained models are available at https://unnat.github.io/cordial-sync .Comment: Accepted to ECCV 2020 (spotlight); Project page: https://unnat.github.io/cordial-syn

Reporting of prognostic markers: current problems and development of guidelines for evidence-based practice in the future

Prognostic markers help to stratify patients for treatment by identifying patients with different risks of outcome (e.g. recurrence of disease), and are important tools in the management of cancer and many other diseases. Systematic review and meta-analytical approaches to identifying the most valuable prognostic markers are needed because (sometimes conflicting) evidence relating to markers is often published across a number of studies. To investigate the practicality of this approach, an empirical investigation of a systematic review of tumour markers for neuroblastoma was performed; 260 studies of prognostic markers were identified, which considered 130 different markers

Formyl Peptide Receptor as a Novel Therapeutic Target for Anxiety-Related Disorders

Formyl peptide receptors (FPR) belong to a family of sensors of the immune system that detect microbe-associated molecules and inform various cellular and sensorial mechanisms to the presence of pathogens in the host. Here we demonstrate that Fpr2/3-deficient mice show a distinct profile of behaviour characterised by reduced anxiety in the marble burying and light-dark box paradigms, increased exploratory behaviour in an open-field, together with superior performance on a novel object recognition test. Pharmacological blockade with a formyl peptide receptor antagonist, Boc2, in wild type mice reproduced most of the behavioural changes observed in the Fpr2/3(-/-) mice, including a significant improvement in novel object discrimination and reduced anxiety in a light/dark shuttle test. These effects were associated with reduced FPR signalling in the gut as shown by the significant reduction in the levels of p-p38. Collectively, these findings suggest that homeostatic FPR signalling exerts a modulatory effect on anxiety-like behaviours. These findings thus suggest that therapies targeting FPRs may be a novel approach to ameliorate behavioural abnormalities present in neuropsychiatric disorders at the cognitive-emotional interface

Escherichia coli Frameshift Mutation Rate Depends on the Chromosomal Context but Not on the GATC Content Near the Mutation Site

Different studies have suggested that mutation rate varies at different positions in the genome. In this work we analyzed if the chromosomal context and/or the presence of GATC sites can affect the frameshift mutation rate in the Escherichia coli genome. We show that in a mismatch repair deficient background, a condition where the mutation rate reflects the fidelity of the DNA polymerization process, the frameshift mutation rate could vary up to four times among different chromosomal contexts. Furthermore, the mismatch repair efficiency could vary up to eight times when compared at different chromosomal locations, indicating that detection and/or repair of frameshift events also depends on the chromosomal context. Also, GATC sequences have been proved to be essential for the correct functioning of the E. coli mismatch repair system. Using bacteriophage heteroduplexes molecules it has been shown that GATC influence the mismatch repair efficiency in a distance- and number-dependent manner, being almost nonfunctional when GATC sequences are located at 1 kb or more from the mutation site. Interestingly, we found that in E. coli genomic DNA the mismatch repair system can efficiently function even if the nearest GATC sequence is located more than 2 kb away from the mutation site. The results presented in this work show that even though frameshift mutations can be efficiently generated and/or repaired anywhere in the genome, these processes can be modulated by the chromosomal context that surrounds the mutation site

Sequential biventricular pacing improves regional contractility, longitudinal function and dyssynchrony in patients with heart failure and prolonged QRS

<p>Abstract</p> <p>Aims</p> <p>Biventricular pacing (BiP) is an effective treatment in systolic heart failure (HF) patients with prolonged QRS. However, approximately 35% of the patients receiving BiP are classified as non-responders. The aim of this study is to evaluate the acute effects of VV-optimization on systolic heart function.</p> <p>Methods</p> <p>Twenty-one HF patients aged 72 (46-88) years, QRS 154 (120-190) ms, were studied with echocardiography, Tissue Doppler Imaging (TDI) and 3D-echo the first day after receiving a BiP device. TDI was performed; during simultaneous pacing (LV-lead pacing 4 ms before the RV-lead) and during sequential pacing (LV 20 and 40 ms before RV and RV 20 and 40 ms before LV-lead pacing). Systolic heart function was studied by tissue tracking (TT) for longitudinal function and systolic maximal velocity (SMV) for regional contractility and signs of dyssynchrony assessed by time-delays standard deviation of aortic valve opening to SMV, AVO-SMV/SD and tissue synchronization imaging (TSI).</p> <p>Results</p> <p>The TT mean value preoperatively was 4,2 ± 1,5 and increased at simultaneous pacing to 5,0 ± 1,2 mm (p < 0,05), and at best VV-interval to 5,4 ± 1,2 (p < 0,001). Simultaneous pacing achieved better TT distance compared with preoperative in 16 patients (76%). However, it was still higher after VV-optimization in 12 patients 57%. Corresponding figures for SMV were 3,0 ± 0,7, 3,5 ± 0,8 (p < 0,01), and 3,6 ± 0,8 (p < 0,001). Also dyssynchrony improved.</p> <p>Conclusions</p> <p>VV-optimization in the acute phase improves systolic heart function more than simultaneous BiP pacing. Long-term effects should be evaluated in prospective randomized trials.</p