Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays

During sleep and awake rest, the hippocampus replays sequences of place cells that have been activated during prior experiences. These have been interpreted as a memory consolidation process, but recent results suggest a possible interpretation in terms of reinforcement learning. The Dyna reinforcement learning algorithms use off-line replays to improve learning. Under limited replay budget, a prioritized sweeping approach, which requires a model of the transitions to the predecessors, can be used to improve performance. We investigate whether such algorithms can explain the experimentally observed replays. We propose a neural network version of prioritized sweeping Q-learning, for which we developed a growing multiple expert algorithm, able to cope with multiple predecessors. The resulting architecture is able to improve the learning of simulated agents confronted to a navigation task. We predict that, in animals, learning the world model should occur during rest periods, and that the corresponding replays should be shuffled.Comment: Living Machines 2018 (Paris, France

A Microsoft-Excel-based tool for running and critically appraising network meta-analyses--an overview and application of NetMetaXL.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.BACKGROUND: The use of network meta-analysis has increased dramatically in recent years. WinBUGS, a freely available Bayesian software package, has been the most widely used software package to conduct network meta-analyses. However, the learning curve for WinBUGS can be daunting, especially for new users. Furthermore, critical appraisal of network meta-analyses conducted in WinBUGS can be challenging given its limited data manipulation capabilities and the fact that generation of graphical output from network meta-analyses often relies on different software packages than the analyses themselves. METHODS: We developed a freely available Microsoft-Excel-based tool called NetMetaXL, programmed in Visual Basic for Applications, which provides an interface for conducting a Bayesian network meta-analysis using WinBUGS from within Microsoft Excel. . This tool allows the user to easily prepare and enter data, set model assumptions, and run the network meta-analysis, with results being automatically displayed in an Excel spreadsheet. It also contains macros that use NetMetaXL's interface to generate evidence network diagrams, forest plots, league tables of pairwise comparisons, probability plots (rankograms), and inconsistency plots within Microsoft Excel. All figures generated are publication quality, thereby increasing the efficiency of knowledge transfer and manuscript preparation. RESULTS: We demonstrate the application of NetMetaXL using data from a network meta-analysis published previously which compares combined resynchronization and implantable defibrillator therapy in left ventricular dysfunction. We replicate results from the previous publication while demonstrating result summaries generated by the software. CONCLUSIONS: Use of the freely available NetMetaXL successfully demonstrated its ability to make running network meta-analyses more accessible to novice WinBUGS users by allowing analyses to be conducted entirely within Microsoft Excel. NetMetaXL also allows for more efficient and transparent critical appraisal of network meta-analyses, enhanced standardization of reporting, and integration with health economic evaluations which are frequently Excel-based.CC is a recipient of a Vanier Canada Graduate Scholarship from the Canadian Institutes of Health Research (funding reference number—CGV 121171) and is a trainee on the Canadian Institutes of Health Research Drug Safety and Effectiveness Network team grant (funding reference number—116573). BH is funded by a New Investigator award from the Canadian Institutes of Health Research and the Drug Safety and Effectiveness Network. This research was partly supported by funding from CADTH as part of a project to develop Excel-based tools to support the conduct of health technology assessments. This research was also supported by Cornerstone Research Group

Training emergency services’ dispatchers to recognise stroke: an interrupted time-series analysis

Background: Stroke is a time-dependent medical emergency in which early presentation to specialist care reduces death and dependency. Up to 70% of all stroke patients obtain first medical contact from the Emergency Medical Services (EMS). Identifying ‘true stroke’ from an EMS call is challenging, with over 50% of strokes being misclassified. The aim of this study was to evaluate the impact of the training package on the recognition of stroke by Emergency Medical Dispatchers (EMDs). Methods: This study took place in an ambulance service and a hospital in England using an interrupted time-series design. Suspected stroke patients were identified in one week blocks, every three weeks over an 18 month period, during which time the training was implemented. Patients were included if they had a diagnosis of stroke (EMS or hospital). The effect of the intervention on the accuracy of dispatch diagnosis was investigated using binomial (grouped) logistic regression. Results: In the Pre-implementation period EMDs correctly identified 63% of stroke patients; this increased to 80% Post-implementation. This change was significant (p=0.003), reflecting an improvement in identifying stroke patients relative to the Pre-implementation period both the During-implementation (OR=4.10 [95% CI 1.58 to 10.66]) and Post-implementation (OR=2.30 [95% CI 1.07 to 4.92]) periods. For patients with a final diagnosis of stroke who had been dispatched as stroke there was a marginally non-significant 2.8 minutes (95% CI −0.2 to 5.9 minutes, p=0.068)reduction between Pre- and Post-implementation periods from call to arrival of the ambulance at scene. Conclusions: This is the first study to develop, implement and evaluate the impact of a training package for EMDs with the aim of improving the recognition of stroke. Training led to a significant increase in the proportion of stroke patients dispatched as such by EMDs; a small reduction in time from call to arrival at scene by the ambulance also appeared likely. The training package has been endorsed by the UK Stroke Forum Education and Training, and is free to access on-line

A Cognitive Architecture Based on a Learning Classifier System with Spiking Classifiers

© 2015, Springer Science+Business Media New York. Learning classifier systems (LCS) are population-based reinforcement learners that were originally designed to model various cognitive phenomena. This paper presents an explicitly cognitive LCS by using spiking neural networks as classifiers, providing each classifier with a measure of temporal dynamism. We employ a constructivist model of growth of both neurons and synaptic connections, which permits a genetic algorithm to automatically evolve sufficiently-complex neural structures. The spiking classifiers are coupled with a temporally-sensitive reinforcement learning algorithm, which allows the system to perform temporal state decomposition by appropriately rewarding “macro-actions”, created by chaining together multiple atomic actions. The combination of temporal reinforcement learning and neural information processing is shown to outperform benchmark neural classifier systems, and successfully solve a robotic navigation task

Методи оцінки ризиків в інформаційній системі аналізу екологічного стану басейну малої ріки

В інформаційній системі аналізу стану басейну малої ріки запропоновано методи оцінки ризиків на основі імовірнісних та статистичних оцінок, формалізації моделі гри з природою, прогнозування процесів підтоплення земель з використанням ланцюгів Маркова, розглянуто багато критеріальні моделі ризиків.In informational and analytical system of the small rivers’ ecological condition estimation the methods of risks modelling on the basis of likelihood and statistical estimations, formalization of models of game with nature, risk modelling and forecasting processes flooded lands using Markov chains are offered, multicriteria models of risks are considered

Systematic review and meta-analysis of the diagnostic accuracy of ultrasonography for deep vein thrombosis

Background Ultrasound (US) has largely replaced contrast venography as the definitive diagnostic test for deep vein thrombosis (DVT). We aimed to derive a definitive estimate of the diagnostic accuracy of US for clinically suspected DVT and identify study-level factors that might predict accuracy. Methods We undertook a systematic review, meta-analysis and meta-regression of diagnostic cohort studies that compared US to contrast venography in patients with suspected DVT. We searched Medline, EMBASE, CINAHL, Web of Science, Cochrane Database of Systematic Reviews, Cochrane Controlled Trials Register, Database of Reviews of Effectiveness, the ACP Journal Club, and citation lists (1966 to April 2004). Random effects meta-analysis was used to derive pooled estimates of sensitivity and specificity. Random effects meta-regression was used to identify study-level covariates that predicted diagnostic performance. Results We identified 100 cohorts comparing US to venography in patients with suspected DVT. Overall sensitivity for proximal DVT (95% confidence interval) was 94.2% (93.2 to 95.0), for distal DVT was 63.5% (59.8 to 67.0), and specificity was 93.8% (93.1 to 94.4). Duplex US had pooled sensitivity of 96.5% (95.1 to 97.6) for proximal DVT, 71.2% (64.6 to 77.2) for distal DVT and specificity of 94.0% (92.8 to 95.1). Triplex US had pooled sensitivity of 96.4% (94.4 to 97.1%) for proximal DVT, 75.2% (67.7 to 81.6) for distal DVT and specificity of 94.3% (92.5 to 95.8). Compression US alone had pooled sensitivity of 93.8 % (92.0 to 95.3%) for proximal DVT, 56.8% (49.0 to 66.4) for distal DVT and specificity of 97.8% (97.0 to 98.4). Sensitivity was higher in more recently published studies and in cohorts with higher prevalence of DVT and more proximal DVT, and was lower in cohorts that reported interpretation by a radiologist. Specificity was higher in cohorts that excluded patients with previous DVT. No studies were identified that compared repeat US to venography in all patients. Repeat US appears to have a positive yield of 1.3%, with 89% of these being confirmed by venography. Conclusion Combined colour-doppler US techniques have optimal sensitivity, while compression US has optimal specificity for DVT. However, all estimates are subject to substantial unexplained heterogeneity. The role of repeat scanning is very uncertain and based upon limited data

Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement

David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyse

Spatio-temporal Models of Lymphangiogenesis in Wound Healing

Several studies suggest that one possible cause of impaired wound healing is failed or insufficient lymphangiogenesis, that is the formation of new lymphatic capillaries. Although many mathematical models have been developed to describe the formation of blood capillaries (angiogenesis), very few have been proposed for the regeneration of the lymphatic network. Lymphangiogenesis is a markedly different process from angiogenesis, occurring at different times and in response to different chemical stimuli. Two main hypotheses have been proposed: 1) lymphatic capillaries sprout from existing interrupted ones at the edge of the wound in analogy to the blood angiogenesis case; 2) lymphatic endothelial cells first pool in the wound region following the lymph flow and then, once sufficiently populated, start to form a network. Here we present two PDE models describing lymphangiogenesis according to these two different hypotheses. Further, we include the effect of advection due to interstitial flow and lymph flow coming from open capillaries. The variables represent different cell densities and growth factor concentrations, and where possible the parameters are estimated from biological data. The models are then solved numerically and the results are compared with the available biological literature.Comment: 29 pages, 9 Figures, 6 Tables (39 figure files in total