Memory-efficient episodic control reinforcement learning with dynamic online k-means

Recently, neuro-inspired episodic control (EC) methods have been developed to overcome the data-inefficiency of standard deep reinforcement learning approaches. Using non-/semi-parametric models to estimate the value function, they learn rapidly, retrieving cached values from similar past states. In realistic scenarios, with limited resources and noisy data, maintaining meaningful representations in memory is essential to speed up the learning and avoid catastrophic forgetting. Unfortunately, EC methods have a large space and time complexity. We investigate different solutions to these problems based on prioritising and ranking stored states, as well as online clustering techniques. We also propose a new dynamic online k-means algorithm that is both computationally-efficient and yields significantly better performance at smaller memory sizes; we validate this approach on classic reinforcement learning environments and Atari games

Sample-efficient reinforcement learning with maximum entropy mellowmax episodic control

Deep networks have enabled reinforcement learning to scale to more complex and challenging domains, but these methods typically require large quantities of training data. An alternative is to use sample-efficient episodic control methods: neuro-inspired algorithms which use non-/semi-parametric models that predict values based on storing and retrieving previously experienced transitions. One way to further improve the sample efficiency of these approaches is to use more principled exploration strategies. In this work, we therefore propose maximum entropy mellowmax episodic control (MEMEC), which samples actions according to a Boltzmann policy with a state-dependent temperature. We demonstrate that MEMEC outperforms other uncertainty- and softmax-based exploration methods on classic reinforcement learning environments and Atari games, achieving both more rapid learning and higher final rewards

Simple scoring system to predict in-hospital mortality after surgery for infective endocarditis

BACKGROUND: Aspecific scoring systems are used to predict the risk of death postsurgery in patients with infective endocarditis (IE). The purpose of the present study was both to analyze the risk factors for in-hospital death, which complicates surgery for IE, and to create a mortality risk score based on the results of this analysis. METHODS AND RESULTS: Outcomes of 361 consecutive patients (mean age, 59.1\ub115.4 years) who had undergone surgery for IE in 8 European centers of cardiac surgery were recorded prospectively, and a risk factor analysis (multivariable logistic regression) for in-hospital death was performed. The discriminatory power of a new predictive scoring system was assessed with the receiver operating characteristic curve analysis. Score validation procedures were carried out. Fifty-six (15.5%) patients died postsurgery. BMI >27 kg/m2 (odds ratio [OR], 1.79; P=0.049), estimated glomerular filtration rate 55 mm Hg (OR, 1.78; P=0.032), and critical state (OR, 2.37; P=0.017) were independent predictors of in-hospital death. A scoring system was devised to predict in-hospital death postsurgery for IE (area under the receiver operating characteristic curve, 0.780; 95% CI, 0.734-0.822). The score performed better than 5 of 6 scoring systems for in-hospital death after cardiac surgery that were considered. CONCLUSIONS: A simple scoring system based on risk factors for in-hospital death was specifically created to predict mortality risk postsurgery in patients with IE

Staphylococcus aureus CC30 Lineage and Absence of sed,j,r-Harboring Plasmid Predict Embolism in Infective Endocarditis

Staphylococcus aureus induces severe infective endocarditis (IE) where embolic complications are a major cause of death. Risk factors for embolism have been reported such as a younger age or larger IE vegetations, while methicillin resistance conferred by the mecA gene appeared as a protective factor. It is unclear, however, whether embolism is influenced by other S. aureus characteristics such as clonal complex (CC) or virulence pattern. We examined clinical and microbiological predictors of embolism in a prospective multicentric cohort of 98 French patients with monomicrobial S. aureus IE. The genomic contents of causative isolates were characterized using DNA array. To preserve statistical power, genotypic predictors were restricted to CC, secreted virulence factors and virulence regulators. Multivariate regularized logistic regression identified three independent predictors of embolism. Patients at higher risk were younger than the cohort median age of 62.5 y (adjusted odds ratio [OR] 0.14; 95% confidence interval [CI] 0.05–0.36). S. aureus characteristics predicting embolism were a CC30 genetic background (adjusted OR 9.734; 95% CI 1.53–192.8) and the absence of pIB485-like plasmid-borne enterotoxin-encoding genes sed, sej, and ser (sedjr; adjusted OR 0.07; 95% CI 0.004–0.457). CC30 S. aureus has been repeatedly reported to exhibit enhanced fitness in bloodstream infections, which might impact its ability to cause embolism. sedjr-encoded enterotoxins, whose superantigenic activity is unlikely to protect against embolism, possibly acted as a proxy to others genes of the pIB485-like plasmid found in genetically unrelated isolates from mostly embolism-free patients. mecA did not independently predict embolism but was strongly associated with sedjr. This mecA-sedjr association might have driven previous reports of a negative association of mecA and embolism. Collectively, our results suggest that the influence of S. aureus genotypic features on the risk of embolism may be stronger than previously suspected and independent of clinical risk factors

Combined Bacterial Meningitis and Infective Endocarditis: When Should We Search for the Other When Either One is Diagnosed?

Auteurs groupes collaboratifs AEPEI study group & the COMBAT study groupInternational audienc