C3PO: Learning to Achieve Arbitrary Goals via Massively Entropic Pretraining

Given a particular embodiment, we propose a novel method (C3PO) that learns policies able to achieve any arbitrary position and pose. Such a policy would allow for easier control, and would be re-useable as a key building block for downstream tasks. The method is two-fold: First, we introduce a novel exploration algorithm that optimizes for uniform coverage, is able to discover a set of achievable states, and investigates its abilities in attaining both high coverage, and hard-to-discover states; Second, we leverage this set of achievable states as training data for a universal goal-achievement policy, a goal-based SAC variant. We demonstrate the trained policy's performance in achieving a large number of novel states. Finally, we showcase the influence of massive unsupervised training of a goal-achievement policy with state-of-the-art pose-based control of the Hopper, Walker, Halfcheetah, Humanoid and Ant embodiments

Presurgical language mapping in children with epilepsy: Clinical usefulness of functional magnetic resonance imaging for the planning of cortical stimulation

Purpose: Presurgical language mapping in dominant hemisphere epilepsy to evaluate the risk of postoperative deficit is particularly difficult in children. Extraoperative invasive cortical stimulation can show some areas critical to language, but not all of them, due to scarce sampling, poor cooperation, cortical immaturity, or network reorganization, whereas functional magnetic resonance imaging (fMRI) displays entire networks involved in, but not necessarily critical to, language. In a homogeneous series of children with epilepsy, we compared the contributions of language fMRI and depth electrode stimulations to optimize language mapping. Methods: Eight children (7.5-15.5 years) with left frontal or temporal epilepsy underwent language fMRI and language stimulation with depth electrodes as part of their comprehensive presurgical workup. fMRI data collected during sentence generation were analyzed using statistical parametric mapping (SPM2) (false discovery rate [FDR] p \u3c 0.05). Bipolar stimulations were performed during language production tasks. By coregistering fMRI and postimplantation computed tomography (CT) images, we were able to directly compare the cortical areas identified by both investigations. Key Findings: fMRI during sentence generation robustly showed activation in the whole perisylvian regions with little reorganization (left hemisphere dominant in 7). Of the 184 electrode contacts tested for language, only 8 were positive (language disruption) in three of the seven patients with periictal language impairment and left language dominance. All of the positive contacts colocalized with an fMRI activated cluster, that is, fMRI did not miss any region critical to language (sensitivity = 100%). However, 54 of the 176 negative contacts were within activated clusters (low specificity). Significance: In children with epilepsy, the sensitivity of fMRI during sentence generation allows for the detection of all critical regions displayed by cortical stimulation within the large perisylvian language network, but with a low specificity. It is, therefore, useful to optimize the placement of intracranial electrodes when language mapping is necessary. Systematic planning of the electrode placement according to language fMRI maps should increase the yield of extraoperative cortical stimulation, which appears rather low in children when compared to adults. © 2011 International League Against Epilepsy

Deep Q-learning from Demonstrations

Deep reinforcement learning (RL) has achieved several high profile successes in difficult decision-making problems. However, these algorithms typically require a huge amount of data before they reach reasonable performance. In fact, their performance during learning can be extremely poor. This may be acceptable for a simulator, but it severely limits the applicability of deep RL to many real-world tasks, where the agent must learn in the real environment. In this paper we study a setting where the agent may access data from previous control of the system. We present an algorithm, Deep Q-learning from Demonstrations (DQfD), that leverages small sets of demonstration data to massively accelerate the learning process even from relatively small amounts of demonstration data and is able to automatically assess the necessary ratio of demonstration data while learning thanks to a prioritized replay mechanism. DQfD works by combining temporal difference updates with supervised classification of the demonstrator's actions. We show that DQfD has better initial performance than Prioritized Dueling Double Deep Q-Networks (PDD DQN) as it starts with better scores on the first million steps on 41 of 42 games and on average it takes PDD DQN 83 million steps to catch up to DQfD's performance. DQfD learns to out-perform the best demonstration given in 14 of 42 games. In addition, DQfD leverages human demonstrations to achieve state-of-the-art results for 11 games. Finally, we show that DQfD performs better than three related algorithms for incorporating demonstration data into DQN.Comment: Published at AAAI 2018. Previously on arxiv as "Learning from Demonstrations for Real World Reinforcement Learning

Optimizing statistical parametric mapping analysis of 18F-FDG PET in children.

International audienceABSTRACT: BACKGROUND: Statistical parametric mapping (SPM) procedure is an objective tool to analyze 18F-fluoro-2-deoxy-d-glucose-positron-emission tomography (FDG-PET) images and a useful complement to visual analysis. However, SPM requires a comparison to control data set that cannot be obtained in healthy children for ethical reasons. Using adults as controls showed some limitations. The purpose of the present study was to generate and validate a group of pseudo-normal children as a control group for FDG-PET studies in pediatrics. METHODS: FDG-PET images of 47 children (mean +/- SD age 10.2 +/- 3.1 years) with refractory symptomatic (MRI-positive, n = 20) and cryptogenic (MRI-negative, n = 27) focal epilepsy planned for surgery were analyzed using visual and SPM analysis. Performances of SPM analysis were compared using two different control groups: (1) an adult control group consisting of healthy young adults (n = 25, 30.5 +/- 5.8 years, adult PET template) and (2) a pediatric pseudo-control group consisting of patients (n = 24, 10.6 +/- 3.1 years, children PET template) with refractory focal epilepsy; but negative MRI and PET are considered normal not only on visual analysis but also on SPM. RESULTS: Among the 47 children, visual analysis succeeded detecting at least one hypometabolic area in 87% of the cases (interobserver kappa = 0.81). Regarding SPM analysis, the best compromise between sensitivity and specificity was obtained with a threshold of p less than 0.001 as an extent of more than 40 voxels. There was a significant concordance to detect hypometabolic areas between both SPM analyses [kappa (K) = 0.59; p < 0.005] and between both SPM and visual analyses (K = 0.45; p < 0.005), in symptomatic (K = 0.74; p < 0.005) as in cryptogenic patients (K = 0.26; p < 0.01). The pediatric pseudo-control group dramatically improved specificity (97% vs. 89%; p < 0.0001) by increasing the positive predictive value (86% vs. 65%). Sensitivity remained acceptable although it was not better (79% vs. 87%, p = 0.039). The main impact was to reduce by 41% the number of hypometabolic cortical artifacts detected by SPM, especially in the younger epileptic patients, which is a key point in clinical practice. CONCLUSIONS: This age-matched pseudo-control group is a way to optimize SPM analysis of FDG-PET in children with epilepsy. It might also be considered for other brain pathologies in pediatrics in the future

Get Back Here: Robust Imitation by Return-to-Distribution Planning

We consider the Imitation Learning (IL) setup where expert data are not collected on the actual deployment environment but on a different version. To address the resulting distribution shift, we combine behavior cloning (BC) with a planner that is tasked to bring the agent back to states visited by the expert whenever the agent deviates from the demonstration distribution. The resulting algorithm, POIR, can be trained offline, and leverages online interactions to efficiently fine-tune its planner to improve performance over time. We test POIR on a variety of human-generated manipulation demonstrations in a realistic robotic manipulation simulator and show robustness of the learned policy to different initial state distributions and noisy dynamics

A novel locus for generalized epilepsy with febrile seizures plus in French families.

International audienceBACKGROUND: Generalized epilepsy with febrile seizures plus (GEFS(+)) is a familial autosomal dominant entity characterized by the association of febrile and afebrile seizures. Mutations in 3 genes--the sodium channel alpha1 subunit gene (SCN1A), the sodium channel beta1 subunit gene (SCN1B), and the gamma2 GABA receptor subunit gene (GABRG2)--and linkage to 2 other loci on 2p24 and 21q22 have been identified in families with GEFS(+), indicating genetic heterogeneity. OBJECTIVES: To localize by means of linkage analysis a new gene for GEFS(+) in a large family with 11 affected members and to test the new locus in 4 additional families with GEFS(+). DESIGN: Family-based linkage analysis. SETTING: University hospital. PATIENTS: Five French families with GEFS(+) and at least 7 available affected members with autosomal dominant transmission. All the patients had febrile seizures and/or afebrile generalized tonic-clonic seizures or absence epilepsy. MAIN OUTCOME MEASURES: We analyzed 380 microsatellite markers and conducted linkage analysis. RESULTS: In the largest family, a 10-cM-density genomewide scan revealed linkage to a 13-Mb (megabase) interval on chromosome 8p23-p21 with a maximum pairwise logarithm of odds (LOD) score of 3.00 (at Theta = 0) for markers D8S351 and D8S550 and a multipoint LOD score of 3.23. A second family with GEFS(+) was also possibly linked to chromosome 8p23-p21 and the region was narrowed to a 7.3-Mb candidate interval, flanked by markers D8S1706 and D8S549. We have not, so far, identified mutations in the coding exons of 6 candidate genes (MTMR9, MTMR7, CTSB, SGCZ, SG223, and ATP6V1B2) located in the genetic interval. CONCLUSIONS: We report a sixth locus for GEFS(+) on chromosome 8p23-p21. Because no ion channel genes are located in this interval, identification of the responsible gene will probably uncover a new mechanism of pathogenesis for GEFS(+)

Lafora progressive myoclonus epilepsy: NHLRC1 mutations affect glycogen metabolism

11 páginas, 8 figuras, 1 tabla.Lafora disease is a fatal autosomal recessive form of progressive myoclonus epilepsy. Patients manifest myoclonus and tonic-clonic seizures, visual hallucinations, intellectual, and progressive neurologic deterioration beginning in adolescence. The two genes known to be involved in Lafora disease are EPM2A and NHLRC1 (EPM2B). The EPM2A gene encodes laforin, a dual-specificity protein phosphatase, and the NHLRC1 gene encodes malin, an E3-ubiquitin ligase. The two proteins interact with each other and, as a complex, are thought to regulate glycogen synthesis. Here, we report three Lafora families with two novel pathogenic mutations (C46Y and L261P) and two recurrent mutations (P69A and D146N) in NHLRC1. Investigation of their functional consequences in cultured mammalian cells revealed that malin(C46Y), malin(P69A), malin(D146N), and malin(L261P) mutants failed to downregulate the level of R5/PTG, a regulatory subunit of protein phosphatase 1 involved in glycogen synthesis. Abnormal accumulation of intracellular glycogen was observed with all malin mutants, reminiscent of the polyglucosan inclusions (Lafora bodies) present in patients with Lafora disease.Peer reviewe

Ketogenic diet for super-refractory status epilepticus (SRSE) with NORSE and FIRES: Single tertiary center experience and literature data

Background and purposeKetogenic diet (KD) is an emerging treatment option for super-refractory status epilepticus (SRSE). We evaluated the effectiveness of KD in patients presenting SRSE including NORSE (and its subcategory FIRES).MethodsA retrospective review of the medical records was performed at the Necker Enfants Malades Hospital. All children with SRSE in whom KD was started during the last 10 years were included. A systematic search was carried out for all study designs, including at least one patient of any age with SRSE in whom KD was started. The primary outcome was the responder rate and Kaplan–Meier survival curves were generated for the time-to-KD response. As secondary outcomes, Cox proportional hazard models were created to assess the impact of NORSE-related factors on KD efficacy.ResultsSixteen children received KD for treatment of SRSE, and three had NORSE presentation (one infectious etiology, two FIRES). In medical literature, 1,613 records were initially identified, and 75 were selected for review. We selected 276 patients receiving KD during SRSE. The most common etiology of SRSE was acute symptomatic (21.3%), among these patients, 67.7% presented with NORSE of immune and infectious etiologies. Other etiologies were remote symptomatic (6.8%), progressive symptomatic (6.1%), and SE in defined electroclinical syndromes (14.8%), including two patients with genetic etiology and NORSE presentation. The etiology was unknown in 50.7% of the patients presenting with cryptogenic NORSE, of which 102 presented with FIRES. Overall, most patients with NORSE benefit from KD (p &lt; 0.004), but they needed a longer time to achieve RSE resolution after starting KD compared with other non-NORSE SRSE (p = 0.001). The response to KD in the NORSE group with identified etiology compared to the cryptogenic NORSE was significantly higher (p = 0.01), and the time to achieve SE resolution after starting KD was shorter (p = 0.04).ConclusionsThe search for underlying etiology should help to a better-targeted therapy. KD can have good efficacy in NORSE; however, the time to achieve SE resolution seems to be longer in cryptogenic cases. These findings highlight the therapeutic role of KD in NORSE, even though this favorable response needs to be better confirmed in prospective controlled studies

Genotypic and phenotypic spectrum of pyridoxine-dependent epilepsy (ALDH7A1 deficiency)

Pyridoxine-dependent epilepsy was recently shown to be due to mutations in the ALDH7A1 gene, which encodes antiquitin, an enzyme that catalyses the nicotinamide adenine dinucleotide-dependent dehydrogenation of L-{alpha}-aminoadipic semialdehyde/L-{Delta}1-piperideine 6-carboxylate. However, whilst this is a highly treatable disorder, there is general uncertainty about when to consider this diagnosis and how to test for it. This study aimed to evaluate the use of measurement of urine L-{alpha}-aminoadipic semialdehyde/creatinine ratio and mutation analysis of ALDH7A1 (antiquitin) in investigation of patients with suspected or clinically proven pyridoxine-dependent epilepsy and to characterize further the phenotypic spectrum of antiquitin deficiency. Urinary L-{alpha}-aminoadipic semialdehyde concentration was determined by liquid chromatography tandem mass spectrometry. When this was above the normal range, DNA sequencing of the ALDH7A1 gene was performed. Clinicians were asked to complete questionnaires on clinical, biochemical, magnetic resonance imaging and electroencephalography features of patients. The clinical spectrum of antiquitin deficiency extended from ventriculomegaly detected on foetal ultrasound, through abnormal foetal movements and a multisystem neonatal disorder, to the onset of seizures and autistic features after the first year of life. Our relatively large series suggested that clinical diagnosis of pyridoxine dependent epilepsy can be challenging because: (i) there may be some response to antiepileptic drugs; (ii) in infants with multisystem pathology, the response to pyridoxine may not be instant and obvious; and (iii) structural brain abnormalities may co-exist and be considered sufficient cause of epilepsy, whereas the fits may be a consequence of antiquitin deficiency and are then responsive to pyridoxine. These findings support the use of biochemical and DNA tests for antiquitin deficiency and a clinical trial of pyridoxine in infants and children with epilepsy across a broad range of clinical scenarios

Genotypic and phenotypic spectrum of pyridoxine-dependent epilepsy (ALDH7A1 deficiency)

Pyridoxine-dependent epilepsy was recently shown to be due to mutations in the ALDH7A1 gene, which encodes antiquitin, an enzyme that catalyses the nicotinamide adenine dinucleotide-dependent dehydrogenation of l-α-aminoadipic semialdehyde/l-Δ1-piperideine 6-carboxylate. However, whilst this is a highly treatable disorder, there is general uncertainty about when to consider this diagnosis and how to test for it. This study aimed to evaluate the use of measurement of urine l-α-aminoadipic semialdehyde/creatinine ratio and mutation analysis of ALDH7A1 (antiquitin) in investigation of patients with suspected or clinically proven pyridoxine-dependent epilepsy and to characterize further the phenotypic spectrum of antiquitin deficiency. Urinary l-α-aminoadipic semialdehyde concentration was determined by liquid chromatography tandem mass spectrometry. When this was above the normal range, DNA sequencing of the ALDH7A1 gene was performed. Clinicians were asked to complete questionnaires on clinical, biochemical, magnetic resonance imaging and electroencephalography features of patients. The clinical spectrum of antiquitin deficiency extended from ventriculomegaly detected on foetal ultrasound, through abnormal foetal movements and a multisystem neonatal disorder, to the onset of seizures and autistic features after the first year of life. Our relatively large series suggested that clinical diagnosis of pyridoxine dependent epilepsy can be challenging because: (i) there may be some response to antiepileptic drugs; (ii) in infants with multisystem pathology, the response to pyridoxine may not be instant and obvious; and (iii) structural brain abnormalities may co-exist and be considered sufficient cause of epilepsy, whereas the fits may be a consequence of antiquitin deficiency and are then responsive to pyridoxine. These findings support the use of biochemical and DNA tests for antiquitin deficiency and a clinical trial of pyridoxine in infants and children with epilepsy across a broad range of clinical scenario