Optimizing the depth and the direction of prospective planning using information values

Evaluating the future consequences of actions is achievable by simulating a mental search tree into the future. Expanding deep trees, however, is computationally taxing. Therefore, machines and humans use a plan-until-habit scheme that simulates the environment up to a limited depth and then exploits habitual values as proxies for consequences that may arise in the future. Two outstanding questions in this scheme are “in which directions the search tree should be expanded?”, and “when should the expansion stop?”. Here we propose a principled solution to these questions based on a speed/accuracy tradeoff: deeper expansion in the appropriate directions leads to more accurate planning, but at the cost of slower decision-making. Our simulation results show how this algorithm expands the search tree effectively and efficiently in a grid-world environment. We further show that our algorithm can explain several behavioral patterns in animals and humans, namely the effect of time-pressure on the depth of planning, the effect of reward magnitudes on the direction of planning, and the gradual shift from goal-directed to habitual behavior over the course of training. The algorithm also provides several predictions testable in animal/human experiments

Cocaine Addiction as a Homeostatic Reinforcement Learning Disorder

Drug addiction implicates both reward learning and homeostatic regulation mechanisms of the brain. This has stimulated 2 partially successful theoretical perspectives on addiction. Many important aspects of addiction, however, remain to be explained within a single, unified framework that integrates the 2 mechanisms. Building upon a recently developed homeostatic reinforcement learning theory, the authors focus on a key transition stage of addiction that is well modeled in animals, escalation of drug use, and propose a computational theory of cocaine addiction where cocaine reinforces behavior due to its rapid homeostatic corrective effect, whereas its chronic use induces slow and long-lasting changes in homeostatic setpoint. Simulations show that our new theory accounts for key behavioral and neurobiological features of addiction, most notably, escalation of cocaine use, drug-primed craving and relapse, individual differences underlying dose-response curves, and dopamine D2-receptor downregulation in addicts. The theory also generates unique predictions about cocaine self-administration behavior in rats that are confirmed by new experimental results. Viewing addiction as a homeostatic reinforcement learning disorder coherently explains many behavioral and neurobiological aspects of the transition to cocaine addiction, and suggests a new perspective toward understanding addiction

Midbrain Dopamine Neurons Signal Belief in Choice Accuracy during a Perceptual Decision

Central to the organization of behavior is the ability to predict the values of outcomes to guide choices. The accuracy of such predictions is honed by a teaching signal that indicates how incorrect a prediction was (“reward prediction error,” RPE). In several reinforcement learning contexts, such as Pavlovian conditioning and decisions guided by reward history, this RPE signal is provided by midbrain dopamine neurons. In many situations, however, the stimuli predictive of outcomes are perceptually ambiguous. Perceptual uncertainty is known to influence choices, but it has been unclear whether or how dopamine neurons factor it into their teaching signal. To cope with uncertainty, we extended a reinforcement learning model with a belief state about the perceptually ambiguous stimulus; this model generates an estimate of the probability of choice correctness, termed decision confidence. We show that dopamine responses in monkeys performing a perceptually ambiguous decision task comply with the model’s predictions. Consequently, dopamine responses did not simply reflect a stimulus’ average expected reward value but were predictive of the trial-to-trial fluctuations in perceptual accuracy. These confidence-dependent dopamine responses emerged prior to monkeys’ choice initiation, raising the possibility that dopamine impacts impending decisions, in addition to encoding a post-decision teaching signal. Finally, by manipulating reward size, we found that dopamine neurons reflect both the upcoming reward size and the confidence in achieving it. Together, our results show that dopamine responses convey teaching signals that are also appropriate for perceptual decisions

MicroRNA-129-1 acts as tumour suppressor and induces cell cycle arrest of GBM cancer cells through targeting IGF2BP3 and MAPK1

Background MicroRNA-129-1 (miR-129-1) seems to behave as a tumour suppressor since its decreased expression is associated with different tumours such as glioblastoma multiforme (GBM). GBM is the most common form of brain tumours originating from glial cells. The impact of miR-129-1 downregulation on GBM pathogenesis has yet to be elucidated. Methods MiR-129-1 was overexpressed in GBM cells, and its effect on proliferation was investigated by cell cycle assay. MiR-129-1 predicted targets (CDK6, IGF1, HDAC2, IGF2BP3 and MAPK1) were also evaluated by western blot and luciferase assay. Results Restoration of miR-129-1 reduced cell proliferation and induced G1 accumulation, significantly. Several functional assays confirmed IGF2BP3, MAPK1 and CDK6 as targets of miR-129-1. Despite the fact that IGF1 expression can be suppressed by miR-129-1, through 30-untranslated region complementary sequence, we could not find any association between IGF1 expression and GBM. MiR-129-1 expression inversely correlates with CDK6, IGF2BP3 and MAPK1 in primary clinical samples. Conclusion This is the first study to propose miR129-1 as a negative regulator of IGF2BP3 and MAPK1 and also a cell cycle arrest inducer in GBM cells. Our data suggests miR-129-1 as a potential tumour suppressor and presents a rationale for the use of miR-129-1 as a novel strategy to improve treatment response in GBM

Significant contribution of small icebergs to the freshwater budget in Greenland fjords

This work is licensed under a Creative Commons Attribution 4.0 International License.Icebergs represent nearly half of the mass loss from the Greenland Ice Sheet and provide a distributed source of freshwater along fjords which can alter fjord circulation, nutrient levels, and ultimately the Meridional Overturning Circulation. Here we present analyses of high resolution optical satellite imagery using convolutional neural networks to accurately delineate iceberg edges in two East Greenland fjords. We find that a significant portion of icebergs in fjords are comprised of small icebergs that were not detected in previously-available coarser resolution satellite images. We show that the preponderance of small icebergs results in high freshwater delivery, as well as a short life span of icebergs in fjords. We conclude that an inability to identify small icebergs leads to inaccurate frequency-size distribution of icebergs in Greenland fjords, an underestimation of iceberg area (specifically for small icebergs), and an overestimation of iceberg life span.NASA grant NNX16AJ90GNSF grant PLR-154353

The non-syndromic familial thoracic aortic aneurysms and dissections maps to 15q21 locus

Background: Thoracic aortic aneurysms and dissections (TAAD) is a critical condition that often goes undiagnosed with fatal consequences. While majority of the cases are sporadic, more than 20 are inherited as a single gene disorder. The most common familial TAA is Marfan syndrome (MFS), which is primarily caused by mutations in fibrillin-1 (FBN1) gene. Patients with FBN1 mutations are at higher risk for dissection compared to other patients with similar size aneurysms. Methods: Fifteen family members were genotyped using Affymetrix-10K genechips. A genome-wide association study was carried out using an autosomal dominant model of inheritance with incomplete penetrance. Mutation screening of all exons and exon-intron boundaries of FBN1 gene which reside near the peak Lod score was carried out by direct sequencing.Results: The index case presented with agonizing substernal pain and was found to have TAAD by transthoracic echocardiogram. The family history was significant for 3 first degree relatives with TAA. Nine additional family members were diagnosed with TAA by echocardiography examinations. The affected individuals had no syndromic features. A genome-wide analysis of linkage mapped the disease gene to a single locus on chromosome 15q21 with a peak Lod score of 3.6 at fibrillin-1 (FBN1) gene locus (odds ratio > 4000:1 in favour of linkage), strongly suggesting that FBN1 is the causative gene. No mutation was identified within the exons and exon-intron boundaries of FBN1 gene that segregated with the disease. Haplotype analysis identified additional mutation carriers who had previously unknown status due to borderline dilation of the ascending aorta.Conclusions: A familial non-syndromic TAAD is strongly associated with the FBN1 gene locus and has a malignant disease course often seen in MFS patients. This finding indicates the importance of obtaining detailed family history and echocardiographic screening of extended relatives of patients with non-syndromic TAAD to improve the outcome. In addition, association of non-syndromic TAAD with the Marfan disease gene locus poses the question whether secondary prevention strategies employed for Marfan syndrome patients should be applied to all patients with familial TAAD. © 2010 Keramati et al; licensee BioMed Central Ltd

Pelvic floor dysfunction and polycystic ovary syndrome

Objectives: To compare the prevalence of pelvic floor muscle dysfunction (PFMD) in patients with and without polycystic ovary syndrome (PCOS); to test PFMD in women with different PCOS phenotypes. Methods: This was a case-control study of 202 women who were recruited in an infertility clinic in Hormozgan, Iran: PCOS (n=103) and control groups who were healthy women whose husbands were diagnosed with male infertility (n=99). According to the presence or absence of menstrual dysfunction (M), hyperandrogenism (HA) and polycystic ovaries on ultrasonoghraphy (PCO), patients with PCOS were divided into three phenotypes: HA+PCO, M+PCO and M+HA+PCO. PFMD was assessed by the Pelvic Floor Distress Inventory-20 (PFDI-20. Results: The reported PFMD symptoms were higher in PCOS (P=0.05) than the non-PCOS group. The mean PFDI score in the HA+M+PCO was higher compared to other phenotypes, although the difference did not reach significance level (P>0.05). The mean LH level was higher in HA+M+PCO than the two other phenotypes. There was a significant positive correlation between LH level and PFDI score (P<0.04). Conclusion: The findings suggest that a high level of LH may cause PFMD. Further studies are needed to determine the precise role of LH levels and potential treatment options in women with PCOS and PFMD. Keywords: polycystic ovary syndrome, pelvic floor distress inventory, pelvic floor muscle dysfunctio

Genome sequencing unveils a regulatory landscape of platelet reactivity

Platelet aggregation at the site of atherosclerotic vascular injury is the underlying pathophysiology of myocardial infarction and stroke. To build upon prior GWAS, here we report on 16 loci identified through a whole genome sequencing (WGS) approach in 3,855 NHLBI Trans-Omics for Precision Medicine (TOPMed) participants deeply phenotyped for platelet aggregation. We identify the RGS18 locus, which encodes a myeloerythroid lineage-specific regulator of G-protein signaling that co-localizes with expression quantitative trait loci (eQTL) signatures for RGS18 expression in platelets. Gene-based approaches implicate the SVEP1 gene, a known contributor of coronary artery disease risk. Sentinel variants at RGS18 and PEAR1 are associated with thrombosis risk and increased gastrointestinal bleeding risk, respectively. Our WGS findings add to previously identified GWAS loci, provide insights regarding the mechanism(s) by which genetics may influence cardiovascular disease risk, and underscore the importance of rare variant and regulatory approaches to identifying loci contributing to complex phenotypes

TVOCs and BTEX concentrations in the air of south pars special economic energy zone

cold season were higher than those in warm season. High concentrations of Benzene in cold and warm seasons were used to identify areas of high exposure risk.survey TVOCs and BTEX in the air of South Pars Special Economic Energy Zone in 2014. Materials and methods: In a cross-sectional study sampling and analysis was done by NIOSH 1501 method. The study was carried out in 336 activated carbon tubes and personal sampling pump in 6 sampling stations during one year. The compounds were extracted by solvent carbon disulfide and analyzed using Gas Chromatography- Flame Ionization Detector (GC-FID). Data analysis was performed in SPSS Ver.18 applying Kruskal-Wallis, Fligner test and ANOVA. Results: The mean concentrations of TVOCs and TBTEX were 229.34 and 31.23 µg/m3 in cold season and 212.19 and 29.89 µg/m3 in warm season, respectively. The mean concentrations of Benzene in all stations were 11.72 µg/m3 which were higher than the threshold levels recommended by Iranian Clean Air Act and US Environmental Protection Agency (USEPA). The ANOVA results showed a significant difference between the concentration of pollutants and hour, month and sampling stations (P&lt;0.05), but no significant difference was found between the concentration of pollutants and seasons (P&gt;0.05). Conclusion: The concentrations of measured pollutants in cold season were higher than those in warm season. High concentrations of Benzene in cold and warm seasons were used to identify areas of high exposure risk. © 2016, AMazandaran University of Medical Sciences. All rights reserved

Retrospective model-based inference guides model-free credit assignment

An extensive reinforcement learning literature shows that organisms assign credit efficiently, even under conditions of state uncertainty. However, little is known about credit-assignment when state uncertainty is subsequently resolved. Here, we address this problem within the framework of an interaction between model-free (MF) and model-based (MB) control systems. We present and support experimentally a theory of MB retrospective-inference. Within this framework, a MB system resolves uncertainty that prevailed when actions were taken thus guiding an MF credit-assignment. Using a task in which there was initial uncertainty about the lotteries that were chosen, we found that when participants’ momentary uncertainty about which lottery had generated an outcome was resolved by provision of subsequent information, participants preferentially assigned credit within a MF system to the lottery they retrospectively inferred was responsible for this outcome. These findings extend our knowledge about the range of MB functions and the scope of system interactions