A neural network model of adaptively timed reinforcement learning and hippocampal dynamics

A neural model is described of how adaptively timed reinforcement learning occurs. The adaptive timing circuit is suggested to exist in the hippocampus, and to involve convergence of dentate granule cells on CA3 pyramidal cells, and NMDA receptors. This circuit forms part of a model neural system for the coordinated control of recognition learning, reinforcement learning, and motor learning, whose properties clarify how an animal can learn to acquire a delayed reward. Behavioral and neural data are summarized in support of each processing stage of the system. The relevant anatomical sites are in thalamus, neocortex, hippocampus, hypothalamus, amygdala, and cerebellum. Cerebellar influences on motor learning are distinguished from hippocampal influences on adaptive timing of reinforcement learning. The model simulates how damage to the hippocampal formation disrupts adaptive timing, eliminates attentional blocking, and causes symptoms of medial temporal amnesia. It suggests how normal acquisition of subcortical emotional conditioning can occur after cortical ablation, even though extinction of emotional conditioning is retarded by cortical ablation. The model simulates how increasing the duration of an unconditioned stimulus increases the amplitude of emotional conditioning, but does not change adaptive timing; and how an increase in the intensity of a conditioned stimulus "speeds up the clock", but an increase in the intensity of an unconditioned stimulus does not. Computer simulations of the model fit parametric conditioning data, including a Weber law property and an inverted U property. Both primary and secondary adaptively timed conditioning are simulated, as are data concerning conditioning using multiple interstimulus intervals (ISIs), gradually or abruptly changing ISis, partial reinforcement, and multiple stimuli that lead to time-averaging of responses. Neurobiologically testable predictions are made to facilitate further tests of the model.Air Force Office of Scientific Research (90-0175, 90-0128); Defense Advanced Research Projects Agency (90-0083); National Science Foundation (IRI-87-16960); Office of Naval Research (N00014-91-J-4100

The hippocampus and cerebellum in adaptively timed learning, recognition, and movement

The concepts of declarative memory and procedural memory have been used to distinguish two basic types of learning. A neural network model suggests how such memory processes work together as recognition learning, reinforcement learning, and sensory-motor learning take place during adaptive behaviors. To coordinate these processes, the hippocampal formation and cerebellum each contain circuits that learn to adaptively time their outputs. Within the model, hippocampal timing helps to maintain attention on motivationally salient goal objects during variable task-related delays, and cerebellar timing controls the release of conditioned responses. This property is part of the model's description of how cognitive-emotional interactions focus attention on motivationally valued cues, and how this process breaks down due to hippocampal ablation. The model suggests that the hippocampal mechanisms that help to rapidly draw attention to salient cues could prematurely release motor commands were not the release of these commands adaptively timed by the cerebellum. The model hippocampal system modulates cortical recognition learning without actually encoding the representational information that the cortex encodes. These properties avoid the difficulties faced by several models that propose a direct hippocampal role in recognition learning. Learning within the model hippocampal system controls adaptive timing and spatial orientation. Model properties hereby clarify how hippocampal ablations cause amnesic symptoms and difficulties with tasks which combine task delays, novelty detection, and attention towards goal objects amid distractions. When these model recognition, reinforcement, sensory-motor, and timing processes work together, they suggest how the brain can accomplish conditioning of multiple sensory events to delayed rewards, as during serial compound conditioning.Air Force Office of Scientific Research (F49620-92-J-0225, F49620-86-C-0037, 90-0128); Advanced Research Projects Agency (ONR N00014-92-J-4015); Office of Naval Research (N00014-91-J-4100, N00014-92-J-1309, N00014-92-J-1904); National Institute of Mental Health (MH-42900

Major improvements to the Heliconius melpomene genome assembly used to confirm 10 chromosome fusion events in 6 million years of butterfly evolution

The Heliconius butterflies are a widely studied adaptive radiation of 46 species spread across Central and South America, several of which are known to hybridize in the wild. Here, we present a substantially improved assembly of the Heliconius melpomene genome, developed using novel methods that should be applicable to improving other genome assemblies produced using short read sequencing. First, we whole-genome-sequenced a pedigree to produce a linkage map incorporating 99% of the genome. Second, we incorporated haplotype scaffolds extensively to produce a more complete haploid version of the draft genome. Third, we incorporated ~20x coverage of Pacific Biosciences sequencing, and scaffolded the haploid genome using an assembly of this long-read sequence. These improvements result in a genome of 795 scaffolds, 275 Mb in length, with an N50 length of 2.1 Mb, an N50 number of 34, and with 99% of the genome placed, and 84% anchored on chromosomes. We use the new genome assembly to confirm that the Heliconius genome underwent 10 chromosome fusions since the split with its sister genus Eueides, over a period of about 6 million yr

Pacific Atmospheric Sulfur Experiment (PASE): dynamics and chemistry of the south Pacific tropical trade wind regime

The Pacific Atmospheric Sulfur Experiment (PASE) was a comprehensive airborne study of the chemistry and dynamics of the tropical trade wind regime (TWR) east of the island of Kiritibati (Christmas Island, 157º, 20′ W, 2º 52′ N). Christmas Island is located due south of Hawaii. Geographically it is in the northern hemisphere yet it is 6–12º south of the intertropical convergence zone (ITCZ) which places it in the southern hemisphere meteorologically. Christmas Island trade winds in August and September are from east south east at 3–15 ms−1. Clouds, if present, are fair weather cumulus located in the middle layer of the TWR which is frequently labeled the buffer layer (BuL). PASE provided clear support for the idea that small particles (80 nm) were subsiding into the tropical trade wind regime (TWR) where sulfur chemistry transformed them to larger particles. Sulfur chemistry promoted the growth of some of these particles until they were large enough to activate to cloud drops. This process, promoted by sulfur chemistry, can produce a cooling effect due to the increase in cloud droplet density and changes in cloud droplet size. These increases in particle size observed in PASE promote additional cooling due to direct scattering from the aerosol. These potential impacts on the radiation balance in the TWR are enhanced by the high solar irradiance and ocean albedo of the TWR. Finally because of the large area involved there is a large factional impact on earth’s radiation budget. The TWR region near Christmas Island appears to be similar to the TWR that persists in August and September, from southwest of the Galapagos to at least Christmas Island. Transport in the TWR between the Galapagos and Christmas involves very little precipitation which could have removed the aerosol thus explaining at least in part the high concentrations of CCN (≈300 at 0.5% supersaturation) observed in PASE. As expected the chemistry of sulfur in the trade winds was found to be initiated by the emission of DMS into the convective boundary layer (BL, the lowest of three layers). However, the efficiency with which this DMS is converted to SO2 has been brought into further question by this study. This unusual result has come about as result of our using two totally different approaches for addressing this long standing question. In the first approach, based on accepted kinetic rate constants and detailed steps for the oxidation of DMS reflecting detailed laboratory studies, a DMS to SO2 conversion efficiency of 60–73% was determined. This range of values lies well within the uncertainties of previous studies. However, using a completely different approach, involving a budget analysis, a conversion value of 100% was estimated. The latter value, to be consistent with all other sulfur studies, requires the existence of a completely independent sulfur source which would emit into the atmosphere at a source strength approximately half that measured for DMS under tropical Pacific conditions. At this time, however, there is no credible scientific observation that identifies what this source might be. Thus, the current study has opened for future scientific investigation the major question: is there yet another major tropical marine source of sulfur? Of equal importance, then, is the related question, is our global sulfur budget significantly in error due to the existence of an unknown marine source of sulfur? Pivotal to both questions may be gaining greater insight about the intermediate DMS oxidation species, DMSO, for which rather unusual measurements have been reported in previous marine sulfur studies. The 3 pptv bromine deficit observed in PASE must be lost over the lifetime of the aerosol which is a few days. This observation suggests that the primary BrO production rate is very small. However, considering the uncertainties in these observations and the possible importance of secondary production of bromine radicals through aerosol surface reactions, to completely rule out the importance of bromine chemistry under tropical conditions at this time cannot be justified. This point has been brought into focus from prior work that even at levels of 1 pptv, the effect of BrO oxidation on DMS can still be quite significant. Thus, as in the case of DMS conversion to SO2, future studies will be needed. In the latter case there will need to be a specific focus on halogen chemistry. Such studies clearly must involve specific measurements of radical species such as BrO

No evidence for maintenance of a sympatric Heliconius species barrier by chromosomal inversions.

Mechanisms that suppress recombination are known to help maintain species barriers by preventing the breakup of coadapted gene combinations. The sympatric butterfly species Heliconius melpomene and Heliconius cydno are separated by many strong barriers, but the species still hybridize infrequently in the wild, and around 40% of the genome is influenced by introgression. We tested the hypothesis that genetic barriers between the species are maintained by inversions or other mechanisms that reduce between-species recombination rate. We constructed fine-scale recombination maps for Panamanian populations of both species and their hybrids to directly measure recombination rate within and between species, and generated long sequence reads to detect inversions. We find no evidence for a systematic reduction in recombination rates in F1 hybrids, and also no evidence for inversions longer than 50 kb that might be involved in generating or maintaining species barriers. This suggests that mechanisms leading to global or local reduction in recombination do not play a significant role in the maintenance of species barriers between H. melpomene and H. cydno

Simulation in coronary artery anastomosis early in cardiothoracic surgical residency training: The Boot Camp experience

We evaluated focused training in coronary artery anastomosis with a porcine heart model and portable task station

Hereditary transthyretin amyloidosis: baseline characteristics of patients in the NEURO-TTR trial

Background: Hereditary transthyretin (ATTRm) amyloidosis is a rare, progressive and fatal disease with a range of clinical manifestations.Objective: This study comprehensively evaluates disease characteristics in a large, diverse cohort of patients with ATTRm amyloidosis.Methods: Adult patients (N = 172) with Stage 1 or Stage 2 ATTRm amyloidosis who had polyneuropathy were screened and enrolled across 24 investigative sites and 10 countries in the NEURO-TTR trial (www.clinicaltrials.gov, NCT01737398). Medical and disease history, quality of life, laboratory data, and clinical assessments were analyzed.Results: The NEURO-TTR patient population was diverse in age, disease severity, TTR mutation, and organ involvement. Twenty-seven different TTR mutations were present, with Val30Met being the most common (52%). One third of patients reported early onset disease (before age 50) and the average duration of neuropathy symptoms was 5.3 years. Symptoms affected multiple organs and systems, with nearly 70% of patients exhibiting broad involvement of weakness, sensory loss, and autonomic disturbance. Over 60% of patients had cardiomyopathy, with highest prevalence in the United States (72%) and lowest in South America/Australasia (33%). Cardiac biomarker NT-proBNP correlated with left ventricular wall thickness (p<.001). Quality of life, measured by Norfolk QoL-DN and SF-36 patient-reported questionnaires, was significantly impaired and correlated with disease severity.Conclusions: Baseline data from the NEURO-TTR trial demonstrates ATTRm amyloidosis as a systemic disease with deficits in multiple organs and body systems, leading to decreased quality of life. We report concomitant presentation of polyneuropathy and cardiomyopathy in most patients, and early involvement of multiple body systems

An Analysis of the Systemic Risks Posed by Fannie Mae and Freddie Mac and an Evaluation of the Policy Options for Reducing those Risks

Fannie Mae and Freddie Mac are government-sponsored enterprises that are central players in U.S. secondary mortgage markets. Over the past decade, these institutions have amassed enormous mortgage- and non-mortgage-oriented investment portfolios that pose significant interest-rate risks to the companies and a systemic risk to the financial system. This paper describes the nature of these risks and systemic concerns and then evaluates several policy options for reducing the institutions’ investment portfolios. We conclude that limits on portfolio size (assets or liabilities) would be the most desirable approach to mitigating the systemic risk posed by Fannie Mae and Freddie Mac