Adaptive patch foraging in deep reinforcement learning agents

Patch foraging is one of the most heavily studied behavioral optimization challenges in biology. However, despite its importance to biological intelligence, this behavioral optimization problem is understudied in artificial intelligence research. Patch foraging is especially amenable to study given that it has a known optimal solution, which may be difficult to discover given current techniques in deep reinforcement learning. Here, we investigate deep reinforcement learning agents in an ecological patch foraging task. For the first time, we show that machine learning agents can learn to patch forage adaptively in patterns similar to biological foragers, and approach optimal patch foraging behavior when accounting for temporal discounting. Finally, we show emergent internal dynamics in these agents that resemble single-cell recordings from foraging non-human primates, which complements experimental and theoretical work on the neural mechanisms of biological foraging. This work suggests that agents interacting in complex environments with ecologically valid pressures arrive at common solutions, suggesting the emergence of foundational computations behind adaptive, intelligent behavior in both biological and artificial agents.Comment: Published in Transactions on Machine Learning Research (TMLR). See: https://openreview.net/pdf?id=a0T3nOP9s

Scintigraphic assessment of sympathetic innervation after transmural versus nontransmural myocardial infarction

To evaluate the feasibility of detecting denervated myocardium in the infarcted canine heart, the distribution of sympathetic nerve endings using 1–123 metaiodobenzylguanidine (MIBG) was compared with the distribution of perfusion using thallium-201, with the aid of color-coded computer functional map in 16 dogs. Twelve dogs underwent myocardial infarction by injection of vinyl latex into the left anterior descending coronary artery (transmural myocardial infarction, n = 6), or ligation of the left anterior descending coronary artery (nontransmural myocardial infarction, n = 6). Four dogs served as sham-operated controls. Image patterns were compared with tissue norepinephrine content and with histofluorescence microscopic findings in biopsy specimens.Hearts with transmural infarction showed zones of absent MIBG and thallium, indicating scar. Adjacent and distal regions showed reduced MIBG but normal thallium uptake, indicating viable but denervated myocardium. Denervation distal to infarction was confirmed by reduced norepinephrine content and absence of nerve fluorescence. Nontransmural myocardial infarction showed zones of wall thinning with decreased thallium uptake and a greater reduction or absence of MIBG localized to the region of the infarct, with minimal extension of denervation beyond the infarct. Norepinephrine content was significantly reduced in the infarct zone, and nerve fluorescence was absent.These findings suggest that 1) MIBG imaging can detect viable and perfused but denervated myocardium after infarction; and 2) as opposed to the distal denervation produced by transmural infarction, nontransmural infarction may lead to regional ischemic damage of sympathetic nerves, but may spare subepicardial nerve trunks that course through the region of infarction to provide a source of innervation to distal areas of myocardium

The variable functional effects of the pacing site in normal and scarred ventricles

The pacing site has been shown to influence functional improvement with cardiac resynchronization therapy. We evaluated the effects of the pacing site on left ventricular (LV) function in an animal model. Equilibrium radionuclide angiography was acquired in sinus rhythm (NSR) and with ventricular pacing, from three pacing sites in seven normal and eight infarcted dogs. QRS duration, electrical activation pattern, wall motion, LV ejection fraction (EF), synchrony of ventricular contraction, and mean arterial pressure (MAP), were related to the pacing site and infarct size, during each of 120 episodes. Little changed during pacing in normals. In infarcted dogs, LV wall motion, and synchrony worsened, LVEF and MAP often fell. These changes related to altered activation patterns which were influenced by the pacing site but were not related to infarct size. Hemodynamic and functional LV changes after infarction were found to vary with the pacing site and associated conduction and synchrony

Chromosome Tips Damaged in Anaphase Inhibit Cytokinesis

Genome maintenance is ensured by a variety of biochemical sensors and pathways that repair accumulated damage. During mitosis, the mechanisms that sense and resolve DNA damage remain elusive. Studies have demonstrated that damage accumulated on lagging chromosomes can activate the spindle assembly checkpoint. However, there is little known regarding damage to DNA after anaphase onset. In this study, we demonstrate that laser-induced damage to chromosome tips (presumptive telomeres) in anaphase of Potorous tridactylis cells (PtK2) inhibits cytokinesis. In contrast, equivalent irradiation of non-telomeric chromosome regions or control irradiations in either the adjacent cytoplasm or adjacent to chromosome tips near the spindle midzone during anaphase caused no change in the eventual completion of cytokinesis. Damage to only one chromosome tip caused either complete absence of furrow formation, a prolonged delay in furrow formation, or furrow regression. When multiple chromosome tips were irradiated in the same cell, the cytokinesis defects increased, suggesting a potential dose-dependent mechanism. These results suggest a mechanism in which dysfunctional telomeres inhibit mitotic exit

Trial-by-Trial Changes in a Priori Informational Value of External Cues and Subjective Expectancies in Human Auditory Attention

Background: Preparatory activity based on a priori probabilities generated in previous trials and subjective expectancies would produce an attentional bias. However, preparation can be correct (valid) or incorrect (invalid) depending on the actual target stimulus. The alternation effect refers to the subjective expectancy that a target will not be repeated in the same position, causing RTs to increase if the target location is repeated. The present experiment, using the Posner’s central cue paradigm, tries to demonstrate that not only the credibility of the cue, but also the expectancy about the next position of the target are changedin a trial by trial basis. Sequences of trials were analyzed. Results: The results indicated an increase in RT benefits when sequences of two and three valid trials occurred. The analysis of errors indicated an increase in anticipatory behavior which grows as the number of valid trials is increased. On the other hand, there was also an RT benefit when a trial was preceded by trials in which the position of the target changed with respect to the current trial (alternation effect). Sequences of two alternations or two repetitions were faster than sequences of trials in which a pattern of repetition or alternation is broken. Conclusions: Taken together, these results suggest that in Posner’s central cue paradigm, and with regard to the anticipatory activity, the credibility of the external cue and of the endogenously anticipated patterns of target location are constantly updated. The results suggest that Bayesian rules are operating in the generation of anticipatory activity as