Dynamic allocation of limited memory resources in reinforcement learning

Biological brains are inherently limited in their capacity to process and store information, but are nevertheless capable of solving complex tasks with apparent ease. Intelligent behavior is related to these limitations, since resource constraints drive the need to generalize and assign importance differentially to features in the environment or memories of past experiences. Recently, there have been parallel efforts in reinforcement learning and neuroscience to understand strategies adopted by artificial and biological agents to circumvent limitations in information storage. However, the two threads have been largely separate. In this article, we propose a dynamical framework to maximize expected reward under constraints of limited resources, which we implement with a cost function that penalizes precise representations of action-values in memory, each of which may vary in its precision. We derive from first principles an algorithm, Dynamic Resource Allocator (DRA), which we apply to two standard tasks in reinforcement learning and a model-based planning task, and find that it allocates more resources to items in memory that have a higher impact on cumulative rewards. Moreover, DRA learns faster when starting with a higher resource budget than what it eventually allocates for performing well on tasks, which may explain why frontal cortical areas in biological brains appear more engaged in early stages of learning before settling to lower asymptotic levels of activity. Our work provides a normative solution to the problem of learning how to allocate costly resources to a collection of uncertain memories in a manner that is capable of adapting to changes in the environment.Peer reviewe

Dementia Presenting with Hypersexuality: A Case Report

Background : This case report highlights the issue of hyper sexuality as a presenting feature in people with dementia and outlines the possible etiology and challenges associated with inappropriate sexual behavior in dementia. A case of a 75-year-old female with vascular dementia who developed hyper sexuality as a presenting complaint is reported here. The management of this patient included a combination of pharmacological and psychosocial intervention leading to the resolution of her inappropriate sexual behavior and improvement in her relationship with her family members. Inappropriate sexual behaviors need to be recognized and managed without compromising the fulfillment of the human′s basic need of sexuality

Probing the Salt Concentration Dependent Nucleobase Distribution in a Single-Stranded DNA–Single-Walled Carbon Nanotube Hybrid with Molecular Dynamics

The hybrids of single-walled carbon nanotube (SWCNT) and single stranded DNA (ssDNA) are novel nanoscale materials having remarkable applications in nanotechnology. The absorption of nucleobases on the surface of a SWCNT depends strongly on the ionic strength of the medium. In this paper, using atomistic molecular dynamics we have shown that at low salt concentration ssDNA wraps on the surface of SWCNT through hydrophobic π–π stacking between the DNA bases and the sp²-hybridized carbon atoms of the carbon nanotube. At high salt concentration, however, the DNA molecule adopts a partially folded structure and the ssDNA–SWCNT wrapping gets weakened significantly due to the self-stacking of the DNA bases. Our study can find relevance in CNT mediated gene delivery processes where subsequent unwrapping of the gene from its carrier is anticipated across the cell membrane regulated by an existing salt concentration gradient

Non-invasive molecularly-specific millimeter-resolution manipulation of brain circuits by ultrasound-mediated aggregation and uncaging of drug carriers

Non-invasive, molecularly-specific, focal modulation of brain circuits with low off-target effects can lead to breakthroughs in treatments of brain disorders. We systemically inject engineered ultrasound-controllable drug carriers and subsequently apply a novel two-component Aggregation and Uncaging Focused Ultrasound Sequence (AU-FUS) at the desired targets inside the brain. The first sequence aggregates drug carriers with millimeter-precision by orders of magnitude. The second sequence uncages the carrier’s cargo locally to achieve high target specificity without compromising the blood-brain barrier (BBB). Upon release from the carriers, drugs locally cross the intact BBB. We show circuit-specific manipulation of sensory signaling in motor cortex in rats by locally concentrating and releasing a GABAA receptor agonist from ultrasound-controlled carriers. Our approach uses orders of magnitude (1300x) less drug than is otherwise required by systemic injection and requires very low ultrasound pressures (20-fold below FDA safety limits for diagnostic imaging). We show that the BBB remains intact using passive cavitation detection (PCD), MRI-contrast agents and, importantly, also by sensitive fluorescent dye extravasation and immunohistochemistry.ISSN:2041-172

Optimal policy for multi-alternative decisions

Every-day decisions frequently require choosing among multiple alternatives. Yet, the optimal policy for such decisions is unknown. Here we derive the normative policy for general multi-alternative decisions. This strategy requires evidence accumulation to nonlinear, time-dependent bounds, that trigger choices. A geometric symmetry in those boundaries allows the optimal strategy to be implemented by a simple neural circuit involving a normalization with fixed decision bounds and an urgency signal. The model captures several key features of the response of decision-making neurons as well as the increase in reaction time as a function of the number of alternatives, known as Hick's law. In addition, we show that, in the presence of divisive normalization and internal variability, our model can account for several so called ‘irrational' behaviors such as the similarity effect as well as the violation of both the independent irrelevant alternative principle and the regularity principle