Beyond trial-by-trial adaptation : a quantification of the time scale of cognitive control

The idea that adaptation to stimulus or response conflict can operate over different time scales takes a prominent position in various theories and models of cognitive control. The mechanisms underlying temporal variations in control are nevertheless poorly understood, which is partly due to a lack of appropriate empirical measures. Inspired by reinforcement learning models, we developed a method to quantify the time scale of control behaviorally, by computing trial-by-trial effects that go beyond the preceding trial. Briefly, we extended the congruency sequence effect from 1 trial to multiple trials into the past and quantified the influence of previous trials on current-trial performance as a function of trial distance. The rate at which this influence changes across trials was taken as a measure of the time scale of control. We applied the method to a flanker task with different conflict frequencies and volatility. Results showed that the time scale of control was smaller in rare-conflict and volatile contexts, compared to frequent-conflict and neutral contexts. This is in agreement with theories differentiating transient from sustained control. The method offers new opportunities to reveal temporal differences in control modes and can easily be applied to various empirical paradigms. (PsycINFO Database Recordstatus: publishe

HyperPPO: A scalable method for finding small policies for robotic control

Models with fewer parameters are necessary for the neural control of memory-limited, performant robots. Finding these smaller neural network architectures can be time-consuming. We propose HyperPPO, an on-policy reinforcement learning algorithm that utilizes graph hypernetworks to estimate the weights of multiple neural architectures simultaneously. Our method estimates weights for networks that are much smaller than those in common-use networks yet encode highly performant policies. We obtain multiple trained policies at the same time while maintaining sample efficiency and provide the user the choice of picking a network architecture that satisfies their computational constraints. We show that our method scales well - more training resources produce faster convergence to higher-performing architectures. We demonstrate that the neural policies estimated by HyperPPO are capable of decentralized control of a Crazyflie2.1 quadrotor. Website: https://sites.google.com/usc.edu/hyperppoComment: Website: https://sites.google.com/usc.edu/hyperpp

Distributed cerebellar plasticity implements generalized multiple-scale memory components in real-robot sensorimotor tasks

The cerebellum plays a crucial role in motor learning and it acts as a predictive controller. Modeling it and embedding it into sensorimotor tasks allows us to create functional links between plasticity mechanisms, neural circuits and behavioral learning. Moreover, if applied to real-time control of a neurorobot, the cerebellar model has to deal with a real noisy and changing environment, thus showing its robustness and effectiveness in learning. A biologically inspired cerebellar model with distributed plasticity, both at cortical and nuclear sites, has been used. Two cerebellum-mediated paradigms have been designed: an associative Pavlovian task and a vestibulo-ocular reflex, with multiple sessions of acquisition and extinction and with different stimuli and perturbation patterns. The cerebellar controller succeeded to generate conditioned responses and finely tuned eye movement compensation, thus reproducing human-like behaviors. Through a productive plasticity transfer from cortical to nuclear sites, the distributed cerebellar controller showed in both tasks the capability to optimize learning on multiple time-scales, to store motor memory and to effectively adapt to dynamic ranges of stimuli.This work was supported by grants of European Union: REALNET (FP7-ICT270434) and Human Brain Project (HBP-604102)

Dopamine-modulated dynamic cell assemblies generated by the GABAergic striatal microcircuit

The striatum, the principal input structure of the basal ganglia, is crucial to both motor control and learning. It receives convergent input from all over the neocortex, hippocampal formation, amygdala and thalamus, and is the primary recipient of dopamine in the brain. Within the striatum is a GABAergic microcircuit that acts upon these inputs, formed by the dominant medium-spiny projection neurons (MSNs) and fast-spiking interneurons (FSIs). There has been little progress in understanding the computations it performs, hampered by the non-laminar structure that prevents identification of a repeating canonical microcircuit. We here begin the identification of potential dynamically-defined computational elements within the striatum. We construct a new three-dimensional model of the striatal microcircuit's connectivity, and instantiate this with our dopamine-modulated neuron models of the MSNs and FSIs. A new model of gap junctions between the FSIs is introduced and tuned to experimental data. We introduce a novel multiple spike-train analysis method, and apply this to the outputs of the model to find groups of synchronised neurons at multiple time-scales. We find that, with realistic in vivo background input, small assemblies of synchronised MSNs spontaneously appear, consistent with experimental observations, and that the number of assemblies and the time-scale of synchronisation is strongly dependent on the simulated concentration of dopamine. We also show that feed-forward inhibition from the FSIs counter-intuitively increases the firing rate of the MSNs. Such small cell assemblies forming spontaneously only in the absence of dopamine may contribute to motor control problems seen in humans and animals following a loss of dopamine cells. (C) 2009 Elsevier Ltd. All rights reserved

Multi-Scale Glycemic Variability: A Link to Gray Matter Atrophy and Cognitive Decline in Type 2 Diabetes

Objective: Type 2 diabetes mellitus (DM) accelerates brain aging and cognitive decline. Complex interactions between hyperglycemia, glycemic variability and brain aging remain unresolved. This study investigated the relationship between glycemic variability at multiple time scales, brain volumes and cognition in type 2 DM. Research Design and Methods Forty-three older adults with and 26 without type 2 DM completed 72-hour continuous glucose monitoring, cognitive tests and anatomical MRI. We described a new analysis of continuous glucose monitoring, termed Multi-Scale glycemic variability (Multi-Scale GV), to examine glycemic variability at multiple time scales. Specifically, Ensemble Empirical Mode Decomposition was used to identify five unique ultradian glycemic variability cycles (GVC1–5) that modulate serum glucose with periods ranging from 0.5–12 hrs. Results: Type 2 DM subjects demonstrated greater variability in GVC3–5 (period 2.0–12 hrs) than controls (P<0.0001), during the day as well as during the night. Multi-Scale GV was related to conventional markers of glycemic variability (e.g. standard deviation and mean glycemic excursions), but demonstrated greater sensitivity and specificity to conventional markers, and was associated with worse long-term glycemic control (e.g. fasting glucose and HbA1c). Across all subjects, those with greater glycemic variability within higher frequency cycles (GVC1–3; 0.5–2.0 hrs) had less gray matter within the limbic system and temporo-parietal lobes (e.g. cingulum, insular, hippocampus), and exhibited worse cognitive performance. Specifically within those with type 2 DM, greater glycemic variability in GVC2–3 was associated with worse learning and memory scores. Greater variability in GVC5 was associated with longer DM duration and more depression. These relationships were independent of HbA1c and hypoglycemic episodes. Conclusions: Type 2 DM is associated with dysregulation of glycemic variability over multiple scales of time. These time-scale-dependent glycemic fluctuations might contribute to brain atrophy and cognitive outcomes within this vulnerable population

Topological properties of inequality and deprivation in an educational system: Unveiling the key-drivers through complex network analysis

This research conceives an educational system as a complex network to incorporate a rich framework for analyzing topological and statistical proper-ties of inequality and learning deprivation at different levels, as well as to simu-late the structure, stability and fragility of the educational system. The model provides a natural way to represent educational phenomena, allowing to test public policies by computation before being implemented, bringing the oppor-tunity of calibrating control parameters for assessing order parameters over time in multiple territorial scales. This approach provides a set of unique advantages over classical analysis tools because it allows the use of large-scale assessments and other evidences for combining the richness of qualitative analysis with quantitative inferences for measuring inequality gaps. An additional advantage, as shown in our results using real data from a Latin American country, is to provide a solution to con-cerns about the limitations of case studies or isolated statistical approaches.info:eu-repo/semantics/acceptedVersio

SELF-REGULATION AND MATH ATTITUDES: EFFECTS ON ACADEMIC PERFORMANCE IN DEVELOPMENTAL MATH COURSES

The purpose of the study was to investigate the relationship among math attitudes, self-regulated learning, and course outcomes in developmental math. Math attitudes involved perceived usefulness of math and math anxiety. Self-regulated learning represented the ability of students to control cognitive, metacognitive, and behavioral aspects of learning. The sample consisted of 376 students who were enrolled in developmental math courses at a community college. Although participants perceived math as fairly relevant to their lives, they did not experience much math anxiety. Participants were somewhat likely to engage in self-regulated learning, but the rates were not particularly high. Of the five self-regulated learning scales (metacognitive self-regulation, effort regulation, environmental management, peer help, and study strategies), students were most likely to regulate their effort and structure their learning environment. Findings from independent samples t-tests, one-way analyses of variance, and correlation analyses highlighted differences in math attitudes, self-regulated learning, and math outcomes based on demographic variables. First generation and part-time college students and students with dependents perceived math as more useful than their counterparts. Continuing generation and part-time students experienced higher levels of math anxiety than first generation and full-time students. Students who were female, non-traditional aged, married or divorced/separated, and those who had dependents were more likely to engage in self-regulatory strategies than their peers. Multiple regression analyses were conducted to determine a) the influence of math attitudes on self-regulated learning and b) the influence of self-regulated learning on final course grades in developmental math. Results indicated that attitudes toward math significantly predicted self-regulated learning and that self-regulated learning significantly predicted final course grades. Students who used self-regulatory strategies earned higher grades in developmental math courses. The results have implications for educational policy and practice. Developmental education programs should include instruction on self-regulatory strategies and should consider supplementing cognitive assessment measures with non-cognitive factors in order to better predict readiness for college coursework and academic potential

On Parametric Optimal Execution and Machine Learning Surrogates

We investigate optimal order execution problems in discrete time with instantaneous price impact and stochastic resilience. First, in the setting of linear transient price impact we derive a closed-form recursion for the optimal strategy, extending the deterministic results from Obizhaeva and Wang (J Financial Markets, 2013). Second, we develop a numerical algorithm based on dynamic programming and deep learning for the case of nonlinear transient price impact as proposed by Bouchaud et al. (Quant. Finance, 2004). Specifically, we utilize an actor-critic framework that constructs two neural-network (NN) surrogates for the value function and the feedback control. The flexible scalability of NN functional approximators enables parametric learning, i.e., incorporating several model or market parameters as part of the input space. Precise calibration of price impact, resilience, etc., is known to be extremely challenging and hence it is critical to understand sensitivity of the execution policy to these parameters. Our NN learner organically scales across multiple input dimensions and is shown to accurately approximate optimal strategies across a wide range of parameter configurations. We provide a fully reproducible Jupyter Notebook with our NN implementation, which is of independent pedagogical interest, demonstrating the ease of use of NN surrogates in (parametric) stochastic control problems.Comment: 33 pages, 8 figures. Github repo at https://github.com/moritz-voss/Parametric_Optimal_Execution_M

Space, Time and Learning in the Hippocampus: How Fine Spatial and Temporal Scales Are Expanded into Population Codes for Behavioral Control

The hippocampus participates in multiple functions, including spatial navigation, adaptive timing, and declarative (notably, episodic) memory. How does it carry out these particular functions? The present article proposes that hippocampal spatial and temporal processing are carried out by parallel circuits within entorhinal cortex, dentate gyrus, and CA3 that are variations of the same circuit design. In particular, interactions between these brain regions transform fine spatial and temporal scales into population codes that are capable of representing the much larger spatial and temporal scales that are needed to control adaptive behaviors. Previous models of adaptively timed learning propose how a spectrum of cells tuned to brief but different delays are combined and modulated by learning to create a population code for controlling goal-oriented behaviors that span hundreds of milliseconds or even seconds. Here it is proposed how projections from entorhinal grid cells can undergo a similar learning process to create hippocampal place cells that can cover a space of many meters that are needed to control navigational behaviors. The suggested homology between spatial and temporal processing may clarify how spatial and temporal information may be integrated into an episodic memory.National Science Foundation (SBE-0354378); Office of Naval Research (N00014-01-1-0624