65 research outputs found
Predecessor Features
Any reinforcement learning system must be able to identify which past events
contributed to observed outcomes, a problem known as credit assignment. A
common solution to this problem is to use an eligibility trace to assign credit
to recency-weighted set of experienced events. However, in many realistic
tasks, the set of recently experienced events are only one of the many possible
action events that could have preceded the current outcome. This suggests that
reinforcement learning can be made more efficient by allowing credit assignment
to any viable preceding state, rather than only those most recently
experienced. Accordingly, we examine ``Predecessor Features'', the fully
bootstrapped version of van Hasselt's ``Expected Trace'', an algorithm that
achieves this richer form of credit assignment. By maintaining a representation
that approximates the expected sum of past occupancies, this algorithm allows
temporal difference (TD) errors to be propagated accurately to a larger number
of predecessor states than conventional methods, greatly improving learning
speed. The algorithm can also be naturally extended from tabular state
representation to feature representations allowing for increased performance on
a wide range of environments. We demonstrate several use cases for Predecessor
Features and compare its performance with other approaches.Comment: Accepted to RLDM 202
Emergence and reconfiguration of modular structure for synaptic neural networks during continual familiarity detection
While advances in artificial intelligence and neuroscience have enabled the
emergence of neural networks capable of learning a wide variety of tasks, our
understanding of the temporal dynamics of these networks remains limited. Here,
we study the temporal dynamics during learning of Hebbian Feedforward (HebbFF)
neural networks in tasks of continual familiarity detection. Drawing
inspiration from the field of network neuroscience, we examine the network's
dynamic reconfiguration, focusing on how network modules evolve throughout
learning. Through a comprehensive assessment involving metrics like network
accuracy, modular flexibility, and distribution entropy across diverse learning
modes, our approach reveals various previously unknown patterns of network
reconfiguration. In particular, we find that the emergence of network
modularity is a salient predictor of performance, and that modularization
strengthens with increasing flexibility throughout learning. These insights not
only elucidate the nuanced interplay of network modularity, accuracy, and
learning dynamics but also bridge our understanding of learning in artificial
and biological realms
Adaptation decorrelates shape representations
Perception and neural responses are modulated by sensory history. Visual adaptation, an example of such an effect, has been hypothesized to improve stimulus discrimination by decorrelating responses across a set of neural units. While a central theoretical model, behavioral and neural evidence for this theory is limited and inconclusive. Here, we use a parametric 3D shape-space to test whether adaptation decorrelates shape representations in humans. In a behavioral experiment with 20 subjects, we find that adaptation to a shape class improves discrimination of subsequently presented stimuli with similar features. In a BOLD fMRI experiment with 10 subjects, we observe that adaptation to a shape class decorrelates the multivariate representations of subsequently presented stimuli with similar features in object-selective cortex. These results support the long-standing proposal that adaptation improves perceptual discrimination and decorrelates neural representations, offering insights into potential underlying mechanisms
Recommended from our members
Functional Network Dynamics of the Language System
During linguistic processing, a set of brain regions on the lateral surfaces of the left frontal, temporal, and parietal cortices exhibit robust responses. These areas display highly correlated activity while a subject rests or performs a naturalistic language comprehension task, suggesting that they form an integrated functional system. Evidence suggests that this system is spatially and functionally distinct from other systems that support high-level cognition in humans. Yet, how different regions within this system might be recruited dynamically during task performance is not well understood. Here we use network methods, applied to fMRI data collected from 22 human subjects performing a language comprehension task, to reveal the dynamic nature of the language system. We observe the presence of a stable core of brain regions, predominantly located in the left hemisphere, that consistently coactivate with one another. We also observe the presence of a more flexible periphery of brain regions, predominantly located in the right hemisphere, that coactivate with different regions at different times. However, the language functional ROIs in the angular gyrus and the anterior temporal lobe were notable exceptions to this trend. By highlighting the temporal dimension of language processing, these results suggest a trade-off between a region's specialization and its capacity for flexible network reconfiguration
Recommended from our members
The validity and reliability of the CAMDEX-DS for assessing dementia in adults with Down syndrome in Brazil.
OBJECTIVE: Alzheimer's disease occurs at a higher prevalence and an earlier age in individuals with Down syndrome (DS) than typically developing individuals. However, diagnosing dementia in individuals with intellectual disability remains a challenge due to pre-existing cognitive deficits. The aim of this study was to investigate the validity and reliability of the Brazilian version of the Cambridge Examination for Mental Disorders of Older People with Down's syndrome and Others with Intellectual Disabilities (CAMDEX-DS) for individuals with DS. METHODS: Two psychiatrists, working independently, evaluated 92 adults with DS ≥ 30 years of age. The concurrent validity of the CAMDEX-DS was analyzed in relation to the gold standard of established international criteria. In a subgroup of 20 subjects, the concurrent validity of the CAMDEX-DS was analyzed in relation to an independent objective assessment of cognitive decline over three years. We analyzed the inter-rater reliability of cognitive assessment. RESULTS: The diagnostic accuracy of the CAMDEX-DS compared to the gold standard was 96.7%. CAMDEX-DS-based diagnosis was considered consistent with cognitive decline. The probability of a participant with dementia having cognitive decline was 83%. Inter-rater reliability for the participant assessment was good, with a kappa of > 0.8 for 93% of the CAMDEX-DS items. CONCLUSION: The CAMDEX-DS can be considered the first valid and reliable instrument for evaluating dementia in adults with DS in Brazil. Its use in such individuals could improve clinical practice and research
The validity and reliability of the CAMDEX-DS for assessing dementia in adults with Down syndrome in Brazil.
OBJECTIVE: Alzheimer's disease occurs at a higher prevalence and an earlier age in individuals with Down syndrome (DS) than typically developing individuals. However, diagnosing dementia in individuals with intellectual disability remains a challenge due to pre-existing cognitive deficits. The aim of this study was to investigate the validity and reliability of the Brazilian version of the Cambridge Examination for Mental Disorders of Older People with Down's syndrome and Others with Intellectual Disabilities (CAMDEX-DS) for individuals with DS. METHODS: Two psychiatrists, working independently, evaluated 92 adults with DS ≥ 30 years of age. The concurrent validity of the CAMDEX-DS was analyzed in relation to the gold standard of established international criteria. In a subgroup of 20 subjects, the concurrent validity of the CAMDEX-DS was analyzed in relation to an independent objective assessment of cognitive decline over three years. We analyzed the inter-rater reliability of cognitive assessment. RESULTS: The diagnostic accuracy of the CAMDEX-DS compared to the gold standard was 96.7%. CAMDEX-DS-based diagnosis was considered consistent with cognitive decline. The probability of a participant with dementia having cognitive decline was 83%. Inter-rater reliability for the participant assessment was good, with a kappa of > 0.8 for 93% of the CAMDEX-DS items. CONCLUSION: The CAMDEX-DS can be considered the first valid and reliable instrument for evaluating dementia in adults with DS in Brazil. Its use in such individuals could improve clinical practice and research
- …