The Neural Race Reduction: Dynamics of Abstraction in Gated Networks

Our theoretical understanding of deep learning has not kept pace with its empirical success. While network architecture is known to be critical, we do not yet understand its effect on learned representations and network behavior, or how this architecture should reflect task this http URL this work, we begin to address this gap by introducing the Gated Deep Linear Network framework that schematizes how pathways of information flow impact learning dynamics within an architecture. Crucially, because of the gating, these networks can compute nonlinear functions of their input. We derive an exact reduction and, for certain cases, exact solutions to the dynamics of learning. Our analysis demonstrates that the learning dynamics in structured networks can be conceptualized as a neural race with an implicit bias towards shared representations, which then govern the model's ability to systematically generalize, multi-task, and transfer. We validate our key insights on naturalistic datasets and with relaxed assumptions. Taken together, our work gives rise to general hypotheses relating neural architecture to learning and provides a mathematical approach towards understanding the design of more complex architectures and the role of modularity and compositionality in solving real-world problems. The code and results are available at this https URL

Strategically managing learning during perceptual decision making

Making optimal decisions in the face of noise requires balancing short-term speed and accuracy. But a theory of optimality should account for the fact that short-term speed can influence long-term accuracy through learning. Here, we demonstrate that long-term learning is an important dynamical dimension of the speed-accuracy trade-off. We study learning trajectories in rats and formally characterize these dynamics in a theory expressed as both a recurrent neural network and an analytical extension of the drift-diffusion model that learns over time. The model reveals that choosing suboptimal response times to learn faster sacrifices immediate reward, but can lead to greater total reward. We empirically verify predictions of the theory, including a relationship between stimulus exposure and learning speed, and a modulation of reaction time by future learning prospects. We find that rats' strategies approximately maximize total reward over the full learning epoch, suggesting cognitive control over the learning process

Maslow’s Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation

Continual learning—learning new tasks in sequence while maintaining performance on old tasks—remains particularly challenging for artificial neural networks. Surprisingly, the amount of forgetting does not increase with the dissimilarity between the learned tasks, but appears to be worst in an intermediate similarity regime. In this paper we theoretically analyse both a synthetic teacher-student framework and a real data setup to provide an explanation of this phenomenon that we name Maslow’s Hammer hypothesis. Our analysis reveals the presence of a trade-off between node activation and node re-use that results in worst forgetting in the intermediate regime. Using this understanding we reinterpret popular algorithmic interventions for catastrophic interference in terms of this trade-off, and identify the regimes in which they are most effective

Safeguarding children in dentistry: 1. Child protection training, experience and practice of dental professionals with an interest in paediatric dentistry

* Few dental professionals with child protection training have experience of making referrals. * There is a wide gap in practice between recognising signs of child abuse and neglect and responding effectively. * This may indicate missed opportunities to save children from continuing abuse. * There is a need for improved child protection information, support and training for dental professionals. Abstract Following several highly publicised inquiries into the deaths of children from abuse and neglect, there has been much recent interest in the role and responsibility of all health professionals to protect children at risk of maltreatment. The findings of a postal questionnaire, sent in March 2005 to 789 dentists and dental care professionals with an interest in paediatric dentistry working in varied settings in the UK, are presented in a two-part report and discussed in the context of current multi-agency good practice in safeguarding and promoting the welfare of children. This first part explores reported child protection training, experience and practice. There was a significant gap between recognising signs of abuse and responding effectively: 67% of respondents had suspected abuse or neglect of a child patient at some time in their career but only 29% had ever made a child protection referral. The dental profession is alerted to the need to ensure necessary appropriate action to safeguard children is always taken when child abuse or neglect are suspected

Connectionist perspectives on language learning, representation and processing.

The field of formal linguistics was founded on the premise that language is mentally represented as a deterministic symbolic grammar. While this approach has captured many important characteristics of the world\u27s languages, it has also led to a tendency to focus theoretical questions on the correct formalization of grammatical rules while also de-emphasizing the role of learning and statistics in language development and processing. In this review we present a different approach to language research that has emerged from the parallel distributed processing or \u27connectionist\u27 enterprise. In the connectionist framework, mental operations are studied by simulating learning and processing within networks of artificial neurons. With that in mind, we discuss recent progress in connectionist models of auditory word recognition, reading, morphology, and syntactic processing. We argue that connectionist models can capture many important characteristics of how language is learned, represented, and processed, as well as providing new insights about the source of these behavioral patterns. Just as importantly, the networks naturally capture irregular (non-rule-like) patterns that are common within languages, something that has been difficult to reconcile with rule-based accounts of language without positing separate mechanisms for rules and exceptions

Anatomical connectivity patterns predict face selectivity in the fusiform gyrus

A fundamental assumption in neuroscience is that brain structure determines function. Accordingly, functionally distinct regions of cortex should be structurally distinct in their connections to other areas. We tested this hypothesis in relation to face selectivity in the fusiform gyrus. By using only structural connectivity, as measured through diffusion-weighted imaging, we were able to predict functional activation to faces in the fusiform gyrus. These predictions outperformed two control models and a standard group-average benchmark. The structure–function relationship discovered from the initial participants was highly robust in predicting activation in a second group of participants, despite differences in acquisition parameters and stimuli. This approach can thus reliably estimate activation in participants who cannot perform functional imaging tasks and is an alternative to group-activation maps. Additionally, we identified cortical regions whose connectivity was highly influential in predicting face selectivity within the fusiform, suggesting a possible mechanistic architecture underlying face processing in humans.United States. Public Health Service (DA023427)National Institute of Mental Health (U.S.) (F32 MH084488)National Eye Institute (T32 EY013935)Poitras FoundationSimons FoundationEllison Medical Foundatio

Can Machines Think? Interaction and Perspective Taking with Robots Investigated via fMRI

Krach S, Hegel F, Wrede B, Sagerer G, Binkofski F, Kircher T. Can Machines Think? Interaction and Perspective Taking with Robots Investigated via fMRI. PLoS ONE. 2008;3(7): e2597.Background When our PC goes on strike again we tend to curse it as if it were a human being. Why and under which circumstances do we attribute human-like properties to machines? Although humans increasingly interact directly with machines it remains unclear whether humans implicitly attribute intentions to them and, if so, whether such interactions resemble human-human interactions on a neural level. In social cognitive neuroscience the ability to attribute intentions and desires to others is being referred to as having a Theory of Mind (ToM). With the present study we investigated whether an increase of human-likeness of interaction partners modulates the participants' ToM associated cortical activity. Methodology/Principal Findings By means of functional magnetic resonance imaging (subjects n = 20) we investigated cortical activity modulation during highly interactive human-robot game. Increasing degrees of human-likeness for the game partner were introduced by means of a computer partner, a functional robot, an anthropomorphic robot and a human partner. The classical iterated prisoner's dilemma game was applied as experimental task which allowed for an implicit detection of ToM associated cortical activity. During the experiment participants always played against a random sequence unknowingly to them. Irrespective of the surmised interaction partners' responses participants indicated having experienced more fun and competition in the interaction with increasing human-like features of their partners. Parametric modulation of the functional imaging data revealed a highly significant linear increase of cortical activity in the medial frontal cortex as well as in the right temporo-parietal junction in correspondence with the increase of human-likeness of the interaction partner (computer<functional robot<anthropomorphic robot<human). Conclusions/Significance Both regions correlating with the degree of human-likeness, the medial frontal cortex and the right temporo-parietal junction, have been associated with Theory-of-Mind. The results demonstrate that the tendency to build a model of another's mind linearly increases with its perceived human-likeness. Moreover, the present data provides first evidence of a contribution of higher human cognitive functions such as ToM in direct interactions with artificial robots. Our results shed light on the long-lasting psychological and philosophical debate regarding human-machine interaction and the question of what makes humans being perceived as human

Distinct contributions of extrastriate body area and temporoparietal junction in perceiving one's own and others' body.

The right temporoparietal cortex plays a critical role in body representation. Here, we applied repetitive transcranial magnetic stimulation (rTMS) over right extrastriate body area (EBA) and temporoparietal junction (TPJ) to investigate their causative roles in perceptual representations of one's own and others' body. Healthy women adjusted size-distorted pictures of their own body or of the body of another person according to how they perceived the body (subjective task) or how others perceived it (intersubjective task). In keeping with previous reports, at baseline, we found an overall underestimation of body size. Crucially, EBA-rTMS increased the underestimation bias when participants adjusted the images according to how others perceived their own or the other woman's body, suggesting a specific role of EBA in allocentric body representations. Conversely, TPJ-rTMS increased the underestimation bias when participants adjusted the body of another person, either a familiar other or a close friend, in both subjective and intersubjective tasks, suggesting an involvement of TPJ in representing others' bodies. These effects were body-specific, since no TMS-induced modulation was observed when participants judged a familiar object. The results suggest that right EBA and TPJ play active and complementary roles in the complex interaction between the perceptions of one's own and other people's body

The combined effect of gender and age on post traumatic stress disorder: do men and women show differences in the lifespan distribution of the disorder?

<p>Abstract</p> <p>Background</p> <p>The aim of the study was to examine the combined effect of gender and age on post traumatic stress disorder (PTSD) in order to describe a possible gender difference in the lifespan distribution of PTSD.</p> <p>Methods</p> <p>Data were collected from previous Danish and Nordic studies of PTSD or trauma. The final sample was composed of 6,548 participants, 2,768 (42.3%) men and 3,780 (57.7%) women. PTSD was measured based on the Harvard Trauma Questionnaire, part IV (HTQ-IV).</p> <p>Results</p> <p>Men and women differed in lifespan distribution of PTSD. The highest prevalence of PTSD was seen in the early 40s for men and in the early 50s for women, while the lowest prevalence for both genders was in the early 70s. Women had an overall twofold higher PTSD prevalence than men. However, at some ages the female to male ratio was nearly 3:1. The highest female to male ratio was found for the 21 to 25 year-olds.</p> <p>Conclusions</p> <p>The lifespan gender differences indicate the importance of including reproductive factors and social responsibilities in the understanding of the development of PTSD.</p