Transfer learning approach for financial applications

Artificial neural networks learn how to solve new problems through a computationally intense and time consuming process. One way to reduce the amount of time required is to inject preexisting knowledge into the network. To make use of past knowledge, we can take advantage of techniques that transfer the knowledge learned from one task, and reuse it on another (sometimes unrelated) task. In this paper we propose a novel selective breeding technique that extends the transfer learning with behavioural genetics approach proposed by Kohli, Magoulas and Thomas (2013), and evaluate its performance on financial data. Numerical evidence demonstrates the credibility of the new approach. We provide insights on the operation of transfer learning and highlight the benefits of using behavioural principles and selective breeding when tackling a set of diverse financial applications problems

A mathematical theory of semantic development in deep neural networks

An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a fundamental conceptual question: what are the theoretical principles governing the ability of neural networks to acquire, organize, and deploy abstract knowledge by integrating across many individual experiences? We address this question by mathematically analyzing the nonlinear dynamics of learning in deep linear networks. We find exact solutions to this learning dynamics that yield a conceptual explanation for the prevalence of many disparate phenomena in semantic cognition, including the hierarchical differentiation of concepts through rapid developmental transitions, the ubiquity of semantic illusions between such transitions, the emergence of item typicality and category coherence as factors controlling the speed of semantic processing, changing patterns of inductive projection over development, and the conservation of semantic similarity in neural representations across species. Thus, surprisingly, our simple neural model qualitatively recapitulates many diverse regularities underlying semantic development, while providing analytic insight into how the statistical structure of an environment can interact with nonlinear deep learning dynamics to give rise to these regularities

Learning at the Ends: From Hand to Tool Affordances in Humanoid Robots

One of the open challenges in designing robots that operate successfully in the unpredictable human environment is how to make them able to predict what actions they can perform on objects, and what their effects will be, i.e., the ability to perceive object affordances. Since modeling all the possible world interactions is unfeasible, learning from experience is required, posing the challenge of collecting a large amount of experiences (i.e., training data). Typically, a manipulative robot operates on external objects by using its own hands (or similar end-effectors), but in some cases the use of tools may be desirable, nevertheless, it is reasonable to assume that while a robot can collect many sensorimotor experiences using its own hands, this cannot happen for all possible human-made tools. Therefore, in this paper we investigate the developmental transition from hand to tool affordances: what sensorimotor skills that a robot has acquired with its bare hands can be employed for tool use? By employing a visual and motor imagination mechanism to represent different hand postures compactly, we propose a probabilistic model to learn hand affordances, and we show how this model can generalize to estimate the affordances of previously unseen tools, ultimately supporting planning, decision-making and tool selection tasks in humanoid robots. We present experimental results with the iCub humanoid robot, and we publicly release the collected sensorimotor data in the form of a hand posture affordances dataset.Comment: dataset available at htts://vislab.isr.tecnico.ulisboa.pt/, IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EpiRob 2017

Informing the structure of executive function in children: a meta-analysis of functional neuroimaging data

The structure of executive function (EF) has been the focus of much debate for decades. What is more, the complexity and diversity provided by the developmental period only adds to this contention. The development of executive function plays an integral part in the expression of children's behavioral, cognitive, social, and emotional capabilities. Understanding how these processes are constructed during development allows for effective measurement of EF in this population. This meta-analysis aims to contribute to a better understanding of the structure of executive function in children. A coordinate-based meta-analysis was conducted (using BrainMap GingerALE 2.3), which incorporated studies administering functional magnetic resonance imaging (fMRI) during inhibition, switching, and working memory updating tasks in typical children (aged 6–18 years). The neural activation common across all executive tasks was compared to that shared by tasks pertaining only to inhibition, switching or updating, which are commonly considered to be fundamental executive processes. Results support the existence of partially separable but partially overlapping inhibition, switching, and updating executive processes at a neural level, in children over 6 years. Further, the shared neural activation across all tasks (associated with a proposed “unitary” component of executive function) overlapped to different degrees with the activation associated with each individual executive process. These findings provide evidence to support the suggestion that one of the most influential structural models of executive functioning in adults can also be applied to children of this age. However, the findings also call for careful consideration and measurement of both specific executive processes, and unitary executive function in this population. Furthermore, a need is highlighted for a new systematic developmental model, which captures the integrative nature of executive function in children

Why the Left Hemisphere Is Dominant for Speech Production: Connecting the Dots

Evidence from seemingly disparate areas of speech/language research is reviewed to form a unified theoretical account for why the left hemisphere is specialized for speech production. Research findings from studies investigating hemispheric lateralization of infant babbling, the primacy of the syllable in phonological structure, rhyming performance in split-brain patients, rhyming ability and phonetic categorization in children diagnosed with developmental apraxia of speech, rules governing exchange errors in spoonerisms, organizational principles of neocortical control of learned motor behaviors, and multi-electrode recordings of human neuronal responses to speech sounds are described and common threads highlighted. It is suggested that the emergence, in developmental neurogenesis, of a hard-wired, syllabically-organized, neural substrate representing the phonemic sound elements of one’s language, particularly the vocalic nucleus, is the crucial factor underlying the left hemisphere’s dominance for speech production

Augmented maturation of executive functions in musically trained children and adolescents

Musically trained individuals have been found to outperform untrained peers in various tasks specific to music performance. Interestingly, they have also been found to outperform their musically untrained peers in tasks not related to music-making, requiring a wide range of cognitive skills, such as executive functions. In a series of studies with the same participants, this thesis examines the developmental paths of different subcomponents of executive functions in a group of participants in their school years, from childhood until adolescence, while undergoing music training. Their neurocognitive development is compared with that of their musically untrained peers. The aim of the studies was to uncover whether music training is associated with improved executive functions and augmented functioning of the brain mechanisms of executive functions. Performance measures included various neuropsychological test measures and tasks created for neuroimaging experiments. Brain activation measures of executive functions were event-related potentials (ERPs) recorded with electroencephalography, and BOLD activation changes acquired with functional magnetic resonance imaging (fMRI). Differences in these measures between musically trained and untrained participants aged 9-21 were examined, as well as longitudinal change in test performance during six years. The results of the studies showed that music training is associated with the enhancement of various executive functions: working memory maintenance, inhibition, set shifting, and selective attention during childhood and adolescence. However, the improvement in set shifting diminishes with age being virtually nonexistent in early adulthood. Results also showed more efficient functioning of neural mechanisms related to executive functions. Musically trained participants exhibited lower distractibility and enhanced processing of targets during tasks for executive functions. Furthermore, still in late adolescence and early adulthood, musically trained participants had more adult-like responses and more efficient functioning of brain mechanisms for attention and executive control during tasks for executive functions than their untrained peers. Together, the results show that music training is associated with enhancement of brain mechanisms of executive functions and a passing advantage in tasks for executive functions, with the largest differences compared to untrained peers seen in school-age and early adolescence. In late adolescence, the advantage in tasks has disappeared but echoes of enhancement can be still seen in the more mature and more efficient functioning of neural mechanisms of executive functions.Musiikkia harrastaneet suoriutuvat paremmin musiikkiin liittyvissä tehtävissä kuin ne, joilla musiikinharrastustaustaa ei ole. Yllättäen muusikot ja muut musiikin parissa harjaantuneet pärjäävät muuta harrastaneita paremmin myös sellaisissa tehtävissä, jotka eivät liity musiikkiin ja jotka edellyttävät laaja-alaisia kognitiivisia taitoja, kuten toiminnanohjausta. Tämä väitöskirja selvitti toiminnanohjauksen eri osa-alueiden kehityskaarta lapsuudesta varhaisaikuisuuteen sarjalla tutkimuksia, joihin osallistui sama musiikkia harrastavien lasten joukko. Näiden osallistujien neurokognitiivisten toimintojen kehittymistä verrattiin muuta harrastavien osallistujien kehitykseen. Tutkimusten tarkoitus oli selvittää, onko musiikin harrastaminen yhteydessä tehostuneisiin toiminnanohjauksen taitoihin, sekä muutoksiin näitä taitoja tukevien aivomekanismien toiminnassa. Taitoja tutkittiin neuropsykologisilla testimenetelmillä sekä aivokuvantamiskokeita varten erikseen kehitetyillä tehtävillä. Toiminnanohjauksen aivomekanismeja tutkittiin rekisteröimällä aivojen herätevasteita aivosähkökäyrällä, ja aivoalueiden veren happipitoisuudesta riippuvaa vastetta toiminnallisella magneettiresonanssikuvantamisella. Tutkimuksissa vertailtiin musiikkia harrastavien ja muuta harrastavien, 9-21-vuotiaiden osallistujien testisuoriutumisen ja aivojen toiminnan eroja. Lisäksi testisuoriutumisen erojen muutosta seurattiin kuuden vuoden ajan. Tutkimusten tulokset osoittavat, että musiikin harrastaminen on yhteydessä usean toiminnanohjauksen osa-alueen tehostumiseen lapsuudessa. Näihin kuuluivat työmuistin ylläpito ja päivittäminen, inhibitio, vaihtaminen ja valikoiva tarkkaavaisuus. Vaihtamisen tehostuminen vähenee kuitenkin iän myötä siten, että varhaisaikuisuudessa eroja musiikkia ja muuta harrastavien välillä ei testisuoriutumisessa ole. Tulosten mukaan musiikkia harrastavien toiminnanohjauksen aivomekanismit toimivat myös eri tavoin kuin muiden osallistujien. Toiminnanohjausta edellyttävissä tehtävissä musiikkia harrastavilla ilmeni vähäisempää häiriöherkkyyttä ja tehostunutta kohteiden käsittelyä. Lisäksi vielä varhaisaikuisuudessa musiikkia harrastavilla oli kehitysasteeltaan kypsempiä aivovasteita sekä tehokkaampaa toimintaa tarkkaavuuteen ja toiminnanohjaukseen liittyvillä aivoalueilla toiminnanohjauksen tehtävien aikana kuin muilla osallistujilla. Kaiken kaikkiaan tulokset osoittavat, että musiikin harrastaminen on yhteydessä toiminnanohjauksen aivomekanismien tehokkaampaan toimintaan sekä ohimenevästi myös parempaan suoriutumiseen toiminnanohjauksen tehtävissä