Multimodal language acquisition based on motor learning and interaction

In this work we propose a methodology for language acquisition in humanoid robots that mimics that in children. Language acquisition is a complex process that involves mastering several different tasks, such as producing speech sounds, learning how to group different sounds into a consistent and manageable number of classes or speech units, grounding speech, and recognizing the speech sounds when uttered by other persons. While it is not known to which extent those abilities are learned or written in our genetic code, this work aims at two intertwined goals: (i) to investigate how much of linguistic structure that can be derived directly from the speech signal directed to infants by (ii) designing, building and testing biological plausible models for language acquisition in a humanoid robot. We have therefore chosen to avoid implementing any pre-programmed linguistic knowledge, such as phonemes, into these models. Instead we rely on general methods such as pattern matching and hierarchical clustering techniques, and show that it is possible to acquire important linguistic structures directly from the speech signal through the interaction with a caregiver. We also show that this process can be facilitated through the use of motor learning