El gesto como facilitador y precursor del desarrollo del lenguaje

Els estudis sobre desenvolupament del llenguatge típicament s'han centrat en l'anàlisi de la parla. Tanmateix, hi ha altres elements comunicatius que són una peça fonamental en la comunicació i cognició humanes: els gestos. Aquests elements visuals estan íntimament integrats amb la parla des del punt de vista temporal i des del punt de vista semanticopragmàtic, i fan que es pugui parlar del llenguatge i la comunicació com uns fets multimodal. En aquest article posem a l'abast dels lectors un resum dels estudis més recents sobre adquisició del llenguatge des d'una perspectiva multimodal, tot i emfasitzant el rol dels gestos com a facilitadors i precursors del llenguatge no només en les etapes més primerenques sinó també en etapes de desenvolupament pragmàtic més tardà. Els estudis demostren que el gest és un component central de l'adquisició del llenguatge i que actua com a precursor i predictor de l’aprenentatge del vocabulari, de la sintaxi, així com del desenvolupament pragmàtic i discursiu.  Finalment apuntem les àrees de recerca més actuals i innovadores i el possible impacte d'aquesta recerca en l'àmbit de la rehabilitació del llenguatge. Studies of language development have typically focused on speech analysis. However, there are other communicative elements that are a fundamental piece in human communication and cognition: gestures. These visual elements are intimately integrated with speech from temporal and semanticopragmatic points of view, and make it possible to speak of language and communication as multimodal facts. In this article we provide readers with a summary of the most recent studies on language acquisition from a multimodal perspective, while emphasizing the role of gestures as facilitators and precursors of language in both the earliest stagesand later stages of pragmatic development. Studies show that gesture is a central component of language acquisition and that it acts as a precursor and predictor of vocabulary learning, syntax, pragmatic and discursive development. Finally, we identify the most current and innovative areas of research and the possible impact of this research in the field of language rehabilitation. Los estudios sobre desarrollo del lenguaje típicamente se han centrado en el análisis del habla. Sin embargo, hay otros elementos comunicativos que son una pieza fundamental en la comunicación y cognición humanas: los gestos. Estos elementos visuales están íntimamente integrados con el habla desde el punto de vista temporal y desde el punto de vista semanticopragmàtic, y hacen que se pueda hablar del lenguaje y la comunicación como unos hechos multimodales. En este artículo ponemos al alcance de los lectores un resumen de los estudios más recientes sobre adquisición del lenguaje desde una perspectiva multimodal, aunque enfatizando el rol de los gestos como facilitadores y precursores del lenguaje no sólo en las etapas más tempranas sino también en etapas de desarrollo pragmático más tardío. Los estudios demuestran que el gesto es un componente central de la adquisición del lenguaje y que actúa como precursor y predictor del aprendizaje del vocabulario, de la sintaxis, así como del desarrollo pragmático y discursivo. Finalmente apuntamos las áreas de investigación más actuales e innovadoras y el posible impacto de esta investigación en el ámbito de la rehabilitación del lenguaje

The integration of prosody and gesture in early intentional communication

This dissertation comprises four experimental studies which investigate the way infants integrate prosody and gesture for intentional communicative purposes. As adult speakers we automatically integrate prosody and gestures at a temporal and pragmatic level, and we use these cues together with social contextual information to convey and understand intentional meanings. My aim is to investigate whether infants use prosodic and gesture features in an integrated way for communicative purposes prior to their use of lexical-semantic cues. The dissertation includes four studies, each one described in a separate chapter. The first study is a longitudinal analysis of how a group of infants produce gesture and speech combinations in natural interactions, with results that show that already at 12 and 15 months of age infants temporally align prosodic and gesture prominences. The second study uses a habituation/test procedure to test the infants‟ early sensitivity to temporal gesture-prosodic integration, showing that 9-month-old infants are sensitive to the alignment between prosodic and gesture prominences. The third study analyzes the longitudinal productions of four infants at the pre-lexical stage and provides evidence that infants use prosodic cues such as pitch range and duration to convey specific intentions like requests, statements, responses, and expressions of satisfaction or discontent. Finally, the fourth study examines how infants responded at 12 months of age to different types of pointing-speech combinations and shows that x infants use prosodic and gestural cues to comprehend communicative intentions behind an attention-directing act. Altogether, this dissertation shows that the temporal integration of gesture and speech occurs at the early stages of language and cognitive development, and that pragmatic uses of prosody and gesture develop before infants master the use of lexical cues. Thus, prosody is the first grammatical component of language that infants use for communicative purposes, revealing that linguistic communication emerges before infants have the ability to use lexical items with semantic meanings. I further claim that infants‟ integration of prosody and gesture at the temporal and pragmatic levels is a reflex of an early emergence of language pragmatics.Aquesta tesi inclou quatre estudis experimentals que investiguen com els infants integren prosòdia i gestualitat amb fins comunicatius. Els adults integrem la prosòdia i la gestualitat de manera temporal i pragmàtica i, juntament amb la informació sociocontextual, ho utilitzem per a transmetre i comprendre significats intencionals. En aquesta tesi es pretén investigar si els infants utilitzen la prosòdia i la gestualitat de manera integrada per a fins comunicatius, abans de ser capaços d‟emprar elements lexicosemàntics. La tesi inclou quatre estudis, cada un en un apartat diferent. El primer estudi analitza longitudinalment les combinacions de gest i parla dels infants en interaccions espontànies, i mostra que a partir dels 12 o 15 mesos els infants alineen temporalment la prominència prosòdica i la prominència gestual. El segon estudi empra el mètode d‟habituació/test per a comprovar l‟habilitat primerenca dels infants a percebre la integració temporal entre prosòdia i gest, i mostra que als 9 mesos els infants ja són capaços de percebre l‟alineació temporal entre la prominència gestual i la prosòdica. El tercer estudi també analitza longitudinalment les produccions dels infants en interaccions espontànies per a mostrar que, abans de produir les primeres paraules, els infants ja utilitzen elements prosòdics com el rang tonal i la durada, a més del gest, per a transmetre actes de parla com ara la petició, les respostes, les oracions declaratives, i les expressions de satisfacció o de descontentament. Finalment, el quart xii estudi investiga la reacció dels infants a diversos tipus de combinacions de parla i gest d‟assenyalar, i mostra que els infants de 12 mesos utilitzen les marques prosòdiques i gestuals del discurs per a entendre les intencions comunicatives que els són dirigides. En conjunt, aquesta tesi mostra que la integració temporal de prosòdia i gest ocorre en les etapes més primerenques del desenvolupament lingüístic i cognitiu, i que el usos pragmàtics de la prosòdia i la gestualitat emergeixen abans que els infants dominin l‟ús d‟elements lèxics. Així, la prosòdia és el primer component gramatical del llenguatge que els infants utilitzen amb finalitat comunicativa, cosa que indica que la comunicació lingüística emergeix abans que els infants tinguin la capacitat de produir ítems lèxics amb significants semàntics. La conclusió general és, doncs, que la integració temporal i pragmàtica de la prosòdia i el gest per part dels infants indica el desenvolupament primerenc de la pragmàtica lingüística

Prosody in the Auditory and Visual Domains: A Developmental Perspective

The development of body movements such as hand or head gestures, or facial expressions, seems to go hand-in-hand with the development of speech abilities. We know that very young infants rely on the movements of their caregivers’ mouth to segment the speech stream, that infants’ canonical babbling is temporally related to rhythmic hand movements, that narrative abilities emerge at a similar time in speech and gestures, and that children make use of both modalities to access complex pragmatic intentions. Prosody has emerged as a key linguistic component in this speech-gesture relationship, yet its exact role in the development of multimodal communication is still not well understood. For example, it is not clear what the relative weights of speech prosody and body gestures are in language acquisition, or whether both modalities develop at the same time or whether one modality needs to be in place for the other to emerge. The present paper reviews existing literature on the interactions between speech prosody and body movements from a developmental perspective in order to shed some light on these issues

Infants temporally coordinate gesture-speech combinations before they produce their first words

This study explores the patterns of gesture and speech combinations from the babbling period to the one-word stage and the temporal alignment between the two modalities. The communicative acts of four Catalan children at 0;11, 1;1, 1;3, 1;5, and 1;7 were gesturally and acoustically analyzed. Results from the analysis of a total of 4,507 communicative acts extracted from approximately 24 h of at-home recordings showed that (1) from the early single-word period onwards gesture starts being produced mainly in combination with speech rather than as a gesture-only act; (2) in these early gesture-speech combinations most of the gestures are deictic gestures (pointing and reaching gestures) with a declarative communicative purpose; and (3) there is evidence of temporal coordination between gesture and speech already at the babbling stage because gestures start before the vocalizations associated with them, the stroke onset coincides with the onset of the prominent syllable in speech, and the gesture apex is produced before the end of the accented syllable. These results suggest that during the transition between the babbling stage and single-word period infants start combining deictic gestures and speech and, when combined, the two modalities are temporally coordinated.A preliminary version of this paper was presented at the conferences Architectures and Mechanisms for Language Processing (Paris, France, September 1-3, 2011) and Gesture and Speech in Interaction (Bielefeld, Germany, September 5-7, 2011). We would like to thank participants at those meetings, especially Marc Swerts, Heather Rusiewicz, and Stefan Kopp, for their helpful comments. We thank Aurora Bel, Louise McNally, and José Ignacio Hualde for being part of the PhD project defense of the first author of the study and for their helpful observations. We also thank Ulf Liszkowski for his useful comments on the analysis of the intentions behind pointing gestures, Simone Bijvoet and Joan Borràs for their help with the statistical analysis, and Alfonso Igualada and Santiago González-Fuente for the inter-rater reliability tests. Finally, we are really grateful to the children and the children’s parents for voluntarily taking part in this study. This research has been funded by two research grants awarded by the Spanish Ministry of Science and Innovation (FFI2009-07648/FILO “The role of tonal scaling and tonal alignment in distinguishing intonational categories in Catalan and Spanish”, and FFI2012-31995 “Gestures, prosody and linguistic structure”), by a grant awarded by the Generalitat de Catalunya (2009SGR-701) to the Grup d’Estudis de Prosòdia, by the Generalitat de Catalunya (2009SGR-701) to the Grup d’Estudis de Prosòdia, by the grant RECERCAIXA 2012 for the project “Els precursors del llenguatge. Una guia TIC per a pares i educadors” awarded by Obra Social ‘La Caixa’, and by the Consolider-Ingenio 2010 (CSD2007-00012) grant

Prosodic structure shapes the temporal realization of intonation and manual gesture movements

Purpose: Previous work on the temporal coordination between gesture and speech found that the prominence in gesture coordinates with speech prominence. This study investigates the anchoring regions in speech and pointing gesture aligning with each other. Our hypotheses are that (a) in contrastive focus conditions the gesture apex is anchored in the intonation peak, (b) the upcoming prosodic boundary influences the timing of gesture and intonation movements. Method: 15 Catalan speakers pointed at a screen while pronouncing a target word with different metrical patterns in a contrastive focus condition and followed by a phrase boundary. A total of 702 co-speech deictic gestures were acoustically and gesturally analyzed. Results: Intonation peaks and gesture apexes showed parallel behavior with respect to their position within the accented syllable: they occurred at the end of the accented syllable in non-phrase-final position, whereas they occurred well before the end of the accented syllable in phrase-final position. Crucially, the position of intonation peaks and gesture apexes was correlated and was bound by prosodic structure. Conclusions: The results refine the phonological synchronization rule (McNeill, 1992), showing that gesture apexes are anchored in intonation peaks, and that gesture and prosodic movements are bound by prosodic phrasing.This research was funded by two research grants awarded by the Spanish Ministry of Science and Innovation (Grant FFI2009-07648/FILO, “The Role of Tonal Scaling and Tonal Alignment in Distinguishing Intonational Categories in Catalan and Spanish,” and Grant FFI2012-31995, “Gestures, Prosody and Linguistic Structure”); by a grant from the Generalitat de Catalunya (2009SGR-701), awarded to the Grup d’Estudis de Prosòdia; by Grant RECERCAIXA 2012 (for the project “Els Precursors del Llenguatge. Una Guia TIC per a Pares i Educadors” awarded by Obra Social ‘La Caixa’); and by the Consolider-Ingenio 2010 (CSD2007-00012) grant

Twelve-month-olds understand social intentions based on prosody and gesture shape

Infants infer social and pragmatic intentions underlying attention-directing gestures, but the basis on which infants make these inferences is not well understood. Previous studies suggest that infants rely on information from preceding shared action contexts and joint perceptual scenes. Here, we tested whether 12-month-olds use information from act-accompanying cues, in particular prosody and hand shape, to guide their pragmatic understanding. In Experiment 1, caregivers directed infants’ attention to an object to request it, share interest in it, or inform them about a hidden aspect. Caregivers used distinct prosodic and gestural patterns to express each pragmatic intention. Experiment 2 was identical except that experimenters provided identical lexical information across conditions and used three sets of trained prosodic and gestural patterns. In all conditions, the joint perceptual scenes and preceding shared action contexts were identical. In both experiments, infants reacted appropriately to the adults’ intentions by attending to the object mostly in the sharing interest condition, offering the object mostly in the imperative condition, and searching for the referent mostly in the informing condition. Infants’ ability to comprehend pragmatic intentions based on prosody and gesture shape expands infants’ communicative understanding from common activities to novel situations for which shared background knowledge is missing.Funded by Spanish Ministry of Science and Innovation. Grant Number: FFI2012-3199

Pre‐schoolers use head gestures rather than prosodic cues to highlight important information in speech

International audiencePrevious evidence suggests that children's mastery of prosodic modulations to signal the informational status of discourse referents emerges quite late in development. In the present study, we investigate the children's use of head gestures as it compares to prosodic cues to signal a referent as being contrastive relative to a set of possible alternatives. A group of French-speaking pre-schoolers were audio-visually recorded while playing in a semi-spontaneous but controlled production task, to elicit target words in the context of broad focus, contrastive focus, or corrective focus utterances. We analysed the acoustic features of the target words (syllable duration and word-level pitch range), as well as the head gesture features accompanying these target words (head gesture type, alignment patterns with speech). We found that children's production of head gestures, but not their use of either syllable duration or word-level pitch range, was affected by focus condition. Children mostly aligned head gestures with relevant speech units, especially when the target word was in phrase-final position. Moreover, the presence of a head gesture was linked to greater syllable duration patterns in all focus conditions. Our results show that (a) 4- and 5-year-old French-speaking children use head gestures rather than prosodic cues to mark the informational status of discourse referents, (b) the use of head gestures may gradually entrain the production of adult-like prosodic features, and that (c) head gestures with no referential relation with speech may serve a linguistic structuring function in communication, at least during language development

Nine-month-old infants are sensitive to the temporal alignment of prosodic and gesture prominences

This study investigated the sensitivity of 9-month-old infants to the alignment between prosodic and gesture prominences in pointing–speech combinations. Results revealed that the perception of prominence is multimodal and that infants are aware of the timing of gesture–speech combinations well before they can produce them.We thank Helena Moliné for her assistance in collecting the data. We also thank the infants involved and their parents for their participation. This research was funded by the FFI2012-31995, 2014 SGR 925 and PSI2010-20294 research grants, and by the 2013 RECERCAIXA grant “Early precursors of language: an ICT guide for parents and caregivers”. A fuller report will be provided upon request