Gestures in spatial descriptions

Most studies of gesture production to date have been based on analyses of narrative discourse in face-to-face interaction. Issues such as the relationship between gesture types and the content of speech, as well as the distribution of particular gesture types across given narrative sequences have been investigated. Depictive gestures, e.g., are frequent where the content concerns the description of concrete objects or actions at a narrative level (McNeill 1992). Little is known about the gesture production in other discourse types, however. Just as different discourse genres have oral characteristics, they are likely to result in different gestural characteristics. In this small-scale study, a preliminary analysis is presented of the gestures produced during a spatial description task during which interlocutors were prevented from seeing each other. This paper will discuss the impact of the discourse type on the use of specific gesture types, especially on deictic gestures. In addition, the traditional issue of why speakers gesticulate at all will be briefly addressed in relation to the question of how visibility conditions affect speakers’ gesture production

Giving language a hand: gesture as a cue based communicative strategy *

All accounts of communicative behaviour in general, and communicative strategies in particular, mention gesture1 in relation to language acquisition (cf. Faerch & Kasper 1983 for an overview). However, few attempts have been made to investigate how spoken language and spontaneous gesture combine to determine discourse referents. Referential gesture and referential discourse will be of particular interest, since communicative strategies in second language discourse often involve labelling problems. This paper will focus on two issues: 1) Within a cognitive account of communicative strategies, gesture will be seen to be part of conceptual or analysis-based strategies, in that relational features in the referents are exploited; 2) It will be argued that communication strategies can be seen in terms of cue manipulation in the same sense as sentence processing has been analysed in terms of competing cues. Strategic behaviour, and indeed the process of referring in general, are seen in terms of cues, combining or competing to determine discourse referents. Gesture can then be regarded as being such a cue at the discourse level, and as a cue-based communicative strategy, in that gesture functions by exploiting physically based cues which can be recognised as being part of the referent. The question of iconicity and motivation vs. the arbitrary qualities of gesture as a strategic cue will be addressed in connection with this

Gesture as a Communication Strategy in Second Language Discourse : A Study of Learners of French and Swedish

Gesture is always mentioned in descriptions of compensatory behaviour in second language discourse, yet it has never been adequately integrated into any theory of Communication Strategies (CSs). This study suggests a method for achieving such an integration. By combining a cognitive theory of speech-associated gestures with a process-oriented framework for CSs, gesture and speech can be seen as reflections of similar underlying processes with different output modes. This approach allows oral and gestural CSs to be classified and analysed within a unified framework. The respective fields are presented in introductory surveys, and a review is provided of studies dealing specifically with compensatory gesture–in aphasia as well as in first and second language acquisition. The experimental part of this work consists of two studies. The production study examines the gestures exploited strategically by Swedish learners of French and French learners of Swedish. The subjects retold a cartoon story in their foreign language to native speakers in conversational narratives. To enable comparisons between learners and proficiency conditions both at individual and group level, subjects performed the task in both their first and their second language. The results show that, contrary to expectations in both fields, strategic gestures do not replace speech, but complement it. Moreover, although strategic gestures are used to solve lexical problems by depicting referential features, most learner gestures instead serve either to maintain visual co-reference at discourse level, or to provide metalinguistic comments on the communicative act itself. These latter functions have hitherto been ignored in CS research. Both similarities and differences can be found between oral and gestural CSs regarding the effect of proficiency, culture, task, and success. The influence of individual communicative style and strategic communicative competence is also discussed. Finally, native listeners’ gestural behaviour is shown to be related to the co-operative effort invested by them to ensure continued interaction, which in turn depends on the proficiency levels of the non-native narrators. The evaluation study investigates native speakers’ assessments of subjects’ gestures, and the effect of gestures on evaluations of proficiency. Native speakers rank all subjects as showing normal or reduced gesture rates and ranges–irrespective of proficiency condition. The influence of gestures on proficiency assessments is modest, but tends to be positive. The results concerning the effectiveness of gestural strategies are inconclusive, however. When exposed to auditory learner data only, listeners believe gestures would improve comprehension, but when learner gestures can be seen, they are not regarded as helpful. This study stresses the need to further examine the effect of strategic behaviour on assessments, and the perception of gestures in interaction. An integrated theory of Communication Strategies has to consider that gestures operate in two ways: as local measures of communicative ‘first-aid’, and as global communication enhancement for speakers and listeners alike. A probabilistic framework is outlined, where variability in performance as well as psycholinguistic and interactional aspects of gesture use are taken into account

Changes in encoding of path of motion after acquisition of a second language

Languages vary typologically in their lexicalization of Path of motion (Talmy 1991). Furthermore, lexicalization patterns are argued to affect syntactic packaging at the level of the clause (e.g. Slobin 1996b) and tend to transfer from a first (L1) to a second language (L2) in second language acquisition (e.g. Cadierno 2004). From this crosslinguistic and developmental evidence, typological preferences for Path expression appear highly robust features of a first language. The current study examines the extent to which preferences for Path encoding really are as enduring as they seem by investigating (1) whether Japanese follows patterns identified for other verb-framed languages like Spanish, and (2) whether patterns established in one’s first language can change after acquisition of a second language. L1 performance of native speakers of Japanese with intermediate-level knowledge of English was compared to that of monolingual speakers of Japanese and English. Results showed that monolingual Japanese speakers followed basic lexicalization patterns typical of other verb-framed languages, but with different realizations of Path packaging within the clause. Moreover, non-monolingual Japanese speakers displayed both English- and Japanese-like patterns for lexicalization with significantly more Path information per clause than either group of monolinguals. Implications for typology and second language acquisition are discussed

L1-L2 convergence in clausal packaging in Japanese and English

This study investigates L1-L2 convergence among bilinguals at an intermediate (CEFR-B2) level of L2 proficiency, focusing on the clausal packaging of Manner and Path of motion. Previous research has shown that Japanese speakers use multi-clause and English speakers single-clause constructions (Allen et al., 2003; Kita & Özyürek, 2003). We compared descriptions of motion from monolingual English and Japanese speakers to L1 and L2 descriptions from Japanese speakers of English as a second (ESL) and foreign (EFL) language. Contrary to previous research, results showed no significant difference between the monolinguals, who predominately used single-clause constructions packaging Manner and Path. However, bilinguals used significantly more multi-clause constructions in both their L1 and L2, with no effects of residence in the L2 community. Following Pavlenko (2011a), findings are interpreted as evidence for L1-L2 convergence. We discuss potential bidirectional cross-linguistic influences underpinning the L1-L2 convergence and implications for the restructuring of bilingual grammars

Multicompetence and native speaker variation in clausal packaging in Japanese

This work was supported by the Max Planck Institute for Psycholinguistics and the Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO; MPI 56-384, The Dynamics of Multilingual Processing, awarded to M Gullberg and P Indefrey).Native speakers show systematic variation in a range of linguistic domains as a function of a variety of sociolinguistic variables. This article addresses native language variation in the context of multicompetence, i.e. knowledge of two languages in one mind (Cook, 1991). Descriptions of motion were elicited from functionally monolingual and non-monolingual speakers of Japanese, with analyses focusing on clausal packaging of Manner and Path. Results revealed that (1) acquisition of a second language (L2) appears to affect how speakers distribute information about motion in and across clauses in their first language (L1); (2) these effects can be seen with rather less knowledge of a second language than the advanced bilingual proficiency level typically studied; and (3) there appears to be little effect of L2 immersion in this domain since Japanese users of English as a second language (ESL) did not differ from Japanese users of English as a foreign language (EFL). We discuss the findings with respect to characterizations of emerging multicompetent grammars, and to implications for the construct of ‘the native speaker’, for language pedagogy and language assessment

What speakers do and what addressees look at: Visual attention to gestures in human interaction live and on video

This study investigates whether addressees visually attend to speakers’ gestures in interaction and whether attention is modulated by changes in social setting and display size. We compare a live face-to-face setting to two video conditions. In all conditions, the face dominates as a fixation target and only a minority of gestures draw fixations. The social and size parameters affect gaze mainly when combined and in the opposite direction from the predicted with fewer gestures fixated on video than live. Gestural holds and speakers’ gaze at their own gestures reliably attract addressees’ fixations in all conditions. The attraction force of holds is unaffected by changes in social and size parameters, suggesting a bottom-up response, whereas speaker-fixated gestures draw significantly less attention in both video conditions, suggesting a social effect for overt gaze-following and visual joint attention. The study provides and validates a video-based paradigm enabling further experimental but ecologically valid explorations of cross-modal information processing

What gestures reveal about the development of semantic distinctions in Dutch children's placement verbs

Placement verbs describe every-day events like putting a toy in a box. Dutch uses two semi-obligatory caused posture verbs (leggen ‘lay’ and zetten ‘set/stand’) to distinguish between events based on whether the located object is placed horizontally or vertically. Although prevalent in the input, these verbs cause Dutch children difficulties even at age five (Narasimhan & Gullberg, in press). Children overextend leggen to all placement events and underextend use of zetten. This study examines what gestures can reveal about Dutch three- and five-year-olds’ semantic representations of such verbs. The results show that children gesture differently from adults in this domain. Three-year-olds express only the path of the caused motion, whereas five-year-olds, like adults, also incorporate the located object. Crucially, gesture patterns are tied to verb use: those children who over-use leggen 'lay' for all placement events only gesture about path. Conversely, children who use the two verbs differentially for horizontal and vertical placement also incorporate objects in gestures like adults. We argue that children's gestures reflect their current knowledge of verb semantics, and indicate a developmental transition from a system with a single semantic component – (caused) movement – to an (adult-like) focus on two semantic components – (caused) movement-and-object

Keeping an eye on gestures: Visual perception of gestures in face-to-face communication

Since listeners usually look at the speaker's face, gestural information has to be absorbed through peripheral visual perception. In the literature, it has been suggested that listeners look at gestures under certain circumstances: 1) when the articulation of the gesture is peripheral; 2) when the speech channel is insufficient for comprehension; and 3) when the speaker him- or herself indicates that the gesture is worthy of attention. The research here reported employs eye tracking techniques to study the perception of gestures in face-to-face interaction. The improved control over the listener's visual channel allows us to test the validity of the above claims. We present preliminary findings substantiating claims 1 and 3, and relate them to theoretical proposals in the literature and to the issue of how visual and cognitive attention are related