The study of language began with the study of ancient written text. This practice has shaped and is still shaping our practices in language documentation. The way we conceptualise language and language use has implications for our documentary practices. We still focus on what we think can be written down and often disregard what we think cannot be written down.
But, typically, when we speak, we cannot only hear each other but also see each other. Language is grounded in face-to-face interaction and speaking is a joint activity (Clark 1996). Language acquisition is a process that takes place in face-to-face contexts and our cognitive system automatically integrates both what we hear and what we see (McGurk & McDonald 1976). When we speak, we use our hands to gesture and the information provided in this visual, gestural modality is also integrated automatically in our mind. The gestures we use contribute crucially to our understanding of what speakers are communicating (Kendon 2004). Communities have developed alternate sign languages used in e.g. mourning practices (Kendon). Deaf people develop fully fledged sign languages in the manual modality (Meir et al. 2012).
However, despite this basic multimodal nature of language use we often still do not document language to its full extent due to restricting our recordings to audio or restricting video recording to a few genres like story telling. In this talk I will exemplify the multimodal nature of language use, focusing on manual gesture in its various forms and functions from indexing to semantic specification, and discourse structure marking.
I will discuss its implications for language documentation practices. The role of video recording and the way language use needs to be video recorded to provide useable material for linguistic and ethnographic documentation and analysis will be highlighted. A methodology for training the much needed video recording will be suggested which embeds the technical training of video technology and recording within a theoretically grounded understanding of language use
