Speakers align with their partner's overspecification during interaction

Speakers often overspecify by encoding more information than is necessary when referring to an object (e.g., “the blue mug” for the only mug in a group of objects). We investigated the role of a partner's linguistic behavior (whether or not they overspecify) on a speaker's own tendency to overspecify. We used a director–matcher task in which speakers interacted with a partner who either consistently overspecified or minimally specified in the color/size dimension (Experiments 1, 2, and 3), as well as with a partner who switched behaviors midway through interaction (Experiments 4 and 5). We found that speakers aligned with their partner's linguistic behavior to produce overspecific or minimally specific descriptions, and we saw little evidence that the alignment was enhanced by lexical or semantic repetition across prime and target trials. Time-course analyses showed that alignment increased over the course of the interaction, and speakers appeared to track a change in the partner's linguistic behavior, altering their reference strategy to continue matching that of their partner's. These results demonstrate the persistent influence of a partner's behavior on speakers across the duration of an interaction

Speakers Align With Their Partner's Overspecification During Interaction

Speakers often overspecify by encoding more information than is necessary when referring to an object (e.g., “the blue mug” for the only mug in a group of objects). We investigated the role of a partner's linguistic behavior (whether or not they overspecify) on a speaker's own tendency to overspecify. We used a director–matcher task in which speakers interacted with a partner who either consistently overspecified or minimally specified in the color/size dimension (Experiments 1, 2, and 3), as well as with a partner who switched behaviors midway through interaction (Experiments 4 and 5). We found that speakers aligned with their partner's linguistic behavior to produce overspecific or minimally specific descriptions, and we saw little evidence that the alignment was enhanced by lexical or semantic repetition across prime and target trials. Time-course analyses showed that alignment increased over the course of the interaction, and speakers appeared to track a change in the partner's linguistic behavior, altering their reference strategy to continue matching that of their partner's. These results demonstrate the persistent influence of a partner's behavior on speakers across the duration of an interaction

Cues to lying may be deceptive:Speaker and listener behaviour in an interactive game of deception

Are the cues that speakers produce when lying the same cues that listeners attend to when attempting to detect deceit? We used a two-person interactive game to explore the production and perception of speech and nonverbal cues to lying. In each game turn, participants viewed pairs of images, with the location of some treasure indicated to the speaker but not to the listener. The speaker described the location of the treasure, with the objective of misleading the listener about its true location; the listener attempted to locate the treasure, based on their judgement of the speaker’s veracity. In line with previous comprehension research, listeners’ responses suggest that they attend primarily to behaviours associated with increased mental difficulty, perhaps because lying, under a cognitive hypothesis, is thought to cause an increased cognitive load. Moreover, a mouse-tracking analysis suggests that these judgements are made quickly, while the speakers’ utterances are still unfolding. However, there is a surprising mismatch between listeners and speakers: When producing false statements, speakers are less likely to produce the cues that listeners associate with lying. This production pattern is in keeping with an attempted control hypothesis, whereby liars may take into account listeners’ expectations and correspondingly manipulate their behaviour to avoid detection

Interpreting nonverbal cues to deception in real time

When questioning the veracity of an utterance, we perceive certain non-linguistic behaviours to indicate that a speaker is being deceptive. Recent work has highlighted that listeners' associations between speech disfluency and dishonesty are detectable at the earliest stages of reference comprehension, suggesting that the manner of spoken delivery influences pragmatic judgements concurrently with the processing of lexical information. Here, we investigate the integration of a speaker's gestures into judgements of deception, and ask if and when associations between nonverbal cues and deception emerge. Participants saw and heard a video of a potentially dishonest speaker describe treasure hidden behind an object, while also viewing images of both the named object and a distractor object. Their task was to click on the object behind which they believed the treasure to actually be hidden. Eye and mouse movements were recorded. Experiment 1 investigated listeners' associations between visual cues and deception, using a variety of static and dynamic cues. Experiment 2 focused on adaptor gestures. We show that a speaker's nonverbal behaviour can have a rapid and direct influence on listeners' pragmatic judgements, supporting the idea that communication is fundamentally multimodal

Beyond words: non-linguistic signals and the recovery of meaning

Beyond the words a speaker produces, meaning can be recovered from the many ways in which the words are delivered. During everyday discourse, the way in which a speaker produces an utterance - both in their spoken delivery and accompanying movements | is an important part of the communication process. This thesis investigates the ways in which meaning is carried within these nonlinguistic behaviours, focussing on how they may provide signals about upcoming message content and about speakers intentions. Previous research has shown that the manner of spoken delivery (for instance, rates of speech, intonation, and fluency) influences listeners content-based expectations as well as pragmatic comprehension. These effects have been evidenced both post-hoc and during the moment-to-moment processing of speech. Considerably less attention has been paid on whether speakers movements and non-verbal behaviours have similar effects: Research on gesture has tended to focus on the content represented by gesture (rather than its potential to signal information about the message and/or speaker). We focus on two ways in which non-linguistic behaviours influence comprehension: Firstly, as signals of speech planning difficulty (and so of upcoming content), and secondly as indicators of a speakers intention to deceive. The former has been studied in relation to speech disfluency, but not to gestures. The latter has been studied extensively in relation to many linguistic and non-linguistic behaviours, but has only recently begun to be addressed with respect to the time course of the process. Focussing on these two areas, we address the broader question about the perceptual relevance of non-linguistic signals in comprehension through a dialogue study and a series of comprehension experiments combining eye and mouse tracking techniques. Extending the VisualWorld Paradigm|commonly used in comprehension studies - to include a video component showing the speaker, we measure listeners' eye movements and mouse coordinates as they select objects in the on-screen display. By directly manipulating the presence of different non-linguistic behaviours, we investigate whether and when these behaviours are interpreted as signals of 1) upcoming difficulty in speech, and 2) the speaker's intention to deceive. Our results demonstrate that, like for speech disfluency, listeners interpret the presence of representational gesturing to inform explicit predictions about upcoming referents. This follows from the findings from our dialogue study which suggest that speakers produce more of this type of gesturing relative to speech when describing shapes which are more conceptually difficult. We also show that listeners reliably perceive certain motoric behaviours of a speaker to indicate deception, and that the influence these behavioural cues have on listeners' interpretation can be detected alongside the unfolding linguistic input. Non-linguistic cues to perceived deception are found to be robust to contexts where speakers produce a variety of cues in different modalities. However, findings point to differences in how listeners link visual and spoken cues with deception. Results indicate that when cues are present in both modalities, visual cues tend to drive listeners' biases towards interpreting an utterance as dishonest. The time course supports a view in which non-linguistic cues influence pragmatic comprehension at an early stage, and suggests that linking visual cues with deception may be more resource demanding than it is for spoken cues. Listeners' associations between the presence of certain non-linguistic behaviours and judgements of deception also hold in situations where there are other alternative explanations for a given cue. However, the availability of an alternative explanations for a given non-linguistic behaviour are found to influence early stages of comprehension, suggesting that listeners may engage in dynamic reasoning about the possible causes of a speaker's manner of delivery. Taken together, the results from studies presented in this thesis highlight the role of both the spoken and visual delivery of an utterance in shaping comprehension, highlighting the fact that communication is fundamentally multi-modal