The nature of gestures’ beneficial role in spatial problem solving

Peer reviewedPostprin

How Do Gestures Influence Thinking and Speaking? The Gesture-for-Conceptualization Hypothesis.

Peer reviewedPostprin

Pointing as an Instrumental Gesture : Gaze Representation Through Indication

The research of the first author was supported by a Fulbright Visiting Scholar Fellowship and developed in 2012 during a period of research visit at the University of Memphis.Peer reviewedPublisher PD

The effect of prototyping material on verbal and non-verbal behaviours in collaborative design tasks

This paper reports a study of 23 controlled experiments, with a total of 99 individual tasks, between pairs of designers collaborating to solve a simple design task using four different types of prototyping media. The aim of the study was to correlate verbal and non-verbal behaviours across different types of media with a range of measurement indicators. Using innovative movement trail images we show how collaborative sketching activity results in attenuated use of interpersonal collaborative space when compared with cardboard, clay, and Lego, which provoked intensive collaboration. Furthermore, the sketching (control) condition resulted in pre-conceived ideas being executed when compared with the three-dimensional media, where ideas emerged through collaboration. This finding suggests that increased creativity in design can result through the careful choice of prototyping media at the beginning of the design process

Touching the invisible: Localizing ultrasonic haptic cues

While mid-air gestures offer new possibilities to interact with or around devices, some situations, such as interacting with applications, playing games or navigating, may require visual attention to be focused on a main task. Ultrasonic haptic feedback can provide 3D spatial haptic cues that do not demand visual attention for these contexts. In this paper, we present an initial study of active exploration of ultrasonic haptic virtual points that investigates the spatial localization with and without the use of the visual modality. Our results show that, when providing haptic feedback giving the location of a widget, users perform 50% more accurately compared to providing visual feedback alone. When provided with a haptic location of a widget alone, users are more than 30% more accurate than when given a visual location. When aware of the location of the haptic feedback, active exploration decreased the minimum recommended widget size from 2cm2 to 1cm2 when compared to passive exploration from previous studies. Our results will allow designers to create better mid-air interactions using this new form of haptic feedback

Cognitive demands of face monitoring: Evidence for visuospatial overload

Young children perform difficult communication tasks better face to face than when they cannot see one another (e.g., Doherty-Sneddon & Kent, 1996). However, in recent studies, it was found that children aged 6 and 10 years, describing abstract shapes, showed evidence of face-to-face interference rather than facilitation. For some communication tasks, access to visual signals (such as facial expression and eye gaze) may hinder rather than help children’s communication. In new research we have pursued this interference effect. Five studies are described with adults and 10- and 6-year-old participants. It was found that looking at a face interfered with children’s abilities to listen to descriptions of abstract shapes. Children also performed visuospatial memory tasks worse when they looked at someone’s face prior to responding than when they looked at a visuospatial pattern or at the floor. It was concluded that performance on certain tasks was hindered by monitoring another person’s face. It is suggested that processing of visual communication signals shares certain processing resources with the processing of other visuospatial information

Co-thought and Co-speech Gestures Are Generated by the Same Action Generation Process

We thank Lucy Foulkes, Rachel Furness, Valentina Lee, and Zeshu Shao for their help with data collection; Paraskevi Argyriou for her help with reliability checks of gesture coding; and Agnieszka Konopka and Josje Praamstra for their help with proofreading this article.Peer reviewedPostprin

A psychometric measure of working memory capacity for configured body movement.

Working memory (WM) models have traditionally assumed at least two domain-specific storage systems for verbal and visuo-spatial information. We review data that suggest the existence of an additional slave system devoted to the temporary storage of body movements, and present a novel instrument for its assessment: the movement span task. The movement span task assesses individuals' ability to remember and reproduce meaningless configurations of the body. During the encoding phase of a trial, participants watch short videos of meaningless movements presented in sets varying in size from one to five items. Immediately after encoding, they are prompted to reenact as many items as possible. The movement span task was administered to 90 participants along with standard tests of verbal WM, visuo-spatial WM, and a gesture classification test in which participants judged whether a speaker's gestures were congruent or incongruent with his accompanying speech. Performance on the gesture classification task was not related to standard measures of verbal or visuo-spatial working memory capacity, but was predicted by scores on the movement span task. Results suggest the movement span task can serve as an assessment of individual differences in WM capacity for body-centric information

Remote Real-Time Collaboration Platform enabled by the Capture, Digitisation and Transfer of Human-Workpiece Interactions

In this highly globalised manufacturing ecosystem, product design and verification activities, production and inspection processes, and technical support services are spread across global supply chains and customer networks. Therefore, a platform for global teams to collaborate with each other in real-time to perform complex tasks is highly desirable. This work investigates the design and development of a remote real-time collaboration platform by using human motion capture technology powered by infrared light based depth imaging sensors borrowed from the gaming industry. The unique functionality of the proposed platform is the sharing of physical contexts during a collaboration session by not only exchanging human actions but also the effects of those actions on the task environment. This enables teams to remotely work on a common task problem at the same time and also get immediate feedback from each other which is vital for collaborative design, inspection and verifications tasks in the factories of the future