Multimodal resources for lexical explanations during webconferencing-supported foreign language teaching: a LEarning and TEaching Corpus investigation.

International audienceWithin the computer-assisted language learning (CALL) field, multimodal research endeavours to consider the simultaneous presence and interaction between verbal communication modes (audio, text chat) present in foreign language learning situations with co-verbal and non-verbal modes (gestures, gaze, posture, other kinesic aspects). This paper explores how trainee-teachers of French as a foreign language, during webconferencing-supported teaching, orchestrate different semiotic resources that are available to them for lexical explanations

Ostensive signals support learning from novel attention cues during infancy

Social attention cues (e.g., head turning, gaze direction) highlight which events young infants should attend to in a busy environment and, recently, have been shown to shape infants' likelihood of learning about objects and events. Although studies have documented which social cues guide attention and learning during early infancy, few have investigated how infants learn to learn from attention cues. Ostensive signals, such as a face addressing the infant, often precede social attention cues. Therefore, it is possible that infants can use ostensive signals to learn from other novel attention cues. In this training study, 8-month-olds were cued to the location of an event by a novel non-social attention cue (i.e., flashing square) that was preceded by an ostensive signal (i.e., a face addressing the infant). At test, infants predicted the appearance of specific multimodal events cued by the flashing squares, which were previously shown to guide attention to but not inform specific predictions about the multimodal events (Wu and Kirkham, 2010). Importantly, during the generalization phase, the attention cue continued to guide learning of these events in the absence of the ostensive signal. Subsequent experiments showed that learning was less successful when the ostensive signal was absent even if an interesting but non-ostensive social stimulus preceded the same cued events

Using multimodal analysis to unravel a silent child’s learning

Although the English Foundation Stage Curriculum for children aged 3 to 5 years recognises that children learn through talk and play and through ‘movement and all their senses’ (DfEE & QCA, 2000: 20), there is comparatively little theoretical understanding of how children learn through diverse ‘modes’, such as body movement, facial expression, gaze, the manipulation of objects and talk, and there is little practical guidance on how practitioners can support children’s ‘multimodal’ learning. Indeed, mounting research evidence indicates that since the introduction of a national early years curriculum and early years assessment schemes, practitioners have felt under increased pressure to focus on children’s verbal skills in order to provide evidence of children’s literacy and numeracy skills in preparation for primary education (see Flewitt, 2005a & 2005b). In the context of these changes, this article relates the story of Tallulah, a 3-year-old girl with a late July birthday, who, like many summer-born children in England, spent one year in an early years setting before moving to primary school aged just 4 years. The article draws on data collected as part of an ESRC-funded study that explored the different ‘modes’ young children use to make and express meaning in the different social settings of home and a preschool playgroup (Flewitt, 2003). Examples are given of how Tallulah communicated her understandings at home through skilful combinations of talk, gaze direction, body movement and facial expression, and how others in the home supported Tallulah’s learning. These are then compared with examples of how Tallulah communicated in playgroup, primarily by combining the silent modes of gaze, body movement and facial expression. The article identifies how the different social settings of home and preschool impacted upon her choices and uses of different expressive modes

GTmoPass: Two-factor Authentication on Public Displays Using Gaze-touch Passwords and Personal Mobile Devices

As public displays continue to deliver increasingly private and personalized content, there is a need to ensure that only the legitimate users can access private information in sensitive contexts. While public displays can adopt similar authentication concepts like those used on public terminals (e.g., ATMs), authentication in public is subject to a number of risks. Namely, adversaries can uncover a user's password through (1) shoulder surfing, (2) thermal attacks, or (3) smudge attacks. To address this problem we propose GTmoPass, an authentication architecture that enables Multi-factor user authentication on public displays. The first factor is a knowledge-factor: we employ a shoulder-surfing resilient multimodal scheme that combines gaze and touch input for password entry. The second factor is a possession-factor: users utilize their personal mobile devices, on which they enter the password. Credentials are securely transmitted to a server via Bluetooth beacons. We describe the implementation of GTmoPass and report on an evaluation of its usability and security, which shows that although authentication using GTmoPass is slightly slower than traditional methods, it protects against the three aforementioned threats

Motor (but not auditory) attention affects syntactic choice

Understanding the determinants of syntactic choice in sentence production is a salient topic in psycholinguistics. Existing evidence suggests that syntactic choice results from an interplay between linguistic and non-linguistic factors, and a speaker’s attention to the elements of a described event represents one such factor. Whereas multimodal accounts of attention suggest a role for different modalities in this process, existing studies examining attention effects in syntactic choice are primarily based on visual cueing paradigms. Hence, it remains unclear whether attentional effects on syntactic choice are limited to the visual modality or are indeed more general. This issue is addressed by the current study. Native English participants viewed and described line drawings of simple transitive events while their attention was directed to the location of the agent or the patient of the depicted event by means of either an auditory (monaural beep) or a motor (unilateral key press) lateral cue. Our results show an effect of cue location, with participants producing more passive-voice descriptions in the patient-cued conditions. Crucially, this cue location effect emerged in the motor-cue but not (or substantially less so) in the auditory-cue condition, as confirmed by a reliable interaction between cue location (agent vs. patient) and cue type (auditory vs. motor). Our data suggest that attentional effects on the speaker’s syntactic choices are modality-specific and limited to the visual and motor, but not the auditory, domain

Storytelling and story-acting: co-construction in action

In the light of sustained interest in the potential value of young children’s narrative play, this paper examines Vivian Gussin Paley’s (1990) approach to storytelling and story-acting, in this case with three to five year-olds. It scrutinizes how children’s narratives are co-constructed during adult-child and peer interactions through spoken and embodied modes, as their stories are scribed by an adult and later dramatised by their peers. Data are drawn from an evaluation of an eight-week training programme, based on Paley’s approach, designed for early years professionals and undertaken in different geographic and demographic locations in England. Naturalistic data collection techniques including video and field notes were used to record the storytelling and story-acting of 18 case study children. The resultant data were subject to close discursive and multimodal analysis of storytelling and story-acting interactions. Findings reveal discursive co-construction ‘in action’ and illustrate how the child story-tellers, story actors and practitioners co-construct narratives through complex combinations of gaze, body posture and speech in responsive and finely-tuned interactional patterns. The study contributes significantly to knowledge about how young children’s narratives are co-constructed through multiple modes in the classroom

Challenges in Transcribing Multimodal Data: A Case Study

open2siComputer-mediated communication (CMC) once meant principally text-based communication mediated by computers, but rapid technological advances in recent years have heralded an era of multimodal communication with a growing emphasis on audio and video synchronous interaction. As CMC, in all its variants (text chats, video chats, forums, blogs, SMS, etc.), has become normalized practice in personal and professional lives, educational initiatives, particularly language teaching and learning, are following suit. For researchers interested in exploring learner interactions in complex technology-supported learning environments, new challenges inevitably emerge. This article looks at the challenges of transcribing and representing multimodal data (visual, oral, and textual) when engaging in computer-assisted language learning research. When transcribing and representing such data, the choices made depend very much on the specific research questions addressed, hence in this paper we explore these challenges through discussion of a specific case study where the researchers were seeking to explore the emergence of identity through interaction in an online, multimodal situated space. Given the limited amount of literature addressing the transcription of online multimodal communication, it is felt that this article is a timely contribution to researchers interested in exploring interaction in CMC language and intercultural learning environments.Cited 10 times as of November 2020 including the prestigious Language Learning Sans Frontiers: A Translanguaging View L Wei, WYJ Ho - Annual Review of Applied Linguistics, 2018 - cambridge.org In this article, we present an analytical approach that focuses on how transnational and translingual learners mobilize their multilingual, multimodal, and multisemiotic repertoires, as well as their learning and work experiences, as resources in language learning. The … Cited by 23 Related articles All 11 versionsopenFrancesca, Helm; Melinda DoolyHelm, Francesca; Melinda, Dool