Children benefit from gestures to understand degraded speech but to a lesser extent than adults

The present study investigated to what extent children, compared to adults, benefit from gestures to disambiguate degraded speech by manipulating speech signals and manual modality. Dutch-speaking adults (N = 20) and 6- and 7-year-old children (N = 15) were presented with a series of video clips in which an actor produced a Dutch action verb with or without an accompanying iconic gesture. Participants were then asked to repeat what they had heard. The speech signal was either clear or altered into 4- or 8-band noise-vocoded speech. Children had more difficulty than adults in disambiguating degraded speech in the speech-only condition. However, when presented with both speech and gestures, children reached a comparable level of accuracy to that of adults in the degraded-speech-only condition. Furthermore, for adults, the enhancement of gestures was greater in the 4-band condition than in the 8-band condition, whereas children showed the opposite pattern. Gestures help children to disambiguate degraded speech, but children need more phonological information than adults to benefit from use of gestures. Children’s multimodal language integration needs to further develop to adapt flexibly to challenging situations such as degraded speech, as tested in our study, or instances where speech is heard with environmental noise or through a face mask

Psychological proximity guides multimodal communication:A behavioural and eye-tracking study

Pointing gestures and demonstratives are two early acquired means of establishing joint attention. While they are frequently used in combination, the exact nature oftheir relationship has yet to be determined

Systematic mappings between semantic categories and types of iconic representations in the manual modality:A normed database of silent gesture

Contains fulltext : 212953.pdf (publisher's version ) (Open Access)An unprecedented number of empirical studies have shown that iconic gestures - those that mimic the sensorimotor attributes of a referent - contribute significantly to language acquisition, perception, and processing. However, there has been a lack of normed studies describing generalizable principles in gesture production and in comprehension of the mappings of different types of iconic strategies (i.e., modes of representation; Müller, 2013). In Study 1 we elicited silent gestures in order to explore the implementation of different types of iconic representation (i.e., acting, representing, drawing, and personification) to express concepts across five semantic domains. In Study 2 we investigated the degree of meaning transparency (i.e., iconicity ratings) of the gestures elicited in Study 1. We found systematicity in the gestural forms of 109 concepts across all participants, with different types of iconicity aligning with specific semantic domains: Acting was favored for actions and manipulable objects, drawing for nonmanipulable objects, and personification for animate entities. Interpretation of gesture-meaning transparency was modulated by the interaction between mode of representation and semantic domain, with some couplings being more transparent than others: Acting yielded higher ratings for actions, representing for object-related concepts, personification for animate entities, and drawing for nonmanipulable entities. This study provides mapping principles that may extend to all forms of manual communication (gesture and sign). This database includes a list of the most systematic silent gestures in the group of participants, a notation of the form of each gesture based on four features (hand configuration, orientation, placement, and movement), each gesture's mode of representation, iconicity ratings, and professionally filmed videos that can be used for experimental and clinical endeavors.17 p

A systematic investigation of gesture kinematics in evolving manual languages in the lab

Item does not contain fulltextSilent gestures consist of complex multi-articulatory movements but are now primarily studied through categorical coding of the referential gesture content. The relation of categorical linguistic content with continuous kinematics is therefore poorly understood. Here, we reanalyzed the video data from a gestural evolution experiment (Motamedi, Schouwstra, Smith, Culbertson, & Kirby, 2019), which showed increases in the systematicity of gesture content over time. We applied computer vision techniques to quantify the kinematics of the original data. Our kinematic analyses demonstrated that gestures become more efficient and less complex in their kinematics over generations of learners. We further detect the systematicity of gesture form on the level of thegesture kinematic interrelations, which directly scales with the systematicity obtained on semantic coding of the gestures. Thus, from continuous kinematics alone, we can tap into linguistic aspects that were previously only approachable through categorical coding of meaning. Finally, going beyond issues of systematicity, we show how unique gesture kinematic dialects emerged over generations as isolated chains of participants gradually diverged over iterations from other chains. We, thereby, conclude that gestures can come to embody the linguistic system at the level of interrelationships between communicative tokens, which should calibrate our theories about form and linguistic content.29 p

Iconicity and Gesture Jointly Facilitate Learning of Second Language Signs at First Exposure in Hearing Nonsigners

When learning spoken second language (L2), words overlapping in form and meaning with one's native language (L1) help break into the new language. When nonsigning speakers learn a sign language as L2, such overlaps are absent because of the modality differences (L1: speech, L2: sign). In such cases, nonsigning speakers might use iconic form‐meaning mappings in signs or their own gestural experience as gateways into the to‐be‐acquired sign language. In this study, we investigated how both these phenomena may contribute jointly to the acquisition of sign language vocabulary by hearing nonsigners. Participants were presented with three types of signs in the Sign Language of the Netherlands (NGT): arbitrary signs, iconic signs with high or low gesture overlap. Signs that were both iconic and highly overlapping with gestures boosted learning most at first exposure, and this effect remained the day after. Findings highlight the influence of modality‐specific attributes supporting the acquisition of a signed lexicon

Co-Speech Gesture Detection through Multi-phase Sequence Labeling

Gestures are integral components of face-to-face communication. They unfold over time, often following predictable movement phases of preparation, stroke, and retraction. Yet, the prevalent approach to automatic gesture detection treats the problem as binary classification, classifying a segment as either containing a gesture or not, thus failing to capture its inherently sequential and contextual nature. To address this, we introduce a novel framework that reframes the task as a multi-phase sequence labeling problem rather than binary classification. Our model processes sequences of skeletal movements over time windows, uses Transformer encoders to learn contextual embeddings, and leverages Conditional Random Fields to perform sequence labeling. We evaluate our proposal on a large dataset of diverse co-speech gestures in task-oriented face-to-face dialogues. The results consistently demonstrate that our method significantly outperforms strong baseline models in detecting gesture strokes. Furthermore, applying Transformer encoders to learn contextual embeddings from movement sequences substantially improves gesture unit detection. These results highlight our framework's capacity to capture the fine-grained dynamics of co-speech gesture phases, paving the way for more nuanced and accurate gesture detection and analysis

Evaluation of effect of creative drama practises on the development of empathy skills in environmental education course by teacher candidates

Bu araştırmanın amacı çevre eğitimi dersinde yaratıcı drama uygulamalarının empati becerisini geliştirmesine etkisinin öğretmen adayları tarafından değerlendirilmesidir. Ayrıca bu çalışmada öğretmen adaylarının çevresinde bulunan varlıklarla empati kurması ve doğanın dilini öğrenmesi amaçlanmıştır. Empati en bilinen tanımıyla kişinin kendisini bir başkasının yerine koymasıdır. Empati becerisinin kazanılması sürecinde yaparak yaşayarak, deneyimleyerek ve role bürünerek oluşturulacak ortamların etkisi yadsınamaz. Bu doğrultuda çevre eğitimi dersi içerisinden konular seçilmiştir. Çevreyle öğretmen adaylarının empati kurması, farkındalık sağlaması, çevreye karşı duyarlılığın oluşması ve insan dışındaki varlıklarla empati kurma becerisine katkı sağlaması amaçlanmış ve bu süreç yaratıcı drama yöntemi kullanılarak planlanmıştır. Araştırmanın çalışma grubunu Nevşehir Hacı Bektaş Veli Üniversitesi 2018-2019 eğitim-öğretim yılında ikinci sınıfta öğrenim gören çevre eğitimi dersi alan 23 öğretmen adayı oluşturmaktadır. Çalışma nitel araştırma yöntemlerinden eylem araştırması modeliyle yürütülmüştür. Veri toplama sürecinde birden fazla veri toplama aracı kullanılmıştır. Bunlar açık uçlu soru formu, katılımcı günlükleri, araştırmacı günlüğü, öğrenci gruplarının yapmış oldukları el ürünleridir. Elde edilen veriler içerik analizi ve betimsel analiz yöntemi kullanılarak analiz edilmiştir. Günlükler içerik analiziyle, açık uçlu soru formu ise betimsel analiz yöntemi kullanılarak analiz edilmiştir. Çevre eğitimi dersinden seçilen konularla beş hafta süren ve yaratıcı drama yöntemi kullanılan planlar hazırlanmıştır. Drama yöntemiyle planlanmış çevre eğitiminden seçilen konuların yer aldığı, derslere başlamadan önce öğretmen adaylarına açık uçlu soru formları uygulanmıştır. Beş haftalık sürecin sonunda da öğretmen adaylarına aynı açık uçlu soru formu uygulanmıştır. Her dersin sonunda hem araştırmacı hem de öğretmen adayları günlük tutmuştur. Araştırmanın sonucunda, drama yöntemiyle işlenen derslerin sonunda öğretmen adaylarının canlı bir varlıkla empati kurmaya çalıştıkları, doğada bulunan herhangi bir varlığın yerine kendilerini koyabilmeyi denedikleri, sadece insanlarla değil hayvanlarla ve bitkilerle de empati kurma noktasında farkındalık kazandıkları ve empati becerilerinin gelişmesini olumlu etkilediği ortaya çıkmıştır. Ayrıca yaratıcı drama yöntemiyle işlenen derslerden öğretmen adaylarının keyif aldığı, mutlu oldukları ve çevreyle ilgili konularda dersten önce yeterince empati kurmadıkları ve bu farkındalığı derslerden sonra sağladıkları gözlemlenmiştir. Sürece katılan öğretmen adaylarının hepsi sürecin sonunda geleneksel yöntemlerle değil yaratıcı drama yöntemiyle ders işlemeyi tercih ettiklerini belirtmiştir.The aim of this study is to evaluation of effect of creative drama practıses on the development of empathy skill by teacher candidetes in environmental education lesson. Empathy is putting oneself in someone else's shoes. In the process of gaining empathy skills, the effect of the environments created by doing, experiencing and taking on a role cannot be denied. In addition, in this study, it was aimed to empathize with the living beings around and learn the language of nature. In this context, topics were chosen from the environmental education lesson. It was aimed to make teacher candidates empathize with the environment, raise awareness, create sensitivity to the environment and contribute to the ability to empathize with non-human beings and this process was planned using the creative drama method. The study group of the research consists of 23 teacher candidates who take the second grade environmental education course at Nevşehir Hacı Bektaş Veli University in the 2018-2019 academic year. The study was conducted with the action research model, one of the qualitative research methods. In the data collection process, more than one data collection tools were used. Open-ended question form, participant diaries, handmade products made by student groups, researcher diary were used as data collection tools. The obtained data were analyzed using content analysis and descriptive analysis method. The diaries were analyzed by content analysis, and the open-ended question form was analyzed using descriptive analysis method. The plans lasting five weeks and using creative drama method were prepared with the topics chosen from the environmental education course. Before starting the lessons planned with the drama method, open-ended question forms were applied to the prospective teachers. At the end of the five-week process, the same open-ended question form was applied to prospective teachers. At the end of each lesson, both the researcher and the teacher candidates kept a diary. As a result of the research; At the end of the lessons taught with the drama method, the teacher candidates try to empathize with a living being and gain awareness in terms of empathy not only with humans but also with animals and plants, and that it positively affects the development of the empathy skills. Also, it was observed that the lessons taught with the creative drama method, the teacher candidates enjoyed themselves, were happy and did not empathize before the lesson on environmental issues and gained this awareness after the lessons. All of the pre-service teachers who participated in the process stated that they preferred to teach with creative drama method rather than traditional methods at the end of the process

Development of interactional discourse markers: Insights from Turkish children's and adults’ oral narratives

Discourse markers (DMs) are linguistic elements that index different relations and coherence between units of talk (Schiffrin, Deborah, 1987. Discourse Markers. Cambridge University Press, Cambridge). Most research on the development of these forms has focused on conversations rather than narratives and furthermore has not directly compared children's use of DMs to adult usage. This study examines the development of three DMs (şey ‘uuhh’, yani ‘I mean’, işte ‘y’know’) that mark interactional levels of discourse in oral Turkish narratives in 60 Turkish children (3-, 5- and 9-year-olds) and 20 Turkish-speaking adults. The results show that the frequency and functions of DMs change with age. Children learn şey, which mainly marks exchange level structures, earliest. However, yani and işte have multi-functions such as marking both information states and participation frameworks and are consequently learned later. Children also use DMs with different functions than adults. Overall, the results show that learning to use interactional DMs in narratives is complex and goes beyond age 9, especially for multi-functional DMs that index an interplay of discourse coherence at different levels

Generalisable patterns of gesture distinguish semantic categories in communicationwithout language

There is a long-standing assumption that gestural forms aregeared by a set of modes of representation (acting,representing, drawing, moulding) with each techniqueexpressing speakers’ focus of attention on specific aspects ofreferents (Müller, 2013). Beyond different taxonomiesdescribing the modes of representation, it remains unclearwhat factors motivate certain depicting techniques overothers. Results from a pantomime generation task show thatpantomimes are not entirely idiosyncratic but rather followgeneralisable patterns constrained by their semantic category.We show that a) specific modes of representations arepreferred for certain objects (acting for manipulable objectsand drawing for non-manipulable objects); and b) that use andordering of deictics and modes of representation operate intandem to distinguish between semantically related concepts(e.g., “to drink” vs “mug”). This study provides yet moreevidence that our ability to communicate through silentgesture reveals systematic ways to describe events and objectsaround us

Pragmatic relativity: Gender and context affect the use of personal pronouns in discourse differentially across languages

Speakers need to use a variety of referring expressions (REs) (e.g. full noun phrases, pronouns or null forms) in pragmatically appropriate ways to produce coherent narratives. Languages, however, differ from each other in terms of a) whether REs as arguments can be dropped or not and b) whether personal pronouns encode gender or not. Here we examine two languages that differ from each other in these two aspects and ask whether the co-reference context (i.e., referents are maintained or re-introduced) and the gender encoding options affect the use of REs differentially. We elicited narratives from Dutch and Turkish speakers about two types of three-person events, one including people of the same and the other of mixed-gender. Speakers of both languages followed a general principle of using full forms such as noun phrases (NPs) while re-introducing a previously mentioned referent into the discourse and reduced forms (overt or null pronoun) while maintaining the same referent; a language independent strategy in discourse production. Turkish speakers, unlike Dutch speakers, used pronouns mainly to mark emphasis. Furthermore, Dutch but not Turkish speakers used pronouns differentially across the two videos. Thus, we argue that linguistic possibilities available in typologically different languages might tune speakers into taking different principles into account to establish coherence in narratives in pragmatically coherent ways