Esteve-Gibert, Núria

Feijóo Antolín, Sara

Raquel, Serrano

Suárez, Maria del Mar

Vasylets, Olena

English

Serrano, Raquel

Diposit Digital de la Universitat de Barcelona

The children’s use of tactile and visual information (vs. acoustic information) when learning non-native phonological contrastsNúria Esteve-Gibert1, Mª del Mar Suárez2, Lena Vasylets2, Sara Feijoo2, Raquel Serrano2Satellite workshop “First and second language acquisition of phonology and its interfaces” Eivissa, January 27-29● Definition: “overlapping information across auditory, visual, tactile, and/or proprioceptive stimulation for properties of objects and events”, ● There is intersensory redundancy when the same information is available and temporally synchronized across two or more sense modalities● By increasing saliency, it promotes infants’ attention and perceptual processing to some properties, objects, and events of the environment (at the expense of others - so infants learn to ignore irrelevant variability)● Pre-exposing learners to bimodal information increases their learning gains in unimodal situations● Although focus on infancy, IRH predicts that its principles can be extended to older infants, children, and adults when task demands and cognitive load increasePerceiving speech through various senses: the Intersensory Redundancy Hypothesis (Bahrick & Lickliter, 2012)Perceiving speech through various senses: the Intersensory Redundancy Hypothesis (Bahrick & Lickliter, 2012)● Definition: “overlapping information across auditory, visual, tactile, and/or proprioceptive stimulation for properties of objects and events”, ● There is intersensory redundancy when the same information is available and temporally synchronized across two or more sense modalities● By increasing saliency, it promotes infants’ attention and perceptual processing to some properties, objects, and events of the environment (at the expense of others - so infants learn to ignore irrelevant variability)● Pre-exposing learners to bimodal information increases their learning gains in unimodal situations● Although focus on infancy, IRH predicts that its principles can be extended to older infants, children, and adults when task demands and cognitive load increaseResearch questionWhether redundancy between acoustic, visual, and tactile phonetic information could help young children (aged 4-5) to learn novel phonemes?Acoustic information in children’s L2 phoneme acquisition- Young infants rely on statistical information of the acoustic input to acquire novel sounds- They learn nonative phonetic contrasts better when they are presented in a bimodal distribution pattern (Maye et al 2008)bimodal distribution: more distinguishable stimuli in the continuum are more frequentunimodal distribution: less distinguishable stimuli in the continuum are more frequente.g.    /ɪ/  -  /iː/ /ɪ/                                                                 /iː/● The inspection of mouth movements while speaking provides redundant information to the acoustic signal (Gogate, Walker-Andrews, & Bahrick, 2001).● In general, visual input influences the young learners’ perception and identification of novel phonemes ○Visual information in children’s L2 phoneme acquisitionYoung infants distinguish two languages (French & English) even if exposed to only VO (Weikum et al 2007)● young infants (up to 12-18 months of age)○ distinguish two languages even if exposed to only VO (Weikum et al 2007)○ look more at the mouth when having to learn new languages (e.g., Pons et al., 2015)○ but not clear evidence that AV (vs AO) exposure to novel phonemes increases infants’ perception of these phonemes (ter Schure et al, 2016)  ● preschoolers (3-5 years of age)○ their sensitivity to visual input does not predict AV speech perception (e.g., Erdener, 2007)○ their looking preference for the mouth does not predict nonnative phonological learning (Esteve-Gibert & Muñoz, 2020)● school-aged children (6+ years of age)○ their lip-reading ability predicts AV nonnative speech perception (e.g., Erdener & Burnham 2013)Visual information in children’s L2 phoneme acquisitionNo evidence to date from L2 acquisition, but evidence from L1 speech perception● Audio-visual-tactile integration can influence the distinction between two phonological categories with different degrees of aspiration (Derrick et al., 2019; Gick & Derrick, 2009)          /pa/-/ba/               /ta/-/da/● Deafblind speakers use tactile signing (Eberhart et al 1993)● PROMPT approach uses tactile-kinesthetic feedback to rehabilitate articulatory speech movements in childhood apraxia of speech (Dale & Hayden, 2012; 2013)● Neuman et al (2012): “A key component of these [multisensory] strategies may be their incorporation of both a tactile and kinaesthetic component (e.g., tracing and manipulating letters with fingers) in addition to the traditional visual and auditory only methods (e.g., instructor presents a printed letter and says the letter name and/or sound). The recommendation is made that teachers and parents use multisensory strategies to help scaffold early literacy learning. However, further carefully controlled studies are critical to determine the exact benefits of multisensory processes and instruction in early literacy learning.”Tactile (haptic) information in children’s L2 phoneme acquisitionPROMPT approachAims of our studyTo investigate if the perception of speech movements through acoustic, visual, and tactile modalities increases the children’s (perceptive) learning of novel L2 phonological categories- Is pre-exposure to unimodal input (audio-only) less effective than to bimodal (audio-visual) or tri-modal input (audio-visual-tactile) in child’s L2 phonological learning?- Is bimodal and tri-modal input effective when task demands and cognitive load are high?Aims of our studyTo investigate if the perception of speech movements through acoustic, visual, and tactile modalities increases the children’s (perceptive) learning of novel L2 phonological categories- Is pre-exposure to unimodal input (audio-only) less effective than to bimodal (audio-visual) or tri-modal input (audio-visual-tactile) in child’s L2 phonological learning?- Is bimodal and tri-modal input effective when task demands and cognitive load are high? IRHThe learning taskParticipants● 47 children, aged between 4 and 5 ○ 39 children spoke Catalan and/or Spanish at home ○ 8 children spoke Catalan and/or Spanish plus another language ■ French, Polish, Persian, German, Urdu, Filipino, Galician■ only in 3 there is more exposure to the other language than to Cat or Sp● No hearing or language disorders● No distinction in the subsequent analyses between Catalan dominant and Spanish dominant because in none of these languages the target phonological contrast is present in the grammarMaterials● /æ/-/ʌ/ contrast (non-existing in Catalan or Spanish)● Minimal pairs of monosyllabic CVC words with voiceless coda● In the 3 learning sessions: 36 non-words produced by 3 native speakers (HVPT)● In the pre & post-test: 12 trained non-words + 6 untrained non-words + 6 untrained wordsProcedure● learning intervention, randomized-controlled trial● children were assigned to one of these three conditions in the training phase: audio-only, audio-visual, audio-visual-tactilepre-test phase (perception AX task)learning phase             (3 session; immediate repetition of AV input)post-test phase (perception AX task)Procedure● learning intervention, randomized-controlled trial● children were assigned to one of these three conditions in the training phase: audio-only, audio-visual, audio-visual-tactilepre-test phase (perception AX task)learning phase             (3 session; immediate repetition of AV input)post-test phase (perception AX task)‘acoustic’ proprioceptive feedback+ ‘visual’ proprioceptive feedback+ ‘tactile’ proprioceptive feedbackResults: perceptive abilities before and after trainingTime: F(1, 260)=1.696, p=.19, ɲ2=.007Condition: F(2, 260)=.859, p=.42, ɲ2=.007Time*Condition: F(2, 260)=.158, p=.85, ɲ2=.001Results: learning gains between pre- and post-testCondition: F(2, 128)=.273, p=.76, ɲ2=.04Word_nonword: F(1, 128)=.554, p=.46, ɲ2=.004Condition*Word_nonword: F(2, 128)=.202, p=.82, ɲ2=.003Condition: F(2,128)=.153, p=.86, ɲ2=.002Trained_untrained: F(1,128)=4.696, p=.03, ɲ2=.037Condition*Trained_untrained: F(2, 128)=.598, p=55., ɲ2=.010Results: learning gains and variabilityCondition: F(2, 128)=.273, p=.76, ɲ2=.04Word_nonword: F(1, 128)=.554, p=.46, ɲ2=.004Condition*Word_nonword: F(2, 128)=.202, p=.82, ɲ2=.003Condition: F(2,128)=.153, p=.86, ɲ2=.002Trained_untrained: F(1,128)=4.696, p=.03, ɲ2=.037Condition*Trained_untrained: F(2, 128)=.598, p=55., ɲ2=.010Discussion● Pre-schoolers’ perceptive abilities in L2 phonological contrasts do not improve more after being exposed to bimodal and tri-modal input, compared to when only exposed to unimodal input● Adding tactile feedback to the visual and acoustic feedback during the training did not lead to higher learning gains. ● The multimodal input was not more effective in challenging situations with higher cognitive load:○ when having to generalize the learned contrasts to new untrained stimuli○ children showed higher gains only in trained stimuli, independently of the learning conditionDiscussion● Our results contradict the predictions of the Intersensory Redundancy Hypothesis:○ Pre-exposing young children to bimodal information did not increase their L2 learning gains in unimodal situations○ The IRH principles are not extended to older children when tasks demands and cognitive load increase● Our results suggest that the IRH is more effective in early stages of perceptual development ● Our results coincide with the very scare existing results on the effectiveness of tactile input in L2 learning (see Özakin, 2020; who used a very similar procedure to ours)Follow-up: New Touch’n’Learn AppTouch ’n’ Learn APPThank you!Gràcies!

The children’s use of tactile and visual information (vs. acoustic information) when learning non-native phonological contrasts

http://diposit.ub.edu/dspace/bitstream/2445/178679/1/2020_eivissa_satellite.pdf

The children’s use of tactile and visual information (vs. acoustic information) when learning non-native phonological contrasts

Abstract

Similar works

Full text

Available Versions

Diposit Digital de la Universitat de Barcelona