This book provides a forum for methodological discussions emanating from researchers engaged in studying how individuals acquire an additional language. Whereas publications in the field of second language acquisition generally report on empirical studies with relatively little space dedicated to questions of method, the current book gave authors the opportunity to more fully develop a discussion piece around a methodological issue in connection with the interpretation of language-learning data. The result is a set of seven thought-provoking contributions from researchers with diverse interests. Three main topics are addressed in these chapters: the role of native-speaker norms in second-language analyses, the impact of epistemological stance on experimental design and/or data interpretation, and the challenges of transcription and annotation of language-learning data, with a focus on data ambiguity. Authors expand on these crucial issues, reflect on best practices, and provide in many instances concrete examples of the impact they have on data interpretation

Edmonds, Amanda

Emilsen, Linda Evenstad

Guan, Qianwen

Gudmestad, Aarnes

Hilton, Heather E.

Horgues, Céline

Huang, Xiaoliang

Leclercq, Pascale

Majdoub, Hedi

Osborne, John

Pélissier, Maud

Rast, Rebekah

Scheuer, Sylwia

Søfteland, Åshild

Trévisiol, Pascale

Watorek, Marzena

Yu, Xinyue Cécilia

English

Language Science Press

Chapter 1Introduction: Reflecting on datainterpretation in SLAAmanda EdmondsUniversité Paul-Valéry Montpellier 3Pascale LeclercqUniversité Paul-Valéry Montpellier 3Aarnes GudmestadVirginia Polytechnic Institute and State UniversityThe past decade has seen a growing number of publications that urge researchersin the field of second language acquisition (SLA) to engage more directly andmore critically with questions of research methodology. These include, amongmany others, Plonsky (2014), who makes clear recommendations for quantita-tive second-language (L2) research and issues a call for change, Leclercq et al.(2014), who call for more transparency in the assessment of L2 proficiency, Mars-den et al. (2016), who make a strong case for the importance of replication inmoving the field forward, Gudmestad & Edmonds (2018), who showcase differ-ent ways to bring critical reflections on method to the fore, and Ortega (2014),who draws attention to the need to move beyond a native-speaker bias in L2research. Although diverse in aim and scope, these endeavours and others likethem share a strong interest in movingmethodological practices forward. Byrnes(2013: 825) goes so far as to characterise this increasing interest as a “methodo-logical turn” within our field. SLA research that has come out of this turn has ledto numerous advances. To take but a few examples, underlying concepts and con-structs have been (re)defined (e.g., Pallotti 2009 on the construct of complexity-accuracy-fluency), certain well-established ways of doing things have been ques-tioned (e.g., Plonsky & Oswald’s (2017) plea to move away from ANOVA), andAmanda Edmonds, Pascale Leclercq & Aarnes Gudmestad. 2020. Introduction: Reflecting ondata interpretation in SLA. in Amanda Edmonds, Pascale Leclercq & Aarnes Gudmestad (eds.),Interpreting language-learning data, 1–8. Berlin: Language Science Press. DOI: 10.5281/zenodo.4032511Amanda Edmonds, Pascale Leclercq & Aarnes Gudmestadnew approaches have been developed and championed (see the numerous recentspecial issues devoted to both wide and narrow methodological issues: Norris etal. 2015; Choi & Richards 2016; De Costa et al. 2017; Edmonds et al. forthcoming).As a result, the methodological landscape in SLA is arguably more diverse thanever before, with Ortega (2013: 5) identifying the increase in “research methodo-logical prowess” as one of the noticeable trends in SLA research.According to King & Mackey (2016: 214), the field of SLAis in its prime. It has left behind the largely unproductive, so-called ’paradigmwars’ between those supporting quantitative and qualitative approaches. Bothcognitively and socially oriented researchers are showing greater awareness ofthe importance of incorporating a range of perspectives. The field is pushingmethodological boundaries in many directions.The pushing referred to by King and Mackey is taking many forms, includ-ing cross-disciplinary pollination and collaboration (e.g., Duff & Byrnes 2019),the use of mixed-methods, leading to an attitude of “methodological inclusivity”(Römer 2019: 478), and a growing number of scientific publications that tacklemethodological issues head on. In this final category, researchers generally aimto stimulate discussion and potentially initiate change, be this through discussionpapers, such as The Douglas Fir Group (2016) or Young (2018), or with empiricalstudies (often through reanalyses of previous published data or meta-analyses),which serve to concretely demonstrate the import and impact of methodologicalchoices (Santos et al. 2008; Leeser & Sunderman 2016; Edmonds & Gudmestad2018; Solon 2018).With the current collected volume, we aim to contribute to this focus on meth-odological issues. Specifically, we bring together a collection of seven chapters,each of which provides a new angle on the treatment or interpretation of lang-uage-learning data, a crucial issue in the building of knowledge in the field ofSLA. Three main lines of reflection are pursued in these chapters.The first concerns the question of how comparisons to a baseline norm canbe carried out in L2 research, as well as what norm might be best adopted. Inthe present volume, this question is addressed from two novel standpoints: thequestion of how to identify interlanguage forms in the dialect-rich environmentof Norway, which provides many different input forms for the same concept(Evenstad Emilsen & Søfteland) and the questioning of a general native baselinein event-related potential (ERP) studies (Pélissier).The second line of reflection, broadly speaking, concerns epistemological stancein research design. By epistemological stance, we refer to a researcher’s viewabout what constitutes knowledge in a given field. One common epistemolog-21 Introduction: Reflecting on data interpretation in SLAical tension in the field of SLA opposes two visions of language learning: “Islearning like acquiring stuff or is it like doing things?” (Young 2018: 45). Thesetwo visions lead to different positions on how to study language learning andeven as regards to what is ultimately worthy of study. Issues connected to therole of epistemological stance are visible in two chapters.WhereasWatorek, Rast,Yu, Trévisiol, Majdoub, Guan & Huang reflect on how to carry out a conceptualreplication, thereby holding constant their epistemological understanding of thephenomenon under study (namely, L2 acquisition from the very initial stages),Gudmestad purposefully sets out to follow to its logical conclusion a shift in epis-temological stance.The third question grapples with more technical issues surrounding the anno-tation, coding, and interpretation of data, especially when faced with ambiguousinterlanguage forms. Issues identified involve both multimodal data (Hilton &Osborne; Scheuer & Horgues) and difficulties specific to the transcription of oraldata (Leclercq).In the first chapter, Evenstad Emilsen & Søfteland offer reflections on SLA in adialect-rich environment. Such environments have received little explicit atten-tion in the research literature, and yet they entail challenges for both learners andfor researchers. For learners, the co-existence of multiple dialects provides an ar-guablymore complex input, one inwhich numerous forms co-exist to express thesame function. For researchers, making data coding and analysis decisions aboutlearner production is particularly challenging, as forms found in interlanguageuse may not correspond to the dominant dialect, but may be present in othervarieties. The authors detail the challenges facing researchers, providing severalexamples. They highlight the difficulties inherent in determining whether formsproduced by learners are evidence of sociolinguistic variation (i.e., variation pres-ent in the input) or instances of interlanguage variation.Pélissier’s contribution questions the comparison of native and non-native per-formance in online processing studies involving ERPs. The author shows thatalthough a large body of research into native language processing has identifieda biphasic ERP pattern when native speakers are asked to process syntactic vi-olations, recent research has called this pattern into question, showing insteadthat there is substantial inter-individual variability among native speakers. Morespecifically, when it comes to syntactic violations, most individuals show onlyone of the two components of the biphasic pattern. For the field of SLA, tradi-tionally preoccupied with comparing native and non-native performance, thisfinding begs the question of how we might meaningfully compare learners andnative speakers. Pélissier explores two approaches that hold some promise inso-3Amanda Edmonds, Pascale Leclercq & Aarnes Gudmestadfar as they allow researchers to account for individual variability: the ResponseMagnitude Index and the Response Dominance Index (Tanner et al. 2014). Thetarget structure in Pélissier’s study is past-tense morphology with auxiliaries inEnglish. Results show that the Response Dominance Index, but not the ResponseMagnitude Index, is useful in accounting for the data analysed.In the third chapter, Watorek and colleagues provide a detailed presentationof the ambitious VILLA project (Varieties of Initial Learners in Language Acquisi-tion: Controlled classroom input and elementary forms of linguistic organisation).This project seeks to provide insight into language acquisition in the first hoursof exposure to a new language. In the original VILLA project, Polish is the targetlanguage, with learners having either Dutch, English, French, German or Ital-ian as their native language. The contribution included in this volume reflectson three conceptual replications of the VILLA project, in order to study the ini-tial acquisition of Modern Standard Arabic, Mandarin Chinese, and Japanese bynative French speakers. The goal of the conceptual replications is to contributeadditional insight into language learning starting from first exposure, but withtypologically diverse languages. This diversity requires the authors to reconsiderthe target of learning (nominal morphology, in the original project), the variablescontrolled for (transparency and frequency), as well as theway of assessing learn-ing. The reflection offered by the authors raises the intriguing question of compa-rability when transposing research design and questions to study new languagecombinations.Gudmestad’s chapter directly addresses the oft-ignored issue of epistemologi-cal stance. In other words, she engages with “what counts” as knowledge in SLA.Using the concrete example of grammatical gender in L2 Spanish, she highlightsthe fact that there exist (at least) two different epistemological understandingsas to what production of gender-marked modifiers reveals about interlanguage.One position (exemplified in Gudmestad’s previous work) considers all instancesof gender marking to reveal the same underlying process, regardless of whetherthe modifier in question is an adjective or a determiner. The second position seestwo different processes at work: on the one hand, the gender marked on deter-miners is thought to reflect the gender attributed by the speaker to the nounin question (a lexical property) and, on the other, gender marked on adjectivesreveals the speaker’s ability to computemorphosyntactic agreement. In her chap-ter, Gudmestad departs from her original stance in order to “try on” the secondposition in a reanalysis of data originally published in Gudmestad et al. (2019).She thereby explores what is gained by adopting new ways of seeing data. Inso doing, Gudmestad essentially participates in what King & Mackey (2016: 214)41 Introduction: Reflecting on data interpretation in SLAterm “layering”: “Layering involves considering theory as well as practice and,in particular, considering varied epistemological stances every time one looks ata traditional problem.”The next chapter provides a concrete and critical reflection on how the toolEXMARaLDA can be profitably used to carry outmulti-tiered annotation of class-room data. Hilton and Osborne report on part of an exploratory study that tookplace in English classes held in two French elementary schools. After detailingthe development of their multi-layered approach to transcribing and annotatingthree weeks of language lessons, the authors focus on data from one lesson fromeach classroom in order to demonstrate how conducting analyses at differentlevels of annotation may lead to the identification of the differences in the twolearning environments that triggered different learning outcomes for the stu-dents (regarding memorization of new vocabulary and utterance construction)Although the authors highlight that the analyses are limited in scale and thuscannot be used to suggest pedagogical implications, they demonstrate that thetwo classrooms are not equally effective, which is visible, for example, in theorganisation of pupil and teacher talk.Chapter six focuses on how theory, data coding, and data transcription inter-sect. To accomplish this goal, Leclercq uses the example of verb-final [e] in L2French. Verb-final [e] in French can correspond to the infinitive form (parler ‘tospeak’), imperfective forms (e.g., parlais ‘(you) speak’, parlait ‘(s/he) speaks’), thefirst-person simple past form (parlai ‘(I) spoke’), and various forms of the pastparticiple (parlé, parlés, parlée, parlées). In otherwords, one spoken form– [paʁle]– is highly homophonous. This leads to a clear challenge for any researcher work-ing on oral productions in L2 French. How does one transcribe a form like [paʁle]when produced by a learner? Leclercq takes up this thorny issue and criticallydetails how other studies in SLA research have dealt with it. She concludes byshowing that some transcription choices result from a premature categorisationof the data, often reflecting theoretical positioning and potentially introducinginterpretative bias.The volume closes with a chapter devoted to identifying and reflecting on po-tential pitfalls involved in analysing data from English-French tandem conversa-tions. Scheuer and Horgues report on data collected from 21 tandem pairs duringa semester-long programme at a French university. Each tandem is made up ofa native speaker of French and of English and was recorded on two occasions(once at the beginning and once at the end of the semester). For each record-ing, approximately half of the speaking time is in each of the two languages.The authors use these data to explore corrective feedback and communication5Amanda Edmonds, Pascale Leclercq & Aarnes Gudmestadbreakdowns, addressing, among other things, which member of the tandem ini-tiated the feedback or signalled the breakdown and what type of issue (lexis,pronunciation, syntax, etc.) led to the feedback or breakdown. The authors of-fer a thought-provoking discussion on the difficulties involved in determiningboth what constitutes corrective feedback or comprehension breakdowns and inpinpointing what linguistic issue was the cause (or causes) for either. They thusprovide clear and concrete examples of dealing with ambiguity in learner data inan L2 analysis.The seven chapters brought together in this volume offer original and timelycontributions on the role of (native-speaker) norms in L2 analyses, on the im-pact of epistemological stance, and on the challenges of transcription and anno-tation of language-learning data. In addressing these issues, the researchers relyon a variety of methodological practices and highlight in their chapters the im-port of methodological choice. These choices have a far-reaching impact, as theyconstrain and orient what observations can be made in research and what con-clusions are ultimately drawn. We hope to have demonstrated with this volumethat reflecting on these decisions – making them explicit and holding them upto study – is indeed a valuable enterprise.ReferencesByrnes, Heidi. 2013. Notes from the editor. The Modern Language Journal 97(4).825–827.Choi, Seongsook & Keith. (eds.) Richards. 2016. Innovation in research methods.Applied Linguistics [Special issue], 37(1).De Costa, Peter, Hima I. Rawal & Irina Zaykovskaya (eds.). 2017. Study abroadin contemporary times: Toward greater methodological diversity and innovation.System [Special issue], 71.The Douglas Fir Group. 2016. A transdisciplinary framework for SLA in a multi-lingual world. The Modern Language Journal 100. 19–47.Duff, Patricia A. & Heidi Byrnes. 2019. SLA across disciplinary borders: New per-spectives, critical questions, and research possibilities. The Modern LanguageJournal 103 (Suppl. 1).Edmonds, Amanda & Aarnes Gudmestad. 2018. Operationalizing variables: Thecase of future-time expression in L2 French. In Aarnes Gudmestad & AmandaEdmonds (eds.), Critical reflections on data in second language acquisition, 125–148. Amsterdam: Benjamins.61 Introduction: Reflecting on data interpretation in SLAEdmonds, Amanda, Pascale Leclercq &Nathalie Auger (eds.). Forthcoming.Meth-odological challenges in SLA: A focus on French. Journal of French LanguageStudies (30) [Special issue].Gudmestad, Aarnes & Amanda Edmonds (eds.). 2018. Critical reflections on datain second language acquisition. Amsterdam: Benjamins.Gudmestad, Aarnes, Amanda Edmonds & Thomas Metzger. 2019. Using varia-tionism and learner corpus research to investigate grammatical gender mark-ing in additional-language Spanish. Language Learning 69. 911–949.King, Kendall A. & Alison Mackey. 2016. Research methodology in second lan-guage studies: Trends, concerns, and new directions. The Modern LanguageJournal 100(Suppl.). 209–227.Leclercq, Pascale, Amanda Edmonds & Heather E. Hilton (eds.). 2014. MeasuringL2 proficiency: Perspectives from SLA. Clevedon, UK: Multilingual Matters.Leeser, Michael J. & Gretchen L. Sunderman. 2016. Methodological implicationsof working memory tasks for L2 processing research. In Gisela Granena,Daniel O. Jackson & Yucel Yilmaz (eds.), Cognitive individual differences in sec-ond language processing and acquisition, 89–104. Amsterdam: Benjamins.Marsden, Emma, Alison Mackey & Luke Plonsky. 2016. The IRIS repository: Ad-vancing research practice and methodology. In Alison Mackey & Emma Mars-den (eds.), Advancing methodology and practice: The IRIS repository of instru-ments for research into second languages, 1–21. New York: Routledge.Norris, John M., Steven J. Ross & Rob Schoonen (eds.). 2015. Improving and ex-tending quantitative reasoning in second language research. Language Learning[Special issue], 65, S1.Ortega, Lourdes. 2013. SLA for the 21st century: Disciplinary progress, transdisci-plinary relevance, and the bi/multilingual turn. Vol. 63. 1–24.Ortega, Lourdes. 2014. Ways forward for a bi/multilingual turn in SLA. In S. May(ed.), The multilingual turn: Implications for SLA, TESOL, and bilingual educa-tion, 32–53. New York: Routledge.Pallotti, Gabriele. 2009. CAF: Defining, refining and differentiating constructs.Applied Linguistics 30(4). 590–601.Plonsky, Luke. 2014. Study quality in quantitative L2 research (1990-2010): Amethodological synthesis and call for reform. Modern Language Journal 98.450–470.Plonsky, Luke & Frederick L. Oswald. 2017. Multiple regression as a flexible al-ternative to ANOVA in L2 research. Studies in Second Language Acquisition 39.579–592.7Amanda Edmonds, Pascale Leclercq & Aarnes GudmestadRömer, Ute. 2019. Corpus research for SLA: The importance of mixing methods.In Viola Wiegand & Michaela Mahlberg (eds.), Corpus linguistics, context andculture, 467–481. Berlin: De Gruyter.Santos, Denise, Suzanne Graham & Robert Vanderplank. 2008. Second languagelistening strategy research: Methodological challenges and perspectives. Eval-uation & Research in Education 21(2). 111–133.Solon, Megan. 2018. Novel sounds: What L2 phonetic data might be telling usthat we do not always hear. In Aarnes Gudmestad & Amanda Edmonds (eds.),Critical reflections on data in second language acquisition, 89–124. Amsterdam:Benjamins.Tanner, Darren, Kayo Inoue & Lee Osterhout. 2014. Brain-based individual differ-ences in online L2 grammatical comprehension. Bilingualism: Language andCognition 17(2). 277–293.Young, Richard F. 2018. Habits of mind: How do we knowwhat we know? In AekPhakiti, Peter De Costa, Luke Plonsky & Sue Starfield (eds.), The Palgrave hand-book of applied linguistics research methodology, 31–53. New York: PalgraveMacmillan.8

Interpreting language-learning data

https://langsci-press.org/catalog/view/278/2557/1986-1

Interpreting language-learning data

Abstract

Similar works

Full text

Available Versions

Language Science Press