Research methods

This paper begins with the assumption that there is no perfect study. Rather, the process of conducting language acquisition research involves numerous decisions, each of which is accompanied by a set of strengths and weaknesses and which must be justified as appropriate to the substantive domain and the research questions being addressed. The chapter describes many of these choices and their corresponding benefits and drawbacks, illustrating key concepts and techniques with examples while making frequent reference to methodological issues and trends currently taking place in the field. In particular, we focus on major decisions related to (a) research designs, both descriptive/observational and (quasi-)experimental; (b) elicitation techniques and instruments for collecting data both linguistic (e.g., grammaticality judgment tasks) and non-linguistic (e.g., questionnaires) in nature; and (c) quantitative (e.g., analysis of variance) and qualitative (e.g., grounded theory) techniques for analyzing data

An exploratory study of differing perceptions of error correction between a teacher and students: Bridging the gap

This study was designed to measure and bridge the mismatch in perceptions of error correction (EC) between a teacher and his students. The participants were 32 students in two intact, beginner-level Spanish classes at a mid-size university in the Southwest of the US. The treatment consisted of two learner training (LT) sessions (see Reiss, 1981) in which the teacher explained to the students his approach to correcting written mistakes. Alternate forms of a questionnaire were administered on three occasions in order to obtain data about the students’ opinions. A review of the literature identified four main issues related to EC that teachers and students disagree on: discouragement, learner readiness, meaning focus, and grammar. These four areas were therefore the focus of both the LT sessions and the questionnaires. The results showed a significant change in the students’ perceptions related to EC after the treatment. Thus, the paper concludes by calling for studies and methodologies that consider a more open and fluid dialogue between second language acquisition (SLA) findings, L2 classrooms, and students’ awareness of their L2 learning

The effects of perception- vs. production-based pronunciation instruction

While research has shown that provision of explicit pronunciation instruction (PI) is facilitative of various aspects of second language (L2) speech learning (Thomson & Derwing, 2015), a growing number of scholars have begun to examine which type of instruction can best impact on acquisition. In the current study, we explored the effects of perception- vs. production-based methods of PI among tertiary-level Japanese students of English. Participants (N = 115) received two weeks of instruction on either segmental or suprasegmental features of English, using either a perception- or a production-based method, with progress assessed in a pre/post/delayed posttest study design. Although all four treatment groups demonstrated major gains in pronunciation accuracy, performance varied considerably across groups and over time. A close examination of our findings suggested that perception-based training may be the more effective training method across both segmental and suprasegmental features

A primer on qualitative research synthesis in TESOL

Secondary research in the form of literature reviews facilitates consolidation and transfer of knowledge. In the field of TESOL, the majority of secondary research is conducted in the form of narrative reviews, which rely on the Plonsky’s selection and interpretation of primary studies and findings. Systematic reviews, which can be broadly categorized into meta-analysis (focusing on quantitative data) and qualitative research synthesis (focusing on qualitative data), are gaining popularity (see Plonsky, 2017) but are still less common. In particular, qualitative data collected from language classrooms, which are often criticized because of their lack of generalizability, are seldom synthesized in a systematic fashion. Against this backdrop, this article first attempts to make a case for conducting qualitative research synthesis in the field of TESOL. Second, this article provides a methodological framework and an example of how qualitative research synthesis can be conducted. The article closes with recommendations to promote qualitative research synthesis in the field of TESOL.Publisher PDFPeer reviewe

R as a Lingua Franca: Advantages of Using R for Quantitative Research in Applied Linguistics

In this article, we suggest that using R, a statistical software environment, is advantageous for quantitative researchers in applied linguistics. We first provide a brief overview of the reasons why R is popular among researchers in other fields and why we recommend its use for analyses in applied linguistics. In order to illustrate these benefits, we report recent works and developments in quantitative data analysis seeking to move the field toward more appropriate practices, many of which take advantage of the flexibility and functionality of R. Finally, in order to facilitate the use of R, we also introduce an R-based web application developed by the first author

Predicting human decisions with behavioral theories and machine learning

Behavioral decision theories aim to explain human behavior. Can they help predict it? An open tournament for prediction of human choices in fundamental economic decision tasks is presented. The results suggest that integration of certain behavioral theories as features in machine learning systems provides the best predictions. Surprisingly, the most useful theories for prediction build on basic properties of human and animal learning and are very different from mainstream decision theories that focus on deviations from rational choice. Moreover, we find that theoretical features should be based not only on qualitative behavioral insights (e.g. loss aversion), but also on quantitative behavioral foresights generated by functional descriptive models (e.g. Prospect Theory). Our analysis prescribes a recipe for derivation of explainable, useful predictions of human decisions

An invasive podosome-like structure promotes fusion pore formation during myoblast fusion

An F-actin–enriched protrusion resembling an invasive podosome promotes fusion pore formation between muscle founder cells and fusion-competent myoblasts

Crowdsourcing hypothesis tests: Making transparent how design choices shape research results

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer fiveoriginal research questions related to moral judgments, negotiations, and implicit cognition. Participants from two separate large samples (total N > 15,000) were then randomly assigned to complete one version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: materials from different teams renderedstatistically significant effects in opposite directions for four out of five hypotheses, with the narrowest range in estimates being d = -0.37 to +0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for two hypotheses, and a lack of support for three hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, while considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim.</div

Creative destruction in science

Drawing on the concept of a gale of creative destruction in a capitalistic economy, we argue that initiatives to assess the robustness of findings in the organizational literature should aim to simultaneously test competing ideas operating in the same theoretical space. In other words, replication efforts should seek not just to support or question the original findings, but also to replace them with revised, stronger theories with greater explanatory power. Achieving this will typically require adding new measures, conditions, and subject populations to research designs, in order to carry out conceptual tests of multiple theories in addition to directly replicating the original findings. To illustrate the value of the creative destruction approach for theory pruning in organizational scholarship, we describe recent replication initiatives re-examining culture and work morality, working parents\u2019 reasoning about day care options, and gender discrimination in hiring decisions. Significance statement It is becoming increasingly clear that many, if not most, published research findings across scientific fields are not readily replicable when the same method is repeated. Although extremely valuable, failed replications risk leaving a theoretical void\u2014 reducing confidence the original theoretical prediction is true, but not replacing it with positive evidence in favor of an alternative theory. We introduce the creative destruction approach to replication, which combines theory pruning methods from the field of management with emerging best practices from the open science movement, with the aim of making replications as generative as possible. In effect, we advocate for a Replication 2.0 movement in which the goal shifts from checking on the reliability of past findings to actively engaging in competitive theory testing and theory building. Scientific transparency statement The materials, code, and data for this article are posted publicly on the Open Science Framework, with links provided in the article