Rehearsing L2 academic vocabulary with cloze exercises: a computer-assisted language learning intervention

Choosing appropriate methods and levels of scaffolding (see, e.g., Wood, Bruner, & Ross, 1976) is a crucial skill in second language instruction. The observation that too little or too much scaffolding for a task leads to an inferior learning outcome, known as the assistance dilemma (Koedinger & Aleven, 2007), has resisted quantitative analysis. However, it is now possible to take advantage of computerized tutors’ ability to precisely measure response latencies and accuracy rates to provide quantitative data to analyze the merits of different methods of scaffolding with regard to students’ performance on individual tasks. The present study describes a computer-aided language learning intervention in which 46 intermediate-level adult ESL speakers used a web-based vocabulary rehearsal program several times over the course of nine weeks. The tutor led participants in completing cloze exercises of the target words, with half of the exercises being presented with a hint in the form of a short definition of the target word and half of the exercises being presented without a hint. The results of the experiment indicate that the presence of the hint significantly increased participants’ accuracy on the task, but also significantly increased time on task. These results suggest that the form of support selected was an appropriate scaffold. However, L1 speakers of Arabic (N = 29) proved exceptional in a few ways: they expressed negative attitudes toward L2 writing tasks in general and did not show any increase in accuracy in the scaffolded condition, despite the fact that speakers of other L1s showed a very large and statistically significant improvement in accuracy in that condition. These issues may relate to Arabic speakers’ exceptional difficulties processing English orthography (Martin, 2011) and warrant future study

Development of parallel cloze tests using cohesion

My thesis is simple: cohesion, the interdependence of textual elements, can be used to build reliable doze tests. Furthermore, reliable doze tests built using cohesion are parallel. This paper seeks to explore those hypotheses, and point out why the issues surrounding them are important. Developing good language tests is difficult. Tests must be both reliable and valid for their interpretations and uses. To be reliable, the potential sources of error in the test itself must be minimized. Reliability is affected by not only the examinee\u27s language ability, but also by test methods, attributes of the test taker not related to the language abilities we want to measure, and random factors. Ideally, only the language abilities that we want to measure are reflected by the test scores, but in practice, this is not the case. Unfamiliarity with a test method, or dislike of a test method, affects the examinee\u27s performance. A test on listening, for example, should not depend on a student\u27s ability to comprehend complex written instructions. Finally, random factors such as lighting, what a test taker had for breakfast, or the temperature in the testing environment can have an effect on scores. In order for a test to be reliable, these influences must be minimized

Storyfier: Exploring Vocabulary Learning Support with Text Generation Models

Vocabulary learning support tools have widely exploited existing materials, e.g., stories or video clips, as contexts to help users memorize each target word. However, these tools could not provide a coherent context for any target words of learners' interests, and they seldom help practice word usage. In this paper, we work with teachers and students to iteratively develop Storyfier, which leverages text generation models to enable learners to read a generated story that covers any target words, conduct a story cloze test, and use these words to write a new story with adaptive AI assistance. Our within-subjects study (N=28) shows that learners generally favor the generated stories for connecting target words and writing assistance for easing their learning workload. However, in the read-cloze-write learning sessions, participants using Storyfier perform worse in recalling and using target words than learning with a baseline tool without our AI features. We discuss insights into supporting learning tasks with generative models.Comment: To appear at the 2023 ACM Symposium on User Interface Software and Technology (UIST); 16 pages (7 figures, 23 tables

Vocabulary Learning Exercises: Evaluating a Selection of Exercises Commonly Featured in Language Learning Materials

This chapter examines common second language (L2) vocabulary learning activities using a framework proposed by Nation (2013a). In particular, Nation’s first guideline, which focuses on efficacy, will be examined in detail. Nation and Webb’s (2011) Technique Feature Analysis (TFA) will be used to determine which components of the activities contribute to learning. The chapter aims to gauge the relative efficacy of three vocabulary learning activities: Learning from flashcards, cloze exercises, and crossword puzzles; shed some light on their strengths and weaknesses; and show how they might be modified to be made more effective

The effects of inquiry- based learning on students’ institutional TOEFL score at english department of IAIN Palangka Raya

The study is aimed at measuring the effect of Inquiry based learning on the students’ institutional TOEFL score, before, during, and after the treatment. The study belongs to an experimental study using repeated measures design. It is carried out at the sixth semester of English Department at IAIN Palangka Raya 2015/ 2016 academic years. The subjects are 23 students. The One Way ANOVA Repeated measure is applied in this study. Before testing the hypotheses, a number of pre-required tests is conducted, such as normality, homogeneity and Sphericity tests. The result of Shapiro- Wilk test for testing normality was that (p= 0.832 > 0.05), the Levene test for testing homogeneity (p= 0.276 > 0.05); and Mauchly test for testing Sphericity (p=0.000<0.05). The One Way ANOVA Repeated measures test found that the value of Wilk’s Lamda= 0.139, F= 64.78, the value of Sig. 0.000 or p< 0. 005, the value of eta squared= 0.861. It could be interpreted that there was a significant statistical difference on the students’ TOEFL scores before the treatment (means 436), during (means 490) and after the implementation of Inquiry Based Learning (means 511)

A set of free cross-platform authoring programs for flexible web-based CALL exercises

[EN] The Mango Suite is a set of three freely downloadable cross-platform authoring programs for flexible network-based CALL exercises. They are Adobe Air applications, so they can be used on Windows, Macintosh, or Linux computers, provided the freely-available Adobe Air has been installed on the computer. The exercises which the programs generate are all Adobe Flash based. The three programs are: (1) Mango-multi, which constructs multiple-choice exercises with an optional sound and/or image; (2) Mango-match, which is for word/phrase matching exercises, and has an added feature intended to promote memorization, whereby an item must be matched correctly not once but an optional consecutive number of times; (3) Mango-gap, which produces seamless gap filling exercises, where the gaps can be as small as desired, down to the level of individual letters, and correction feedback is similarly detailed. Sounds may also be inserted at any desired points within the text, so that it is suitable for listening or dictation exercises. Each exercise generated by any of the programs is produced in the form of a folder containing all of the necessary files for immediate upload and deployment (except that if sound files are used in a Mango-gap exercise, they must be copied to the folder manually). The html file in which the flash exercise is embedded may be edited in any way to suit the user, and an xml file controlling the appearance of the exercise itself may be edited through a wysiwyg interface in the authoring program. The programs aim to combine ease of use with features not available in other authoring programs, toprovide a useful teaching and research tool.O’brien, M. (2012). A set of free cross-platform authoring programs for flexible web-based CALL exercises. The EuroCALL Review. 20(2):59-68. https://doi.org/10.4995/eurocall.2012.11378SWORD5968202Butler, A. C. and Roediger, H. L. (2008). Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & Cognition, 36(3): 604-616. Available from: http://duke.edu/~ab259/pubs/Butler&Roediger(2008).pdf https://doi.org/10.3758/MC.36.3.604Folse, K. S. (2004). Vocabulary Myths: Applying Second Language Research to Classroom Teaching. Ann Arbor: University of Michigan Press. https://doi.org/10.3998/mpub.23925Godwin-Jones, R. (2010). Emerging technologies - from memory palaces to spacing algorithms: approaches to second-language vocabulary learning. Language Learning & Technology, 14(2): 4-11. Available from: http://llt.msu.edu/vol14num2/emerging.pdfGoto, T., Kojiri, T., Watanabe, T., Iwata, T. and Yamada, T. (2010). Automatic Generation System of Multiple-Choice Cloze Questions and its Evaluation. Knowledge Management & E-Learning, 2(3): 210-224. Available from: http://kmel-journal.org/ojs/index.php/online-publication/article/view/72/53Hewer, S. (2011). Text Manipulation. In: Davies, G. (ed.) Introduction to Computer Assisted Language Learning (CALL) - Module 1.4 of Information and Communications Technologies for Language Teachers (ICT4LT), Slough: Thames Valley University [Online]. Available from: http://www.ict4lt.org/en/en_mod1-4.htm#textmanipHorst, M., Cobb, T. and Nicolae, I. (2005). Expanding academic vocabulary with an interactive on-line database, Language Learning & Technology, 9(2): 90-110. Available from: http://llt.msu.edu/vol9num2/horst/default.htmlKim, D., & Gilman, D. A. (2008). Effects of Text, Audio, and Graphic Aids in Multimedia Instruction for Vocabulary Learning. Educational Technology & Society, 11(3), 114-126. Available from: http://www.ifets.info/journals/11_3/9.pdfKrashen, S. D. (1981). Principles and Practice in Second Language Acquisition. London: Prentice-Hall International.Ma, Q, and Kelly, P. (2006). Computer assisted vocabulary learning: design and evaluation. Computer Assisted Language Learning, 19(1), 15-45. https://doi.org/10.1080/09588220600803998Nakata, T. (2008). English vocabulary learning with word lists, word cards and computer: implications from cognitive psychology research for optimal spaced learning. ReCALL, 20(1): 3-20. https://doi.org/10.1017/S0958344008000219Pino, J., Heilman, M. and Eskenaz, M. (2008). Selection Strategy to Improve Cloze Question Quality. In: Intelligent Tutoring Systems for Ill-Defined Domains: Assessment and Feedback in Ill-Defined Domains, Proceedings of the 9th International Conference on Intelligent Tutoring Systems, Montreal Canada: 22-34. Available from: http://www.philippe-fournier-viger.com/ill-defined/Workshop-ITS08-ill-defined.pdfRoediger, H. L. and Marsh, E.J. (2005). The Positive and Negative Consequences of Multiple-Choice Testing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(5): 1155-1159. Available from: http://www.niu.edu/~britt/courses/Roediger_Marsh_pos_neg_testing.pdf https://doi.org/10.1037/0278-7393.31.5.1155Yun, S., Miller, P. C., Baek, Y., Jung, J. and Ko, M. (2008). Improving recall and transfer skills through vocabulary building in web-based second language learning: An examination by item and feedback type, Educational Technology & Society, 11(4): 158-172. Available from: http://www.ifets.info/journals/11_4/12.pdf

Development, implementation, and evaluation of an online English placement test at college level: a case study.

The primary purpose of the present project was to research the case study of current English placement practices at Intercollege in view of incorporating change, improvement and efficiency, within the framework of current work based learning and applied linguistics (and more particularly English online language testing) research discipline. The review of work based learning and current theories and practices in applied linguistics research discipline helped establish the characteristics of an insider researcher and the research approach and research techniques that would best serve such a project. The review of current theories and practices in second language (L2) teaching and learning in general, and in L2 testing in particular revealed that there is an extensive range of practices: these range from testing discrete points to integrative tasks. Tests are also delivered both in pen-and-paper as well as in electronic form, the latter being either computer based testing (CBT) or computer adaptive testing (CAT). The review of current English placement practices at Intercollege indicated the need for a new English placement test, developed in a scientific way, informed by current theories and practices, based on current test design models and taking advantage of more efficient methods of delivery, and placement. This review also revealed the need for more efficiency in the mode of delivery, administration, marking, reporting and test duration. Finally, this study of the current English placement practices at Intercollege established the need for a placement test that would incorporate a mechanism of continuous testing of reliability and validity as well as improvement. The detailed study of the specific context, setting, particular language programme, resources, test-takers, instructors, etc. informed by current theories and practices in second language (L2) testing online, helped in the development of the New English Placement Test Online (NEPTON) test specifications, and as a consequence, the development of the proposed test itself. The study of test delivery modes and the consideration of the specific work based conditions and requirements. For example administration, delivery, time and money efficiency, urgent need of an improved and more efficient English placement test (EPT) resulted in the selection of computer based testing delivery, with many features of the computer adaptive testing delivery mode incorporated in it such as randomized selection of test items and fewer items. The test item writing and item modération process resulted in the formation of a substantial pool of varied items in different skills, text types, topics, settings, and covering a variety of lexical and grammatical points and communicative, authentic-like situations in ali six levels. The field test which was took place in May 2004 in pen-and-paper form by almost 1200 students in ali three Intercollege campuses helped check the content and the test trial which took place in the period of August-September in its electronic form helped come up with the test cutoff points, and the fine-tuning of the test. The item analysis ensured the appropriateness of ali items. Pre-test questionnaires established test-takers' biographical data and information about test-taker computer familiarity. The test face validity (stakeholders' attitudes and feelings about the NEPTON) was established through the use of pre and post-test questionnaires. Experts in the area Coming, from the three campuses, also studied the test specifications and the test itself (both in its electronic and pen-and-paper format) and completed a questionnaire, thus contributing to the establishment of the test content and construct validity. The test reliability was established through a split half reliability index process and a series of other aspects or processes such as the size of the item bank, the instructions, the moderation process, and the item analysis, which are explained in chapter 5 in more details. The research project consists of two components: (a) The report, which describes the way work based and applied linguistics research approaches were used to investigate the case study of English placement test at college level at Intercollege in Cyprus and to what extent this has broad change, improvement and effìciency to current practices; and (b) The evidence of such a research project, which is the New English Placement Test Online (NEPTON), in other words, the test itself, developed, implemented and evaluated in order to materialize this change, improvement and efficiency aimed at by this project

Test-taking Strategies on Reading Comprehension Tests: A Review of Major Research Themes

There is a growing body of thought and research on strategy use on tests of reading comprehension. Nevertheless, there have been few research reviews that have treated major themes involved in thought and research on test-taking strategies, specifically in the context of reading comprehension. Hence, this paper reviews the themes that are central to the discussion of strategy choice and use on reading comprehension tests. Research themes that form the foci of the paper include discussion of test-taking strategies as they relate to the process of reading test-taking, formats of reading tests, validation of reading tests, level of language proficiency, and performance on tests of reading comprehension

Predicting and Manipulating the Difficulty of Text-Completion Exercises for Language Learning

The increasing levels of international communication in all aspects of life lead to a growing demand of language skills. Traditional language courses compete nowadays with a wide range of online offerings that promise higher flexibility. However, most platforms provide rather static educational content and do not yet incorporate the recent progress in educational natural language processing. In the last years, many researchers developed new methods for automatic exercise generation, but the generated output is often either too easy or too difficult to be used with real learners. In this thesis, we address the task of predicting and manipulating the difficulty of text-completion exercises based on measurable linguistic properties to bridge the gap between technical ambition and educational needs. The main contribution consists of a theoretical model and a computational implementation for exercise difficulty prediction on the item level. This is the first automatic approach that reaches human performance levels and is applicable to various languages and exercise types. The exercises in this thesis differ with respect to the exercise content and the exercise format. As theoretical basis for the thesis, we develop a new difficulty model that combines content and format factors and further distinguishes the dimensions of text difficulty, word difficulty, candidate ambiguity, and item dependency. It is targeted at text-completion exercises that are a common method for fast language proficiency tests. The empirical basis for the thesis consists of five difficulty datasets containing exercises annotated with learner performance data. The difficulty is expressed as the ratio of learners who fail to solve the exercise. In order to predict the difficulty for unseen exercises, we implement the four dimensions of the model as computational measures. For each dimension, the thesis contains the discussion and implementation of existing measures, the development of new approaches, and an experimental evaluation on sub-tasks. In particular, we developed new approaches for the tasks of cognate production, spelling difficulty prediction, and candidate ambiguity evaluation. For the main experiments, the individual measures are combined into an machine learning approach to predict the difficulty of C-tests, X-tests and cloze tests in English, German, and French. The performance of human experts on the same task is determined by conducting an annotation study to provide a basis for comparison. The quality of the automatic prediction reaches the levels of human accuracy for the largest datasets. If we can predict the difficulty of exercises, we are able to manipulate the difficulty. We develop a new approach for exercise generation and selection that is based on the prediction model. It reaches high acceptance ratings by human users and can be directly integrated into real-world scenarios. In addition, the measures for word difficulty and candidate ambiguity are used to improve the tasks of content and distractor manipulation. Previous work for exercise difficulty was commonly limited to manual correlation analyses using learner results. The computational approach of this thesis makes it possible to predict the difficulty of text-completion exercises in advance. This is an important contribution towards the goal of completely automated exercise generation for language learning