Investigating the Construct of Topical Knowledge in a Scenario-Based Assessment Designed to Simulate Real-Life Second Language Use

Abstract

The vast development of digital technology and the widespread use of social network platforms have reshaped how we live in the world. For L2 learners to maximally utilize their language proficiency to function effectively as members of modern society, they need not only the necessary L2 knowledge, skills, and abilities (KSAs) but also essential topical knowledge. While many researchers believe that topical knowledge should be viewed as an integral component of L2 communicative competence, the role of topical knowledge has not always been accounted for in an assessment context due to the difficulty of operationalizing the construct. Scenario-based assessment, an innovative, technology-based assessment approach, allows great affordances for expanding the measured constructs of an assessment. It is designed expressly for learners to demonstrate their KSAs in a context that simulates real-life language use. Through the utilization of a sequence of thematically-related tasks, along with simulated character interaction, scenario-based assessment offers opportunities to examine L2 learners’ communicative competence in a purposeful, interactive, and contextually meaningful manner. In this study, a scenario-based language assessment (SBLA) was developed to measure high-intermediate L2 learners’ topical knowledge and their L2 KSAs as part of the broadened construct of L2 communicative competence. To fulfill the scenario goal, learners were required to demonstrate their listening, reading, and writing abilities to build and share knowledge. In addition, learners’ prior topical knowledge was measured and their topical learning was tracked using the same set of topical knowledge items. A total of 118 adult EFL learners participated in the study. The results showed that the SBLA served as an appropriate measure of high-intermediate learners’ L2 proficiency. The topical knowledge items were found to function appropriately, supporting the use of the SBLA to measure topical knowledge as part of the broadened construct of communicative competence. In addition, most learners exhibited substantial topical learning over the course of the SBLA, suggesting that with proper contextualization, learning can be facilitated within an assessment. In sum, this study demonstrated the potential value of scenario-based assessment as an approach to measure complex constructs of communicative language competence in L2 context