Assessment in a globalized economy: A task-based approach to assess the proficiency of Dutch in specific occupational domains

For language tests to be in tune with the target context, a constant attention to the shifting characteristics of real-world language use is required. This holds particularly true for language tests in the occupational domain since domestic labor market demands are continuously changing and increasingly met through the recruitment of foreign workers. This paper describes how subject experts were involved in the cyclical validation process of a test of Dutch for the professional domain. A survey of recruitment agents, employers, policy makers, language instructors, examiners, and former test takers indicated that a task-based test targeting the language skills involved in service-oriented work settings such as administration and health care at level B2 of the CEFR was favored. The involvement of subject specialists recruited among the test’s various stakeholders proved to be of vital importance throughout the development and validation process to ensure content validity and avoid biases

Assessment in a Globalized Economy: A Task-based Approach to Assess the Proficiency of Dutch in Specific Occupational Domains

For language tests to be in tune with the target context, a constant attention to the shifting characteristics of real-world language use is required. This holds particularly true for language tests in the occupational domain since domestic labor market demands are continuously changing and increasingly met through the recruitment of foreign workers. This paper describes how subject experts were involved in the cyclical validation process of a test of Dutch for the professional domain. A survey of recruitment agents, employers, policy makers, language instructors, examiners, and former test takers indicated that a task-based test targeting the language skills involved in service-oriented work settings such as administration and health care at level B2 of the CEFR was favored. The involvement of subject specialists recruited among the test’s various stakeholders proved to be of vital importance throughout the development and validation process to ensure content validity and avoid biases.No ISBN/ISSNstatus: publishe

Automated Essay Scoring in Foreign Language Students Based on Deep Contextualised Word Representations

We introduce a method for automated grading of handwritten essays written by foreign language learners of French. The handwriting recognition system allows digitising the essays for further processing and functions at a low character error rate. The transcriptions are then vectorised using embeddings from state-of-the-art pre-trained natural language processing models. On top of the extracted word-level features, a deep recurrent network was trained for grade predictions for essays, using the nine different grading criteria as target variables. Scores on these criteria were previously obtained from human expert raters for more than 6’000 student essays. We present preliminary findings on prediction accuracy and discuss possible future developments and applications of the system