Construct validity of the Nepalese school leaving english reading test

There has been a steady interest in investigating the validity of language tests in the last decades. Despite numerous studies on construct validity in language testing, there are not many studies examining the construct validity of a reading test. This paper reports on a study that explored the construct validity of the English reading test in the Nepalese school leaving examination. Eight students were asked to take the test and think-aloud, followed by retrospective interviews. Additionally, seven experts were asked to make judgments regarding the skills tested by the test. The findings provide grounded insights into students’ response behaviors prompted by the reading tasks, and indicate some threats to the construct validity of the test. Additionally, the study reports a low level of agreement among the experts, and a big gap between the skills used by the students and the skills that the experts thought were being examined by the test

Using the think aloud protocol to investigate vocabulary strategies used in two vocabulary tests

My interest in vocabulary research probably stems from my personal struggle with English words. As an English as a second language learner I have had the frustrating experience of not finding the appropriate words for a given context. While I did not want to sound too formal or bookish, I was not certain about the degree of informality of some words. Preparing to become an English as a foreign language teacher, I became interested in strategies for teaching and learning vocabulary. I also became interested in language testing, especially in the social effects of tests on the lives of language learners

Bias and Equivalence in Cross-Cultural Research

Bias and equivalence are key concepts in the methodology of cross-cultural studies. Bias is a generic term for any challenge of the comparability of cross-cultural data; bias leads to invalid conclusions. The demonstration of equivalence (lack of bias) is a prerequisite for any cross-cultural comparison. we first describe considerations that are relevant when choosing instruments in a cross-cultural study, notably the question of whether an existing or new instrument is to be preferred.We then describe the definition, manifestation, and sources of three types of bias (construct, method, and item bias), and three levels of equivalence (construct, measurement unit, and full score equivalence). We provide strategies to minimize bias and achieve equivalence that apply either to the design, implementation, or statistical analysis phase of a study. The need to integrate these strategies in cross-cultural studies is emphasized so as to increase the validity of conclusions regarding cross-cultural similarities and differences and rule out alternative explanations of cross-cultural differences

The feasibility of capturing learner interactions based on logs informed by eye-tracking and remote observation studies

Two small studies, one an eye-tracking study and the other a remote observation study, have been conducted to investigate ways to identify two kinds of online learner interactions: users flicking through the web pages in "browsing" action, and users engaging with the content of a page in "learning" action. The video data from four participants of the two small studies using the OpenLearn open educational resource materials offers some evidence for differentiating between 'browsing' and 'learning'. Further analysis of the data has considered possible ways of identifying similar browsing and learning actions based on automatic user logs. This research provides a specification for researching the pedagogical value of capturing and transforming logs of user interactions into external forms of representations. The paper examines the feasibility and challenge of capturing learner interactions giving examples of external representations such as sequence flow charts, timelines, and table of logs. The objective users information these represent offer potential for understanding user interactions both to aid design and improve feedback means that they should be given greater consideration alongside other more subjective ways to research user experience