Investigation of the Rasch model in its application to foreign language proficiency testing

The main theme of this study is the use of the Rasch model for dichotomously-scored items in the analysis of second/foreign language proficiency test data. Analytic procedures deriving from this model are applied to response data from English proficiency tests of two different types: (i) a cloze-type test, which embodies the notion of proficiency as being measurable by a single, global test, and (ii) the objectively-scored sections of the English Language. Testing Service (ELTS) test,, in which proficiency is. viewed as being divisible into sub-components, each measured by a separate subtest. The total numbers of testees involved are 854 and 1,503 respectively. The theoretical background relating to item response models is first explained, via "a discussion of traditional procedures for the analysis of test data. The relationship between the Rasch model and other response models of similar mathematical form is considered, and further details of its operation provided. The results of the Rasch analyses are compared with those from traditional analyses of the same data. The Rasch statistics are shown to be more informative, and therefore preferable, on several counts. Further investigations are carried out on both data sets, in order to assess the fit between model and data, to check for possible violations of specific model assumptions, and, to check for, expected model features. For, both the cloze-type data, and the ELTS, data (analysed in separate subtests), observed and expected item characteristic curves show reasonable conformity, though with some Instances of serious misfit In both cases. No evidence for departure from unidimensionality is found for the cloze data, but there is some indication that ELTS modules, when combined withGeneral components, may vary in their departure from unidimensionality

The construction and validation of an Arabic placement test for first year students at the University of Malaya

The issue which has always been discussed by scholars in the area of language teaching and testing is whether the test is valid, i.e. whether it tests what it is supposed to test and whether the test is reliable, i.e. consistent in assessing the candidates. This research attempts to construct and to validate an Arabic placement test for new students at the Academy of Islamic Studies at the University of Malaya in Malaysia. The design of the test was based on the syllabus at the Academy and the test specification, prepared during this research. Four sub-tests are constructed: Reading, Grammar, Writing and Dictation. To ensure the validity of the test, two analyses, internal and external, are conducted. The internal validity analysis is concerned with face and content validity. Three groups of students from different levels of academic background and countries participated in the pilot study for the purpose of internal validity analysis. Modifications were made to some items of the sub-tests at the end of the pilot study. The external validity analysis is concerned with concurrent and predictive validity. The correlation coefficients (r) between the total mark of the sub-tests and one of the two measures for concurrent validity indicate that the relationships are moderate: between .40 and .60. As for predictive validity, the r between the sub-tests and the total mark of the final examination are between .60 and .64: a moderate relationship too. In the analysis of the reliability of the tests using the internal consistency method, the reliability coefficients (r^) for the sub-tests are very high: ranging between .87 and .90. The correlation analysis between the total score of the sub-tests also indicates a very high relationship: five correlation coefficients (r) are between .70 and .75 and only one correlation has the r of .69. The conclusion of the study states that all four sub-tests prove to be successful in assessing the students' proficiency in Arabic and therefore could be used for the purpose of grouping students for the teaching and learning of Arabic at the Academy

Reasoning ability

Im Rahmen dieser Arbeit wird das Ziel der regelgeleiteten Testentwicklung zur Erfassung der fluiden Intelligenz mittels figuralen Analogieaufgaben verfolgt. Schlussfolgerndes Denken wurde als Testgegenstand gewählt, da es als zentrale Komponente kognitiver Fähigkeiten und analytischer Intelligenz gilt. In der Untersuchung wurden 484 Schüler der gymnasialen Oberstufe getestet und die resultierenden Daten der Testitems wurden mit Analyseverfahren der klassischen Testtheorie und Item Response Theorie ausgewertet. Der Einfluss der im Test angewandten Regeln und Elemente auf die Aufgabenschwierigkeit wurde mittels Linear Logistischer Testmodelle untersucht. Die prä-experimentell angenommene Aufgabenstruktur konnte so validiert werden, da die im Test angewandten Regeln zur Erklärung der Itemschwierigkeiten herangezogen werden konnten

The concurrent validity of learning potential and psychomotor ability measures for the selection of haul truck operators in an open-pit mine

The purpose of the present study was to determine the concurrent validity of learning potential and psychomotor ability measures for the prediction of haul truck operator (N=128) performance in an open-pit mine. Specific aims were to determine the nature of the relationship between learning potential and psychomotor ability; whether there are higher order cognitive or psychomotor factors present in the combined use of the TRAM 1 and Vienna Test System measures; and the relative contribution of learning potential and psychomotor ability in the prediction of haul truck operator performance. The validity of learning potential and psychomotor ability measures was partially supported. A positive correlation between general (cognitive) ability (g) and psychomotor ability was reported. Factor analysis provided relatively consistent evidence for a general (cognitive) ability factor (g) underlying performance on all measures. The relative contribution of learning potential and psychomotor ability in the prediction of performance could not be established.Industrial and Organisational PsychologyM. Com. (Industrial and Organisational Psychology

An Experimental Study in Diagnostic Testing and Concept Development in Secondary School Biology

Through a survey on the role of concepts in the field of science learning and teaching, the researcher pointed out that the teaching of biology should be made towards concept attainment and development

AN ITEM BANK DEVELOPMENT TO INCLUDE TEST ITEMS ASSESSING ORTHOGRAPHIC AND PHONOLOGICAL PROCESSING SKILLS AT THE FACULTY OF NURSING

In recent years, the importance of assessing students’ orthographic and phonological processing skills has been acknowledged, especially in L1 reading. Moreover, the development of an item bank for an in-house English placement test would enable faculty to assign students to appropriate English-language classes, which in turn would likely lead to students’ successful completion of English-language programs in the tertiary-level institution. Little has been reported, however, on the L2/FL reading contexts. This study thus investigated the process of developing an item bank with orthographic and phonological processing skills for the Faculty of Nursing. The study involved identification of the orthographic and phonological features of the faculty’s English curriculum and materials. It also explored the orthographic and phonological features of two commercially produced English proficiency tests, the TOEFL and the TOEIC, and determined whether these tests correspond to the Faculty of Nursing curriculum requirements. The study also used Rasch analysis to validate the development of test items to assess orthographic and phonological skills, and explored whether these test items correspond to the requirements of the faculty’s English curriculum. Analysis of the faculty’s curriculum and the commercially produced English proficiency tests revealed that the two tests may not be appropriate tools to measure students’ orthographic and phonological processing skills. The Rasch analysis— including separation, reliability, test targeting, and unidimensionality for a total of 147 items—yielded 90 equated test items. Moreover, the test items showed sufficient spreads: 9 (10%) were grouped at the beginner level, 74 (82%) at the intermediate level, and 7 (8%) at the advanced level

Examinations as instruments for educational change : investigating the washback effect of the Nepalese English exams

This study examines the washback effect of a final examination. Despite the general criticisms of a final examination for its negative influence on education, no empirical evidence was noted in the existing literature; rather some evidence for positive washback was found. The study was based on the assumption that the detrimental effect of a final exam is not inherent; whether the washback is negative or positive is dependent on the design of the exam. Furthermore, the power that the exam has to influence teaching and learning, if exploited properly, can make it work as an instrument for educational change. The context of the study was English language teaching and testing at school in Nepal. The washback effect of the School Leaving Certificate English exam, an exit exam based on prescribed textbooks in terms of its content and discrete-point approach in terms of its format, was examined. A new exam of reading, grammar, note-taking and writing based on the course objectives of the SLC English course in terms of its content, and integrative-communicative approach in terms of its format was designed to use as the criterion measure of the English proficiency of the students. The new exam was administered to school leavers and students of the previous year. The performance of the school leavers on the new exam was compared with their performance on the SLC English exam, and with the performance of the students of the previous year on the new exam. The results indicate that SLC English exam had a negative washback on the teaching and learning of the SLC English course because it failed to allow the students and the teacher to work for the course objectives of the SLC English. It was concluded that washback is an inherent quality of a final exam; people whose future is affected by the exam-results work for the exam regardless of the quality of the exam. Whether the washback is negative or positive is dependent on what the exam measures; if it is congruent with the sentiment and the purposes of the course objectives, it can achieve beneficial washback; if not it is bound to produce harmful washback. Innovations through the former type of exam would lead the teaching for the exam to be in accordance with it. The implications for language testing in general, and the Nepalese ELT situation in particular are presented. It is suggested that the SLC English exam should be replaced by an exam similar to the one used, in order to bring about change in the teaching of the SLC English course. Recommendations for further research are made

Psychological Assessment in South Africa

This book provides an overview of the research related to psychological assessment across South Africa. The thirty-six chapters provide a combination of psychometric theory and practical assessment applications in order to combine the currently disparate research that has been conducted locally in this field. Existing South African texts on psychological assessment are predominantly academic textbooks that explain psychometric theory and provide brief descriptions of a few testing instruments. Psychological Assessment in South Africa provides in-depth coverage of a range of areas within the broad field of psychological assessment, including research conducted with various psychological instruments. The chapters critically interrogate the current Eurocentric and Western cultural hegemonic practices that dominate the field of psychological assessment. The book therefore has the potential to function both as an academic text for graduate students, as well as a specialist resource for professionals, including psychologists, psychometrists, remedial teachers and human resource practitioners