A Practical Procedure for the Construction and Reliability Analysis of Fixed Length Tests with Random Drawn Test Items

A procedure to construct valid and fair fixed-length tests with randomly drawn items from an item bank is described. The procedure provides guidelines for the set-up of a typical achievement test with regard to the number of items in the bank and the number of items for each position in a test. Further, a procedure is proposed to calculate the relative difficulty for individual tests and to correct the obtained score for each student based on the mean difficulty for all students and the particular test of a student. Also, two procedures are proposed for the problem to calculate the reliability of tests with randomly drawn items. The procedures use specific interpretations of regularly used methods to calculate Cronbach’s alpha and KR20 and the Spearman-Brown prediction formula. A simulation with R is presented to illustrate the accuracy of the calculation procedures and the effects on pass-fail decisions

Efficiënt en hoger van kwaliteit.

Sinds april 2013 beschikt de Vrije Universiteit Amsterdam over een speciale digitale toetszaal. De zaal stelt de VU in staat om digitale toetsen voor cohorten tot 385 studenten in 1 keer af te nemen. Inmiddels heeft de VU al meer dan 22.000 digitale toetsen afgenomen in de zaal. De verwachting is dat dat er zo’n 50.000 of meer op jaarbasis kunnen worden. Silvester Draaijer, adviseur ICT en Onderwijs, schetst het proces en de gebruikerservaringen uit het afgelopen jaar en kijkt vooruit naar de komende jaren

Using online quizzing and quiz results to improve study success.

The need to increase student success rates, especially in the first years of higher education, is becoming stronger and stronger. Online quizzing in the first years of undergraduate education is regarded as one possible way to improve time-on-task, student engagement and retrieval ability. Several studies are published that address the incorporation of online quizzing in higher education. These studies show mixed results with regard to study success. For example, in general, student opinions towards the use of online quizzes is positive, however the effect on time-on-task is just modest and the correlations with success rate vary from no correlation to significant correlations with modest effects. The studies incorporate various course set-ups and qualitative and quantitative analyses methods and take a different number of variables into account. These could be the cause of the different findings. In this paper, an overview of several studies is presented and discussed. Online quizzing will be related to the ideas of deliberate practice and possibilities of data-mining and feedback (learning analytics) to improve the possible generation of data on which students and educators can act on and to increase the impact of online quizzing