Computerised ACER advanced test BL : analysis of equivalency, test anxiety, and the effects of input device using New Zealand university participants : a thesis presented in partial fulfilment of the requirements for the degree of Master of Arts in Psychology at Massey University

Abstract

Study I examined the effects of the computerised format of the ACER Advanced Test BL (ACER-BL) on the test scores and anxiety of undergraduate participants, compared with the traditional paper-and-pencil format. Forty-one students were assigned to either a computer or paper-and-pencil treatment group using a stratified random design. Participants sequentially completed a general background questionnaire, the ACER-BL, an anxiety questionnaire, the ACER-BL, and a final anxiety questionnaire, with a 10 minute test-retest period between the ACER-BL administrations. There were no significant differences in ACER-BL score, and subsection scores, between the 2 treatment groups on either administration. The internal consistency reliability of each formats was moderate to high, and there was a high test-retest reliability for each format. While the mean scores for each treatment group were higher for the second test administration compared with the first, this result only reached significance for the computerised group. Gender, Undergraduate Year, and Typing Ability significantly influenced test score, although these failed to remain significant when treatment group was included in each analysis. These results suggest that the computerised version of the ACER-BL is equivalent to the paper-and-pencil version. Generally, there was no significant difference in reported test anxiety measures between the treatment groups, with mean reported anxiety indicating "slight anxiety." These anxiety results suggest little influence of test format on test anxiety. Study 2 examined the influence of input device (keyboard, numeric pad, and mouse) on ACER-BL scores and test anxiety of undergraduate participants. Using stratified random assignment, 90 subjects were tested on all three input devices using a one factor repeated measures design. Each participant sequentially completed a general background questionnaire, the ACER-BL, an anxiety questionnaire, the ACER-BL, an anxiety questionnaire, the ACER-BL, and a final anxiety questionnaire, with a 10 minute delay between each ACER-BL administration. There was no significant main effect of input device on test score, and there was no significant order effect for input device. Between- subjects analyses indicated a significant increase in mean test score across administrations for the keyboard and numeric pad, but no significant change in mean scores with the mouse. These results were also reflected in the analyses of mean input response time. While there was no significant effect of any measured participant characteristic on input device scores, mathematical ability and undergraduate year each had a significant influence on mean scores in the first ACER-BL administration. Participants with higher mathematical ability or more years at university had significantly higher mean test scores than participants with less mathematical ability or first year undergraduates respectively. While mean reported anxiety on all test anxiety measures decreased over the ACER-BL administrations, all mean reported anxiety indicated "slight anxiety." These anxiety results suggest little influence of input device on test anxiety. The lack of test-retest comparisons between the computerised and paper-and-pencil formats of a test was discussed along with the need for future computerised testing research to use participants from the general population