Assessing the Validity and Reliability of Dichotomous Test Results Using Item Response Theory on a Group of First Year Engineering Students

Abstract

Traditional measurement instruments employed to assess the performance of student’s studying on STEM (Science, Technology, Engineering and Mathematics) related programmes typically involve classification based on final scores. The validity and reliability of these instruments and test forms are important considerations when assessing whether a student understands content and if not, where and in what way they are struggling. The aim of this study is to examine, validate and analyse the test results of first-year engineering student’s at an Institute of Higher Learning in Ireland who took the Purdue Spatial Visualisation Test of Rotation (PSVT:R). Results obtained were analysed using the RASCH measurement model to see if it could be used to provide an alternative means of measuring student learning and to help identify those who may require extra assistance to overcome academic deficiencies, particularly where spatial skills have been linked to success. Findings may be used to inform on future improvements to teaching approaches and styles