Language dissimilarity is a liability in translated tests which can result in
construct non-equivalence as the construct under measure in one test cannot be
generalised to the construct of another test in a different language. This study examines
the validity of using bilingual test booklet to assess Form Two Malaysian students’
mathematical achievement. The equivalence of the two test booklets were inspected using
differential item functioning methodology. The sample consists of 4,768 Form Two
students from 34 schools in two states in Malaysia. The instruments used were bilingual
mathematics test (English and Malay languages) and English-only mathematics test. Both
tests contain the same 40 multiple-choice items where 20 are word-problem and another
20 are computation items that were obtained from 1999 and 2003 TIMSS released items.
Random group design with spiral administration was employed. Differential Item
Functioning (DIF) analyses performed were based on the Mantel Haenszel method,
Multidimensional model and IRT 1-PL model. Only one common DIF item was flagged
which made up 2.5% of the test items. Therefore, the two test booklets are equivalent.
Test validity was not compromised with the use of the bilingual test. This study has
important implication on Mathematics test item writers who need to remove language
redundancy when developing test items and constructing tests
