Interoperability with CAA: does it work in practice?

Abstract

IMS has been promising question and test interoperability (QTI) for a number of years. Reported advantages of interoperability include the avoidance of “lock in” to one proprietary system, the ability to integrate systems from different vendors, and the facilitation of an exchange of questions and tests between institutions. The QTI specification, while not yet an international standard for the exchange of questions, tests and results, now appears to be stable enough for vendors to have developed systems which implement such an exchange in a fairly sophisticated way. The costs to software companies of implementing QTI “compliance” in their existing CAA systems, however, are high. Allowing users to move their data to other systems may not seem to make commercial sense either. As awareness of the advantages of interoperability increases within education, software companies are realising that adding QTI import and export facilities to their products can be a selling point. A handful of vendors have signed up to the concept of interoperability and have taken part in the IMS QTI Working Group. Others state that their virtual learning environments or CAA systems are “conformant” with IMS QTI but do these assertions stand up when the packages are tested together? The CETIS Assessment Special Interest Group has been monitoring developments in this area for over a year and has carried out an analysis of tools which exploit the QTI specifications. This paper describes to what extent the tools genuinely interoperate and examines the likely benefits for users and future prospects for CAA interoperability