Methods used for research regarding iteration in instructional design

Abstract

This paper focuses on the search for suitable research methods for research regarding iteration in instructional design. More specifically my research concerned the question how instructional designers can be supported during an iterative design process. Although instructional design and development models are very useful, they do not help instructional designers to customise and continuously adapt the design process itself. They also do not help them to deal with the large variety of pragmatic factors (ranging from financial constraints to conflicts in the design team). Now, at the end of my own Ph.D. research, I understand why I did not find research studies regarding iteration in instructional design in the literature: iteration is difficult to observe and measure. Studying iteration during the design process requires a longer time-span, a case that is realistic in size and complexity, and the occurrence of the kind of events that would trigger iteration in practice. Thus, controlled laboratory studies are not suitable, which is unfortunate because think-aloud protocols would give the most information about why and how designers iterate. Case studies, on the other hand, would be long and labour-intensive, and they give the researcher no control over what happens. Furthermore, the differences between cases are likely to be so large that it will be difficult to compare them in order to come to a more general theory regarding iteration. In my own evaluation studies the use of an existing instructional design method and prototype tool, and a realistic case offered an environment in which it was possible to define and study iteration. The empirical setting allowed some control over what happened and made it possible to record a considerable amount of information about the subjects' design process. The results of two empirical evaluation studies were subsequently compared to case studies concerning real design problems