This session comprises four papers that consider how systematic review methods may be
developed in order to make the best use of complex evidence in education and health.
The methods and approaches reflected upon in these papers are not drawn from a single
research tradition, but share a common goal of broadening the methodological scope of
systematic reviews and better understanding the utilisation of knowledge produced in this
way. The first paper (Henry Potts) reports an ongoing review using a meta-narrative
approach to make sense of the diverse sources of knowledge regarding electronic patient
records. The review method has stressed the importance of understanding knowledge
from within the research tradition in which it was produced; it is argued that this has
important implications for the way that evidence is utilised in the policy making process.
The second paper (Geoff Wong) reflects upon the experience of using an explicit realist
approach in the synthesis of the evidence in Internet based learning. This realist synthesis
offers a method of making sense of the highly heterogeneous and context dependent
evidence which exists in this field thus enabling greater insights into what makes such
educational interventions ‘work’. The third paper (Rod Sheaff) reports a review of the
predominantly qualitative research literature on organisational structures and their
impacts upon policy outcomes in health systems. A scoping study found 14389 relevant
papers of which 1568 were selected for review. These studies were very variable in the
amount and quality of the qualitative data, hence 'evidence', which they reported. The
paper describes an attempt to adapt realist methods so as to synthesise such bodies of
research in ways which take account of this variation in the strength of qualitative
evidence. The fourth paper (Mark Pearson) draws upon the work of Donald Campbell
and colleagues in order to gain a fuller understanding of how systematic reviews are
utilised in the policy making process. It is argued that interpretive approaches to
understanding policy making (such as rhetorical analysis) need to be tempered with a
more nuanced understanding of research validity. The case is made that interpretive
approaches not only can, but should, be melded with research validity to increase
understanding of the policy making process
