Capturing 'what works' in complex process evaluation research: the use of calendar instruments

Abstract

Calendar or timeline techniques have developed in parallel and are used in life course research, and health behaviour and treatment studies. Both types of research seek to reconstruct histories or events in order to understand phenomena. Unsurprisingly a research strategy that seeks to represent events from memory is fraught with recall error thereby influencing consistency, completeness, and accuracy of data. Strategies can be employed to improve data quality so informants can more accurately access long term memory. One such strategy involves producing a graphical timeframe against which historical information can be represented. This is said to stimulate memory facilitating accuracy of recall and fidelity of data. There are minor variations in the application of calendar techniques, unsurprising given the different methodological heritage, nevertheless there are common characteristics. These include: graphical display of the dimension of time, use of one or more thematic axis (representing the data domains) and event or landmark cues that temporally bound the research. The Department of Health (England) in 2008 funded a series of public health initiatives in nine ‘Healthy Towns’. These initiatives were targeted on facilitating healthier lifestyles in local populations and importantly learning from projects about “what works”. One “Healthy Town” – Healthy Halifax – funded ten embedded project streams all designed to encourage adoption of health lifestyles by the population living in four wards with poorest health outcomes. The challenge presented to the local evaluation team was capturing which, if any, of the projects made a difference to health lifestyles of local populations. Calendar technique were incorporated in research design to accurately represent the life history of each project and capture the antecedents, attributes and consequences of project delivery that might illuminate ‘what works’. This presentation will offer a critical appraisal of the utility of calendar technique as a methodological approach for capturing process evaluation