How useful is what we have? Limitations of Cochrane Reviews, the case of substance treatment in pregnancy

Abstract

Aims: To describe the public health promise and methodological problems of Cochrane reviews. Cochrane reviews are central to evidence-based medicine. However by focusing exclusively on randomized controlled trials (RCTs) a large swath of the published literature is excluded from their purview. Our experience conducting three of the four published reviews that examined interventions for substance use in pregnancy has brought particular methodological issues to light. Pregnancy is timelimited event and therefore amenable to a RCT intervention. However this study design is employed mostly in academic centers serving poor inner-city women. How useful then is this evidence? We begin with a brief overview of the Cochrane Library with particular emphasis on the addiction and obstetric literature. Then the following methodological topics are addressed: the tensions between clinically useful, clinically expedient, and sociologically meaningful outcomes; the difference between experimental and real world interventions; problems in external validity due to RCT recruitment in pregnancy; and confounding and bias in exposure assignment. We contrast our results from Cochrane reviews with the observational studies that were excluded from our reviews. Finally we argue for a more flexible design for systematic reviews that can allow the inclusion of more diverse study designs. The current Cochrane software (RevMan 5.0) appears to offer hope for this. Conclusions: Pregnancy is a unique moment in a woman’s life course where individual motivation for behavioral change is coupled with the availability of social support (such as prenatal care). Furthermore, interventions for drug problems in pregnancy have the possibility of not only improving birth outcomes, but affecting maternal health and child development across the life course. The summary of quality evidence is therefore necessary in the guidance of both clinical medicine and current policy. Support: Part of this study was funded by the Alcohol Education Research Council, UK. 61