Littell, Julia H

Maynard, Brandy R

English

Scholarship;  Research;  and Creative Work at Bryn Mawr College

Bryn Mawr College
Scholarship, Research, and Creative Work at Bryn Mawr
College
Graduate School of Social Work and Social
Research Faculty Research and Scholarship
Graduate School of Social Work and Social
Research
2014
Review of Systematic Review Methods: The
Science of Research Synthesis
Julia H. Littell
Bryn Mawr College, jlittell@brynmawr.edu
Brandy R. Maynard
This work is licensed under a Creative Commons Attribution 3.0 License.
Let us know how access to this document benefits you.
Follow this and additional works at: http://repository.brynmawr.edu/gsswsr_pubs
Part of the Social Work Commons
This paper is posted at Scholarship, Research, and Creative Work at Bryn Mawr College. http://repository.brynmawr.edu/gsswsr_pubs/57
For more information, please contact repository@brynmawr.edu.
Citation
Littell, Julia H. and Maynard, Brandy R., "Review of Systematic Review Methods: The Science of Research Synthesis" (2014).
Graduate School of Social Work and Social Research Faculty Research and Scholarship. Paper 57.
http://repository.brynmawr.edu/gsswsr_pubs/57
 1 
Review of Mark Petticrew and Helen Roberts, Systematic Reviews in the Social Sciences 
(Blackwell, 2006) 
 
By Julia H. Littell, Bryn Mawr College 
 
Social science can aid in developing solutions to social problems by providing evidence on 
unmet needs and the processes, implementation issues, and impacts of social programmes. But as 
the volume and complexity of social research increase, decision makers are often confronted 
with conflicting evidence. Questions remain about qualities of social research that affect its 
credibility and its relevance for decision-making. It is widely acknowledged that a synthesis of 
results of multiple studies usually provides better information for policy and practice than results 
of a single study. Hence, a new cottage industry has emerged to review social research and distill 
this information for use in decision making.  
Systematic approaches to reviewing research came of age in the last twenty years, 
although their roots are much older. Now well accepted in the fields of health care, systematic 
reviews are beginning to influence policy and practice in the fields of social care. Yet there are 
important gaps between the emerging science of research synthesis and the common practice of 
reviewing social science research. For example, traditional narrative reviews proliferate, despite 
their well-known limitations; many published meta-analyses are not based on systematic 
attempts to identify, retrieve, and critically assess potentially-relevant studies; and many so-
called systematic reviews are only partially systematic. 
The seminal texts on systematic review methods are geared toward topics in health care 
or are generic (intended to apply to reviews on any topic). Although the logic, methods, and 
 2 
principles of systematic reviews transcend disciplines and fields of practice, it is important for 
social scientists and decision makers to see how these ideas and methods can be used to their 
best advantage in the fields of social care.  
In Systematic Reviews in the Social Sciences, Mark Petticrew and Helen Roberts provide 
an accessible, entertaining, and informative look at applications of the science of research 
synthesis in the fields of social care. The book fills an important niche. It serves as an excellent 
introduction to systematic reviews and a practical guide for those who want to conduct and use 
them in the social sciences. 
 Beginning with a cogent discussion of the need for systematic reviews, the authors spell 
out the limitations of traditional narrative reviews, which they properly term “haphazard 
reviews.” They provide a history of systematic reviews in the social sciences which, while 
admittedly unsystematic, is rich in detail not found elsewhere. Then they carefully explicate the 
steps in the review process, considering the issues of problem formulation, decisions about the 
nature and scope of a review, strategies for locating potentially-relevant studies, approaches to 
assessing qualities of evidence, methods for synthesizing results across studies, exploration of 
potential sources of variation and bias, and strategies for successful dissemination of systematic 
reviews. Even the most technical information is presented in straightforward terms, with lively 
examples, quotations, and illustrations. 
While the book offers an excellent starting point for readers just beginning to conduct 
systematic reviews, it also provides many practical tips and resources for seasoned reviewers. 
For example, Petticrew and Roberts provide a discussion of strategies for finding relevant 
literature that is at once conceptually sophisticated, accessible, and highly pragmatic. They 
describe the search process in terms of its sensitivity and specificity (i.e., getting “the right stuff” 
 3 
and avoiding “the wrong stuff”). They include useful examples of keyword strings and filters, 
extensive lists of potentially-relevant databases and registers, and many sources of unpublished 
and fugitive (gray) literature. They even provide tips on translating non-English language papers.  
Throughout the book, the authors take an ecumenical approach, acknowledging different 
models and diverse approaches to systematic reviews. Their balanced view is appealing and 
constructive. They resist narrow hierarchies of evidence, in favor of a broader conceptualization 
of study quality that emphasizes the fit between study methods and purpose. Several schemes for 
evaluating specific study designs are provided, yet the reader is cautioned about the limitations 
of study-quality checklists and scales. The threats-to-validity approach is mentioned, but more 
could be said about potential applications of this approach and its tenet that validity is a property 
of the inferences that can be drawn from a study, not a property of methods (Shadish, Cook, & 
Campbell, 2002). Reviewers may need additional examples of study quality assessment methods, 
and Petticrew and Roberts provide plenty of references that lead readers to these examples. 
The authors’ knowledge of relevant literature is broad and deep, and their ability to share 
this knowledge with readers is impressive. They communicate the potential utility and 
limitations of systematic reviews with considerable enthusiasm and care. Serious readers will 
find a wealth of information in the references provided at the end of each chapter. In short, the 
book is very well-conceived, well-written, and carefully documented.  
Systematic Reviews in the Social Sciences succeeds on many levels: as an introduction for 
the casual observer, a practical how-to guide for beginners, and a set of valuable resources for 
experienced reviewers. The book can be used to teach the logic, principles, and practices of 
research synthesis to the next generation of social scientists. It demystifies the systematic review 
 4 
process, which should help decision makers critically assess, understand, and use systematic 
reviews. 
The book is, of course, not the final word on this topic. The science of research synthesis 
is still developing. Readers should stay tuned to this literature as new approaches emerge and 
debates about review methodologies unfold. 
 
Reference 
 
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental 
designs for general causal inference. Boston: Houghton Mifflin. 
 


Review of Systematic Review Methods: The Science of Research Synthesis

https://repository.brynmawr.edu/cgi/viewcontent.cgi?article=1056&context=gsswsr_pubs

Review of Systematic Review Methods: The Science of Research Synthesis

Abstract

Similar works

Full text

Available Versions

Scholarship; Research; and Creative Work at Bryn Mawr College