Skip to main content
Article thumbnail
Location of Repository

The limitations of randomized controlled trials in predicting effectiveness

By Nancy Cartwright and Eileen Munro


What kinds of evidence reliably support predictions of effectiveness for health and social care interventions? There is increasing reliance, not only for health care policy and practice but also for more general social and economic policy deliberation, on evidence that comes from studies whose basic logic is that of JS Mill's method of difference. These include randomized controlled trials, case–control studies, cohort studies, and some uses of causal Bayes nets and counterfactual-licensing models like ones commonly developed in econometrics. The topic of this paper is the 'external validity' of causal conclusions from these kinds of studies. We shall argue two claims. Claim, negative: external validity is the wrong idea; claim, positive: 'capacities' are almost always the right idea, if there is a right idea to be had. If we are right about these claims, it makes big problems for policy decisions. Many advice guides for grading policy predictions give top grades to a proposed policy if it has two good Mill's-method-of difference studies that support it. But if capacities are to serve as the conduit for support from a method-of-difference study to an effectiveness prediction, much more evidence, and much different in kind, is required. We will illustrate the complexities involved with the case of multisystemic therapy, an internationally adopted intervention to try to diminish antisocial behaviour in young people

Topics: RA Public aspects of medicine
Publisher: Wiley-Blackwell
Year: 2010
DOI identifier: 10.1111/j.1365-2753.2010.01382.x
OAI identifier:
Provided by: LSE Research Online

Suggested articles


  1. (2007). A Metaphysics for Scientific Realism: Knowing the Unobservable. doi
  2. (1988). Causal Inference in Retrospective Studies. Evaluation Review, doi
  3. (2007). Causal Powers: What Are They? Why Do We Need Them? What Can and Cannot be Done with Them?
  4. (1975). Causal Powers. doi
  5. (1998). Dispositions. doi
  6. (2007). Error in Economics: The Methodology of Evidence-Based Economics. doi
  7. (2007). Hunting Causes and Using Them: Approaches in Philosophy and Economics. New York: doi
  8. (2007). Inclusion: The Politics of Difference in Medical Research. doi
  9. (2009). Index. Available at Last accessed: 2
  10. (2007). Multisystemic Therapy for social, emotional, and behavioural problems in youth aged 10-17. The Cochrane Collaboration, doi
  11. (2003). Multisystemic therapy: An overview:
  12. (2007). Nature's Metaphysics: Laws and Properties. doi
  13. (1989). Nature’s Capacities and Their Measurement. doi
  14. (1836). On the Definition of Political Economy and on the Method of Philosophical Investigation in that Science, reprinted in Collected Works of
  15. (2009). Organizational Biography. Available at
  16. (1999). the ABC Group. doi
  17. (2008). The Transportability of Multisystemic Therapy to Sweden: Short-Term Results from a Randomized Trial of Conduct-Disordered Youths. doi
  18. (2003). Transportability of Multisystemic Therapy: Evidence for Multi-Level Influences’.
  19. (2009). Treatment Model. Available at, last accessed: 2

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.