Ideas often have a way of spreading. This is certainly true in the case of American states; innovative policies are often adopted by others in a process known as policy diffusion. But in trying to determine which policies spread, could we be giving too much of a focus on those that are successful? In new research which examines interstate compacts, Andrew Karch, Sean C. Nicholson-Crotty, Neal D. Woods, and Ann O’M. Bowman find that an emphasis on successful policies may be leading scholars to overestimate the importance of some factors, such as neighboring state activity, and underestimate the importance of others, like the number of previous adopters

Bowman, Ann

Karch, Andrew

Nicholson-Crotty, Sean

Woods, Neal

LSE Research Online

How looking only at policy diffusion “successes” between statesmay be misleadingblogs.lse.ac.uk/usappblog/2016/06/08/how-looking-only-at-policy-diffusion-successes-between-states-may-be-misleading/Ideas often have a way of spreading. This is certainly true in the case of American states; innovativepolicies are often adopted by others in a process known as policy diffusion. But in trying todetermine which policies spread, could we be giving too much of a focus on those that aresuccessful? In new research which examines interstate compacts, Andrew Karch, Sean C.Nicholson-Crotty, Neal D. Woods, and Ann O’M. Bowman find that an emphasis on successfulpolicies may be leading scholars to overestimate the importance of some factors, such asneighboring state activity, and underestimate the importance of others, like the number of previousadopters.How political phenomena spread from one jurisdiction to another, typically known as diffusion, is acentral concern in political science.  Nearly 800 articles on the subject have been published inpolitical science journals, over half in the last decade.  Interest in diffusion cuts across virtuallyevery subfield of the discipline.  Scholars of comparative politics, international relations, andAmerican politics have examined the diffusion of democracy, international norms about humanrights, pension reform, political institutions, and many other political phenomena.  Few topics havesimultaneously engaged so many scholars in so many disparate areas of the discipline.Existing studies, however, often examine only episodes during which an innovation was adopted bya large number of political jurisdictions.  For instance a recent review looks at studies thatinvestigated the diffusion of 23 distinct policies in the American states.  The policies thatresearchers chose to study had diffused fairly widely; the average number of adopters across thesestudies was just over 29.This could be problematic because “successful” (i.e. widely-diffused) policies may not berepresentative of all policies.  Critics argue that constructing a compelling account of why diffusionoccurs, or fails to occur, requires that scholars investigate both “successes” and episodes of limitedadoption or non-adoption.  Failing to do so may produce a cumulative knowledge base that is builton a systematically unrepresentative sample of policies, thus leading scholars to make incorrect orquestionable inferences about the general nature of diffusion dynamics.  This “pro-innovation bias”could take several forms.  Researchers could identify certain factors as influential even though theydo not have a significant effect on program adoption when the full range of policies is considered.  The reverse couldalso occur.  Finally, researchers could identify the appropriate factors as influential but inflate or deflate theirsubstantive impact.  The appearance of any of these patterns would expose the limitations of thinking aboutdiffusion from only the perspective of successful cases.The main purpose of our recent research is to assess whether these concerns are warranted. To our knowledgeours is the first study to quantitatively assess the potential effects of a pro-innovation bias in diffusion research. Weestimate this bias by turning to the American states to examine state participation in interstate compacts,constitutionally-sanctioned agreements between two or more states that are used to address joint problems in manypolicy areas.  Interstate compacts are not perfect analogues for the policies on which diffusion research tends tofocus, yet in both cases state officials have the opportunity to adopt a preexisting template.  Moreover, previousresearch suggests that compact membership and policy innovation are driven by similar forces.  A major advantageof our approach is that we can observe the entire universe of interstate compacts rather than examining a smallpurposive sample of innovations.1/4Figure 1 gives an indication of the range of adoption for various interstate compacts. Of the 43 national compacts inexistence, 20 have only been adopted by ten or fewer states.  Eight, however, have been adopted by 41 or more. The wide variation in state participation helps address the pro-innovation bias issue.  While all 50 states entered intothe original Interstate Compact for Juveniles and 47 states joined the Emergency Management Assistance Compact,the only states to join the Interstate Insurance Receivership Compact are Illinois, Michigan, and Nebraska.  Fivenational compacts technically have not gone into effect because they claim only a single member.  Thus nationalcompacts include examples of widespread adoption, limited adoption, and non-adoption, a rarity in diffusionresearch.  They are also substantively meaningful policy initiatives that address the most salient issues in statepolicymaking—criminal justice, education, social welfare, and business practices.Figure 1 – Rates of American state adoption of national interstate compactsOur empirical strategy relies on pooled event history analysis, which can be used to examine multiple policyinnovations and ensure that any results are not due to the idiosyncrasies of an individual policy or state. Specifically, we estimate a model that allows us to assess whether the effects of many well-known predictors ofadoption vary between a traditional sample of widely adopted compacts (the 19 national compacts joined by 20states or more) and the full universe of 43 national compacts.Our analysis shows that an emphasis on diffusion “successes” may lead scholars to overestimate the impact ofsome factors but understate the effect of others.This can be seen by looking at the differences in the marginal effects of these variables across the traditional and fullsamples, which are plotted in Figure 2 along with their 95% confidence intervals. Each point in the plot representsthe effect of a 0 to 1 shift for indicators which are one of two possibilities and a one-standard-deviation shift forcontinuous measures.  Figure 2 – Differences between models looking only at “successful” compacts and at all compacts (with 95percent confidence interval)2/4In some cases, our results highlight how restricting the sample to widely diffused policies produces anoverestimation of key effects.  The impact of neighboring state activity, the most common measure ofinterdependence, is over twice as large in the traditional sample as it is in the full model.  The effect of policycomplexity is also much larger in the traditional sample than in the full model.  Indeed the observed impact is morethan four times as large.In other cases looking solely at widely diffused policies appears to underestimate important effects.  A notabledifference in this regard concerns the impact of the number of previous adopters, a proxy for policy learning.  Theimpact of this variable on the probability of adoption is positive and significant in both samples, but its effect isalmost twice as large in the full sample relative to the traditional sample.  The general impact of professionalassociations and interest groups also appears to be underestimated in diffusion research.  In the traditional sample,the involvement of organizations like the Council of State Governments in compact development had no observableeffect on the likelihood of adoption.  In the full model, however, their involvement has a significant positive effect.Our analysis suggests that focusing exclusively on episodes of widespread adoption may cause scholars to drawunwarranted inferences about the factors that influence policy decisions. While these results must be interpretedcarefully given the substantive differences between interstate compacts and “conventional” policy innovations, theyimply that scholars must be cognizant of the ways in which case selection affects our understanding of the diffusionprocess. Our findings suggest that scholars must engage in more careful theorizing about the conditions underwhich external forces and policy attributes are likely to influence the adoption decision.  Rather than presuming thatdevelopments in neighboring states will have an impact, for example, they must articulate specific mechanisms—such as communications networks, program externalities, and demographic similarities—that promoteinterdependence.  Additional theoretical development along these lines is necessary in order to more fullyunderstand what factors differentiate diffusion successes from instances of limited adoption.This article is based on the paper, ”Policy Diffusion and the Pro-Innovation Bias” which appeared in PoliticalResearch Quarterly.Featured image credit: Aaron Brown (Flickr, CC-BY-NC-2.0)3/4Please read our comments policy before commenting.  Note:  This article gives the views of the author, and not the position of USApp– American Politics and Policy, nor ofthe London School of Economics.  Shortened URL for this post: http://bit.ly/1ZDyFAV ______________________ About the authorsAndrew Karch – University of MinnesotaAndrew Karch is Arleen C. Carlson Associate Professor of Political Science at the University ofMinnesota.  He is the author of Democratic Laboratories: Policy Diffusion among the AmericanStates and Early Start: Preschool Politics in the United States. Sean C. Nicholson-Crotty – Indiana UniversitySean C. Nicholson-Crotty is an Associate Professor in the School of Public and EnvironmentalAffairs at Indiana University.  His research interests include public management, federalism andintergovernmental relations, and comparative state policy.  He is the author of Governors, Grants,and Elections: Fiscal Federalism in the American States.Neal D. Woods – University of South CarolinaNeal D. Woods is an Associate Professor of Political Science at the University of South Carolina. His research focuses on public policy and public administration. Ann O’M. Bowman – Texas A&M UniversityAnn O’M. Bowman is Professor and Hazel Davis and Robert Kennedy Endowed Chair in theDepartment of Public Service and Administration at the Bush School at Texas A&M University.  Herresearch interests center on state and local government institutions and policy.CC BY-NC 3.0 2015 LSE USAPP4/4

How looking only at policy diffusion "successes" between states may be misleading

Abstract

Similar works

Full text

Available Versions

LSE Research Online