Information retrieval in systematic reviews: a case study of the crime prevention literature

OBJECTIVES: A defining feature of a systematic review is the data collection; the assembling of a meticulous, unbiased, and reproducible set of primary studies. This requires specialist skills to execute. The aim of this paper is to marshal tacit knowledge, gained through a systematic search of the crime prevention literature, to develop a ‘how-to guide’ for future evidence synthesists in allied fields. METHODS: Empirical results from a recent systematic search for evidence in crime prevention are supplied to illustrate key principles of information retrieval. RESULTS: Difficulties in operationalizing a systematic search are expounded and possible solutions discussed. Empirical results from optimizing the balance between sensitivity and precision with the criminological literature are presented. An estimation of database overlap for crime prevention studies is provided to guide other evidence synthesists in streamlining the search process. CONCLUSIONS: A high-quality search will involve a substantial time investment in honing the research question, specifying the precise scope of the work, and trialing and testing of search tactics. Electronic databases are a lucrative source of eligible studies, but they have important limitations. The diversity of expression across the criminological literature needs to be captured by the use of many search terms—both natural language and controlled vocabulary—in database searches. Complementary search tactics should be employed to locate eligible studies without common vocabulary. Grey literature should be ardently pursued, for it has a central role in the crime prevention evidence base

The canonical controller and its regularity

This paper deals with properties of canonical controllers. We first specify the behavior that they implement. It follows that a canonical controller implements the desired controlled behavior if and only if the desired behavior is implementable. We subsequently investigate the regularity of the controlled behavior. We prove that a canonical controller is regular if and only if every controller is regular. In other words, canonical controllers are maximally irregular

A victim-centred cost–benefit analysis of a stalking prevention programme

Research suggests that stalking inflicts great psychological and financial costs on victims. Yet costs of victimisation are notoriously difficult to estimate and include as intangible costs in cost–benefit analysis. This study reports an innovative cost–benefit analysis that used focus groups with multi-agency teams to collect detailed data on operational resources used to manage stalking cases. This method is illustrated through the presentation of one case study. Best- and worst-case counterfactual scenarios were generated using the risk assessment scores and practitioner expertise. The findings suggest that intervening in high-risk stalking cases was cost-beneficial to the state in all the case studies we analysed (even if it incurs some institutional costs borne by the criminal justice system or health) and was often cost-beneficial to the victims too. We believe that this method might be useful in other fields where a victim- or client-centred approach is fundamental

Evidencing the impact of Neighbourhood Watch

This report outlines the routes or processes through which Neighbourhood Watch activities might have an impact on crime reduction and other, associated, outcomes. These are broken down into chains of events (the ‘theories of change’ or ‘mechanisms’). Commonly used in evaluation projects, a theory of change is intended to simply but elegantly explain how and why something works . The first step is determining the intended outcomes of the activity, i.e. crime reduction or increased neighbourliness; the second is determining the logical sequence of specific actions and processes that are required to make that outcome likely to happen. The result is a process map that links activities and required conditions to produce intermediate changes and final outcomes. Articulating a theory of change before conducting any evaluation has the advantage of exposing measurement points along the process where data can be collected to evidence whether something is working as assumed. Therefore, measurement points along each theory of change we present are highlighted. Subsequently, the advantages and disadvantages of different data sets that can measure and evidence these points in the theory of change are summarised

A systematic review of tagging as a method to reduce theft in retail environments

Background: Retailers routinely use security tags to reduce theft. Presently, however, there has been no attempt to systematically review the literature on security tags. Guided by the acronym EMMIE, this paper set out to (1) examine the evidence that tags are effective at reducing theft, (2) identify the key mechanisms through which tags are expected to reduce theft and the conditions that moderate tag effectiveness, and (3) summarise information relevant to the implementation and economic costs of tagging. Methods: In this mixed-methods review, we performed systematic keyword searches of the published and unpublished literature, hand searched relevant journals, conducted forward and backward citation searches and consulted with four retailers. Studies were included if they reported an explicit goal of reducing the theft or shrinkage of items through the use of security tags in retail environments. Results: We identified 50 eligible studies, eight of which reported quantitative data on the effectiveness of tags in retail environments. Across these eight studies, five showed positive results associated with the introduction of tags, but heterogeneity in the type of tag and reported outcome measures precluded a meta-analysis. We identified three mechanisms through which tags might plausibly reduce theft-increase the risks, reduce the rewards, increase the effort-which were found to vary by tag type, and their activation dependent on five broad categories of moderator: retail store and staff, customers (including shoplifters), tag type, product type, and the involvement of the police and criminal justice system. Implementation challenges documented in the literature related mainly to staffing issues and tagging strategy. Finally, although estimates are available on the costs of tagging, our searches identified no highquality published economic evaluations of tagging. Conclusions: Through applying the EMMIE framework this review highlighted the complexity involved in security tagging in retail environments, whereby different kinds of tags are expected to reduce theft through different casual mechanisms which are dependent on a distinctive configuration of conditions. Based on the available evidence it is difficult to determine the effectiveness of tags as a theft reduction measure, albeit there is suggestive evidence that more visible tags are associated with greater reductions in theft than less visible tags

Inter-rater reliability in systematic review methodology: exploring variation in coder decision-making

A methodologically sound systematic review is characterized by transparency, replicability and a clear inclusion criteria. However, little attention has been paid to reporting the details of inter-rater reliability (IRR) when multiple coders are used to make decisions at various points in the screening and data extraction stages of a study. Prior research has mentioned the paucity of information on IRR, including number of coders involved, at what stages and how IRR tests were conducted, and how disagreements were resolved. This paper examines and reflects on the human factors that affect decision-making in systematic reviews via reporting on three IRR tests, conducted at three different points in the screening process, for two distinct reviews. Results of the two studies are discussed in the context of inter rater and intra rater reliability in terms of the accuracy, precision and reliability of coding behaviour of multiple coders. Findings indicated that coding behaviour changes both between and within individuals over time, emphasising the importance of conducting regular and systematic inter and intra-rater reliability tests, especially when multiple coders are involved, to ensure consistency and clarity at the screening and coding stages. Implications for good practice while screening/coding for systematic reviews are discussed

How Strong is the Evidence-Base for Crime Reduction End Users?

To support the development and implementation of evidence-based crime reduction, we systematically identified and appraised 70 systematic reviews of single crime reduction measures published between 1975 and 2015. Using the EMMIE framework, we find that the quality of reporting on the Effectiveness of crime reduction measures is reasonably strong, particularly in systematic reviews published by the Cochrane and Campbell Collaborations. In contrast, evidence concerning the Mechanisms underpinning a crime reduction intervention, the conditions that Moderate effectiveness, Implementation challenges and the Economic costs and benefits of crime reduction was largely absent from the assessed systematic reviews. We conclude that there is a distinct lack of systematic review evidence in crime reduction that currently speaks to the knowledge needs of practitioners (i.e., how to make an intervention “work” for them)