A survey of intelligence analysts’ strategies for solving analytic tasks

Analytic performance may be assessed by the nature of the process applied to intelligence tasks and analysts are expected to use a 'critical' or deliberative mindset. However, there is little research on how analysts do their work. We report the findings of a quantitative survey of 113 intelligence analysts who were asked to report how often they would apply strategies involving more or less critical thinking when performing representative tasks along the analytic workflow. Analysts reported using ‘deliberative’ strategies significantly more often than ‘intuitive’ ones when capturing customer requirements, processing data, and communicating conclusions. Years of experience working in the intelligence community, skill level, analytic thinking training, and time spent working collaboratively (opposed to individually) were largely unrelated to reported strategy use. We discuss the implications of these findings for both improving intelligence analysis and developing an evidence-based approach to policy and practice in this domain

Intelligence analysis support guide: development and validation

Research shows that intelligence analysts do not routinely follow a logical workflow, do not always use critical thinking, and that analysts’ training and experience are unrelated to analysts’ performance. The Analysis Support Guide (ASG) aims to capture, communicate, and encourage good analytic practice. The ASG is informed by organizational intelligence doctrine and past research on intelligence analysis. The ASG includes the generic analytic workflow, prompts for good practice at each stage of the workflow, indicators of good and poor analytic practice, and an analytic investigation questionnaire. The findings of a small-scale content validation study of the ASG are reported here. Fourteen analysts provided detailed feedback on its content. The results informed a revision of the ASG that is currently used to train new and experienced analysts. The ASG can also inform the development of analytic technologies and future research on the psychology of intelligence analysi

Ordinal structure of the generic analytic workflow: a survey of intelligence analysts

We present a model of the generic analytic workflow and measure the extent of ordinal structure that analysts apply to analytic work, as well as how this is affected by their training and experiences. The workflow comprises six stages that follow from one another: capture requirements, plan analytic response, obtain data, process data, interpret outputs, and communicate conclusions. A survey of 144 intelligence analysts revealed that only 16% structured their workflows in a logical ordinal way. The extent of ordinal structure applied to analytic work was unrelated to analytic thinking training, years of analytic experience, and proportion of time spent working collaboratively. These findings have implications for the training and assessment of analysts, as well as for the design of analytic tools

Factors predicting conviction in child stranger rape

Background. Public knowledge of child stranger rape is shaped largely by media portrayals of a small number of cases, often marked by sensational trials, which may result in juror misconceptions of this offense. It is important to understand the factors that may influence jury verdicts in order to maximize the chance of guilty defendants being convicted. Objective. The aim is to explore the factors that predict juries’ decisions to convict or acquit in child stranger rape cases. Participants and Setting. The study utilizes a police database of recorded child stranger rape cases from a UK urban force from 2001-2015. Seventy cases that were tried by jury were analyzed. We investigated the extent to which 19 child-, accused- and offense-related factors predict jury verdicts. Methods. A four stage analytic process was employed: (a) Kendall’s tau-b measured inter-correlations among the factors; (b) Chi-Square and Welch t-tests measured associations between factors and verdicts; (c) binary logistic regression measured the power of factors in predicting verdicts; and (d) Stein’s formula was used to cross-validate the model. Results. Verdicts were predicted by two offense-related factors. A weapon increased the odds of conviction by 412%. An outdoor location increased the odds by 360%. Conclusions. The findings have potential implications for prosecution case building and courtroom policy. Prosecutors could gather as much information as possible from victims about the factors found to be of importance to juries. Judges could challenge incorrect beliefs and stereotypes by instructing juries

Factors predicting jury convictions in stranger rape cases

Background: Despite there being no legal distinction between different types of rapes (e.g., those committed by strangers to the victim versus those committed by perpetrators known to the victim), stereotypical beliefs about rape have meant that these can be treated differently by the justice system. The aim is to explore the factors that predict juries’ decisions to convict or acquit in stranger rape cases. Methods: We measured the importance of a range of 20 perpetrator-, victim-, and offense-related factors in predicting outcomes for 394 stranger rape cases tried by a jury. A four-stage analytic process was employed: (a) Kendall’s tau-b measured intercorrelations among the factors (predictors); (b) Chi-square and Welch t-tests measured associations between factors and verdicts; (c) binary logistic regression measured the power of factors in predicting verdicts; and (d) Stein’s formula was used to cross-validate the model. Results: Jury verdicts were predicted by five offense-related factors and one victim-related factor. None of the perpetrator-related factors were significant predictors of convictions for stranger rape. Conclusion: The findings have potential implications for victims of stranger rape, as well as prosecution and courtroom policy. We show that if a perpetrator is identified and charged, the likelihood of securing a conviction by a jury is high for victims of stranger rape. We suggest that prosecutors could gather as much information as possible from victims about the factors found to be of importance to juries, and judges could instruct juries on assumptions about the characteristics of the offense in order to challenge incorrect beliefs and stereotypes. Ultimately, this could be used to encourage victims of stranger rape to report and testify in court

Critical review of analytic techniques

In this paper, we classify 75 analytic techniques in terms of their primary function. We then highlight where across the stages of the generic analytic workflow the techniques might be best applied. Importantly, most of the techniques have some shortcomings, and none guarantee an accurate or bias-free analytic conclusion. We discuss how the findings of the present paper can be used to develop criteria for evaluating analytic techniques as well as the performance of analysts. We also discuss which sets of techniques ought to be consolidated as well as reveal gaps that need to be filled by new techniques

A re-examination of the acquittal biasing effect of offence seriousness

The justice system should operate free of any bias, and jurors’ judgments of a defendant’s guilt should be based on evidential factors alone. However, research suggests that this does not always occur. We aimed to investigate the biasing effect of offence seriousness, a case-related, extra-legal factor, on juror decision-making. In an experiment, we examined the effect of this extra-legal factor on 118 members of the jury eligible publics’ interpretations of Beyond Reasonable Doubt (BRD), probability of commission, verdict, and confidence in verdict. We found that defendants charged with more serious offences were judged to be less likely to have committed the crime. However, offence seriousness did not have a significant effect on interpretations of BRD or verdict. The present findings suggest a need to instruct jurors on the application of legal (probative) factors alone

Instructions on reasonable doubt: defining the standard of proof and the juror's task

[see journal for abstract] summary provided here: In the present paper, we measure the effectiveness of two specific instructions on RD that have been used and challenged in the US. These are described in more detail below. We examine the effect of the two instructions on people’s quantitative interpretations of the standard, as well as the degree of inter- and intra-individual variability in interpretations. In addition, we attempt to explain the effectiveness of these instructions by examining the effect of their precise wording on interpretations of RD

Effects of judicial instructions and juror characteristics on interpretations of beyond reasonable doubt

Purpose and Methods: The standard of proof, beyond reasonable doubt (BRD), serves as a threshold for reaching verdicts in criminal cases. Past research has demonstrated that factors such as the wording of judicial instructions defining the standard can influence people’s interpretation of it. In addition, there is some concern that instructions may not be effective for the wider jury-eligible population. In an experimental study involving members of the general public, we examined the effect of two commonly used judicial instructions (i.e., sure and firmly convinced) against a situation when BRD was undefined, on people’s quantitative interpretations of BRD as well as on their self-reported understanding of the standard and confidence in applying it. We also explored the effect of juror characteristics (i.e., gender, age and education). Results: Compared to when the standard was undefined, the sure instruction helped to reduce inter-individual variability in interpretations of BRD and the firmly convinced instruction increased people’s understanding of the standard. However, neither instruction was effective in increasing confidence in applying the standard or in reducing observed individual differences. Conclusion: These findings underscore the importance of developing evidence-based judicial instructions that can benefit the broad jury-eligible population equally and in a variety of ways