Towards an evidence-based approach to communicating uncertainty in intelligence analysis

Intelligence products have a degree of uncertainty associated with them. This is typically expressed using linguistic probabilities (e.g., ‘likely’), and some organizations have adopted standardized lexicons for communicating uncertainty. This paper empirically shows that intelligence analysts use a wide heterogeneity of language to communicate uncertainty. This does not include all of the phrases in standardized lexicons used by the intelligence community. In addition, analysts may use some phrases differently to that advocated. Miscommunication of uncertainty can have deleterious effects on decision-making, and so standardization of uncertainty communication should be evidence-based. This paper discusses ways in which such evidence can be generated

A survey of intelligence analysts’ perceptions of analytic tools

This article presents a survey of 278 intelligence analysts’ views of fully operational analytic technologies and their newly developed replacements. It was found that usability was an important concept in analysts’ reasons for and against using analytic tools. The perceived usability of a tool was not necessarily indicative of its perceived usefulness. Analysts’ decisions to recommend an analytic tool to others were best predicted by how usable analysts perceived the tool to be rather than how useful they considered the tool to be. These findings have implications for the development and implementation of new analytic technologies in the intelligence community

An empirical note on perceptions of partial apologies

This study examined perceptions of partial apologies. Eighty young people rated the extent to which five components of apology (i.e., accepting responsibility, acknowledging harm, expressing remorse, offering reparation, and promising forbearance) implied each other. Statistical analyses across different types of partial apologies showed no significant differences in the extent to which a particular uncommunicated component of apology was implied by one type of partial apology than another. Analyses within each type of partial apology revealed significant differences in the extent to a partial apology implied one type of uncommunicated component than another: Acknowledging harm or offering reparation implied promising forbearance to a lesser extent than accepting responsibility and expressing remorse. Expressing remorse or promising forbearance implied accepting responsibility to a greater extent than acknowledging harm and offering reparation. It is important to understand perceptions of partial apologies because they more prevalent than full apologies, and may be considered less effective

Effects of a victim’s response to an offender’s apology: when the victim becomes the bad guy

The ‘apology-acceptance’ script that may prevail during the victim–offender mediation process suggests that victims may feel obliged or pressured to accept an offender’s offer of an apology. Violations of this expectation in terms of rejection of an apology or no recognition of it may influence the outcomes of mediation in several ways. Two experiments examined the effects of a victim’s response to an offender’s offer of a full apology on offenders’ perceptions of the victim’s response, emotional reactions, perceptions of the victim, attitudes towards the dispute and attitudes towards mediation. Experiment 1 compared the effects of a rejection, acceptance and no recognition of an apology, and Experiment 2 further investigated the effects of an acceptance versus no recognition of an apology. It was found that offenders who had their apology rejected considered the victim’s response as least appropriate and were least satisfied by it. ‘Rejected’ offenders felt more anger towards the victim and had more negative impressions of the victim. Offenders who had their apology accepted felt more guilt and shame. They were, however, also more willing to reach an agreement and were more likely to perceive the conflict as being resolved. ‘Accepted’ offenders were also more likely to participate in mediation in the future and more willing to recommend mediation to others. The present research also demonstrated that no recognition of an apology has adverse effects similar to a rejection of an apology

Sentencing multiple- versus single-offence cases: does more crime mean less punishment?

The “totality principle” in law aims to show mercy to offenders in multiple-offence cases and retain ordinal proportionality in punishing those who commit different categories of offence. The effect of this principle in practice, however, is largely unknown. The present study involved an analysis of data released by the Sentencing Council for England and Wales to estimate the prevalence of multiple-offence cases and compare the penalties they received against comparable single-offence cases. Multiple-offence cases represented approximately half of the cases in the sample which included violent, property, drugs and driving offences. Offence-specific regression analyses revealed that multiple/single-offence case status was not a significant predictor of receiving a custodial sentence or of custody length. Thus, by applying the totality principle, sentencers may be letting multiple-offence offenders ‘off lightly.’ Potential explanations for this unintentional effect on decision-making lies in how the totality principle is defined and interpreted, and recommendations are made for revising the guideline on application of the totality principle

Kenneth R. Hammond’s contributions to the study of human judgment and decision making

Kenneth R. Hammond (1917–2015) made several major contributions to the science of human judgment and decision making. As a student of Egon Brunswik, he kept Brunswik’s legacy alive – advancing his theory of probabilistic functionalism and championing his method of representative design. Hammond pioneered the use of Brunswik’s lens model as a framework for studying how individuals use information from the task environment to make clinical judgments, which was the precursor to much ‘policy capturing’ and ‘judgment analysis’ research. Hammond introduced the lens model equation to the study of judgment processes, and used this to measure the utility of different forms of feedback in multiple-cue probability learning. He extended the scope of analysis to contexts in which individuals interact with one another – introducing the interpersonal learning and interpersonal conflict paradigms. Hammond developed social judgment theory which provided a comprehensive quantitative approach for describing and improving judgment processes. He proposed cognitive continuum theory which states that quasi-rationality is an important middle-ground between intuition and analysis and that cognitive performance is dictated by the match between task properties and mode of cognition. Throughout his career, Hammond moved easily from basic laboratory work to applied settings, where he resolved policy disputes, and in doing so, he pointed to the dichotomy between theories of correspondence and coherence. In this paper, we present Hammond’s legacy to a new generation of judgment and decision making scholars

A survey of intelligence analysts’ strategies for solving analytic tasks

Analytic performance may be assessed by the nature of the process applied to intelligence tasks and analysts are expected to use a 'critical' or deliberative mindset. However, there is little research on how analysts do their work. We report the findings of a quantitative survey of 113 intelligence analysts who were asked to report how often they would apply strategies involving more or less critical thinking when performing representative tasks along the analytic workflow. Analysts reported using ‘deliberative’ strategies significantly more often than ‘intuitive’ ones when capturing customer requirements, processing data, and communicating conclusions. Years of experience working in the intelligence community, skill level, analytic thinking training, and time spent working collaboratively (opposed to individually) were largely unrelated to reported strategy use. We discuss the implications of these findings for both improving intelligence analysis and developing an evidence-based approach to policy and practice in this domain

Male youth perceptions of violent extremism: towards a test of rational choice theory

Understanding how people perceive the pros and cons of risky behaviors such as terrorism or violent extremism represents a first step in developing research testing rational choice theory aiming to explain and predict peoples’ intentions to engage in, or support, these behaviors. Accordingly, the present study provides a qualitative, exploratory analysis of a sample of 57 male youths’ perceptions of the benefits and drawbacks of: (a) accessing a violent extremist website, (b) joining a violent extremist group, and (c) leaving such a group. Youth perceived significantly more drawbacks than benefits of joining a violent extremist group (p = .001, d = .46) and accessing a violent extremist website (p = .001, d = .46). The perceived benefits of engagement referred to gaining knowledge/awareness, being part of a group/similar people, and fighting the enemy/for a cause. The drawbacks referred to being exposed to negative material and emotions, having violent/criminal beliefs and behaviors, and getting in trouble with the law. The perceived benefits of disengagement referred to no longer committing illegal acts, and regaining independence/not being manipulated. The drawbacks referred to exposing oneself to harm and reprisal. These findings provide an insight into how male youth think about (dis)engagement in violent extremism, and can inform future quantitative research designed to explain and predict (dis)engagement in violent extremism. Eventually, such research may inform the development of evidence based prevention and intervention strategies

Words or numbers? Communicating probability in intelligence analysis

Intelligence analysis is fundamentally an exercise in expert judgment made under conditions of uncertainty. These judgments are used to inform consequential decisions. Following the major intelligence failure that led to the 2003 war in Iraq, intelligence organizations implemented policies for communicating probability in their assessments. Virtually all chose to convey probability using standardized linguistic lexicons in which an ordered set of select probability terms (e.g., highly likely) is associated with numeric ranges (e.g., 80-90%). We review the benefits and drawbacks of this approach, drawing on psychological research on probability communication and studies that have examined the effectiveness of standardized lexicons. We further discuss how numeric probabilities can overcome many of the shortcomings of linguistic probabilities. Numeric probabilities are not without drawbacks (e.g., they are more difficult to elicit and may be misunderstood by receivers with poor numeracy). However, these drawbacks can be ameliorated with training and practice, whereas the pitfalls of linguistic probabilities are endemic to the approach. We propose that, on balance, the benefits of using numeric probabilities outweigh their drawbacks. Given the enormous costs associated with intelligence failure, the intelligence community should reconsider its reliance on using linguistic probabilities to convey probability in intelligence assessments. Our discussion also has implications for probability communication in other domains such as climate science

Intelligence analysis support guide: development and validation

Research shows that intelligence analysts do not routinely follow a logical workflow, do not always use critical thinking, and that analysts’ training and experience are unrelated to analysts’ performance. The Analysis Support Guide (ASG) aims to capture, communicate, and encourage good analytic practice. The ASG is informed by organizational intelligence doctrine and past research on intelligence analysis. The ASG includes the generic analytic workflow, prompts for good practice at each stage of the workflow, indicators of good and poor analytic practice, and an analytic investigation questionnaire. The findings of a small-scale content validation study of the ASG are reported here. Fourteen analysts provided detailed feedback on its content. The results informed a revision of the ASG that is currently used to train new and experienced analysts. The ASG can also inform the development of analytic technologies and future research on the psychology of intelligence analysi