How analysts think: a preliminary study of human needs and demands for AI-based conversational agents

For conversational agents to provide benefit to intelligence analysis they need to be able to recognise and respond to the analysts intentions. Furthermore, they must provide transparency to their algorithms and be able to adapt to new situations and lines of inquiry. We present a preliminary analysis as a first step towards developing conversational agents for intelligence analysis: that of understanding and modeling analyst intentions so they can be recognised by conversational agents. We describe in-depth interviews conducted with experienced intelligence analysts and implications for designing conversational agent intentions using Formal Concept Analysis

Common ground in collaborative intelligence analysis: an empirical study

This paper reports an empirical exploration of how different configurations of collaboration technology affect peoples’ ability to construct and maintain common ground while conducting collaborative intelligence analysis work. Prior studies of collaboration technology have typically focused on simpler conversational tasks, or ones that involve physical manipulation, rather than the complex sensemaking and inference involved in intelligence work. The study explores the effects of video communication and shared visual workspace (SVW) on the negotiation of common ground by distributed teams collaborating in real time on intelligence analysis tasks. The experimental study uses a 2x2 factorial, between-subjects design involving two independent variables: presence or absence of Video and SVW. Two-member teams were randomly assigned to one of the four experimental media conditions and worked to complete several intelligence analysis tasks involving multiple, complex intelligence artefacts. Teams with access to the shared visual workspace could view their teammates’ eWhiteboards. Our results demonstrate a significant effect for the shared visual workspace: the effort of conversational grounding is reduced in the cases where SVW is available. However, there were no main effects for video and no interaction between the two variables. Also, we found that the “conversational grounding effort” required tended to decrease over the course of the tas

Common ground in collaborative intelligence analysis: an empirical study

This paper reports an empirical exploration of how different configurations of collaboration technology affect peoples’ ability to construct and maintain common ground while conducting collaborative intelligence analysis work. Prior studies of collaboration technology have typically focused on simpler conversational tasks, or ones that involve physical manipulation, rather than the complex sensemaking and inference involved in intelligence work. The study explores the effects of video communication and shared visual workspace (SVW) on the negotiation of common ground by distributed teams collaborating in real time on intelligence analysis tasks. The experimental study uses a 2x2 factorial, between-subjects design involving two independent variables: presence or absence of Video and SVW. Two-member teams were randomly assigned to one of the four experimental media conditions and worked to complete several intelligence analysis tasks involving multiple, complex intelligence artefacts. Teams with access to the shared visual workspace could view their teammates’ eWhiteboards. Our results demonstrate a significant effect for the shared visual workspace: the effort of conversational grounding is reduced in the cases where SVW is available. However, there were no main effects for video and no interaction between the two variables. Also, we found that the “conversational grounding effort” required tended to decrease over the course of the tas

MOTHER-CHILD CONVERSATIONS ABOUT OTHER PEOPLE: THE ROLE OF MOTHERS’ PERSONAL INTELLIGENCE

The dissertation focuses on two studies that explore an intriguing context in which variations in personal intelligence are apparent: the way parents talk with their children about other people. Fifty 6-9 year-olds and their mothers participated in Study 1. Study 1 documented individual differences in mother-child conversations about others and their relationship with mothers’ personal intelligence and children’s conversational variables, and also examined children’s use of trait labels and social behavior ratings. Forty-two 4-5 year-olds and 43 7-8 year-olds participated in Study 2 with their mothers. Study 2 replicated many of Study 1 findings, including significant correlations between mothers’ conversational variables, children’s conversational variables, and an association between mothers’ personal intelligence level and personality talk variables. Furthermore, Study 2 extended findings to a younger cohort of participants. Procedures for coding and analysis of personality talk are delineated. Study contributions are described in relation to literature on mother-child reminiscence and personal intelligence

Conversational intelligence analysis

Social networks foster the development of social sensing to gather data about situations in the environment. Making sense of this information is, however, a challenge because the process is not linear and additional sensed information may be needed to better understand a situation. In this paper we explore how two complementary technologies, Moira and CISpaces, operate in unison to support collaboration among human-agent teams to iteratively gather and analyse information to improve situational awareness. The integrated system is developed for supporting intelligence analysis in a coalition environment. Moira is a conversational interface for information gathering, querying and evidence aggregation that supports cooperative data-driven analytics via Controlled Natural Language. CISpaces supports collaborative sensemaking among analysts via argumentation-based evidential reasoning to guide the identification of plausible hypotheses, including reasoning about provenance to explore credibility. In concert, these components enable teams of analysts to collaborate in constructing structured hypotheses with machine-based systems and external collaborators

Developing conversational agents for use in criminal investigations

The adoption of artificial intelligence (AI) systems in environments that involve high risk and high consequence decision making is severely hampered by critical design issues. These issues include system transparency and brittleness, where transparency relates to (i) the explainability of results and (ii) the ability of a user to inspect and verify system goals and constraints, and brittleness (iii) the ability of a system to adapt to new user demands. Transparency is a particular concern for criminal intelligence analysis, where there are significant ethical and trust issues that arise when algorithmic and system processes are not adequately understood by a user. This prevents adoption of potentially useful technologies in policing environments. In this paper, we present a novel approach to designing a conversational agent (CA) AI system for intelligence analysis that tackles these issues.We discuss the results and implications of three different studies; a Cognitive Task Analysis to understand analyst thinking when retrieving information in an investigation, Emergent Themes Analysis to understand the explanation needs of different system components, and an interactive experiment with a prototype conversational agent. Our prototype conversational agent, named Pan, demonstrates transparency provision and mitigates brittleness by evolving new CA intentions. We encode interactions with the CA with human factors principles for situation recognition and use interactive visual analytics to support analyst reasoning. Our approach enables complex AI systems, such as Pan, to be used in sensitive environments and our research has broader application than the use case discussed

Developing conversational agents for use in criminal investigations

The adoption of artificial intelligence (AI) systems in environments that involve high risk and high consequence decision-making is severely hampered by critical design issues. These issues include system transparency and brittleness, where transparency relates to (i) the explainability of results and (ii) the ability of a user to inspect and verify system goals and constraints; and brittleness, (iii) the ability of a system to adapt to new user demands. Transparency is a particular concern for criminal intelligence analysis, where there are significant ethical and trust issues that arise when algorithmic and system processes are not adequately understood by a user. This prevents adoption of potentially useful technologies in policing environments. In this article, we present a novel approach to designing a conversational agent (CA) AI system for intelligence analysis that tackles these issues. We discuss the results and implications of three different studies; a Cognitive Task Analysis to understand analyst thinking when retrieving information in an investigation, Emergent Themes Analysis to understand the explanation needs of different system components, and an interactive experiment with a prototype conversational agent. Our prototype conversational agent, named Pan, demonstrates transparency provision and mitigates brittleness by evolving new CA intentions. We encode interactions with the CA with human factors principles for situation recognition and use interactive visual analytics to support analyst reasoning. Our approach enables complex AI systems, such as Pan, to be used in sensitive environments, and our research has broader application than the use case discussed