Coalitions of things: supporting ISR tasks via Internet of Things approaches

In the wake of rapid maturing of Internet of Things (IoT) approaches and technologies in the commercial sector, the IoT is increasingly seen as a key ‘disruptive’ technology in military environments. Future operational environments are expected to be characterized by a lower proportion of human participants and a higher proportion of autonomous and semi-autonomous devices. This view is reflected in both US ‘third offset’ and UK ‘information age’ thinking and is likely to have a profound effect on how multinational coalition operations are conducted in the future. Much of the initial consideration of IoT adoption in the military domain has rightly focused on security concerns, reflecting similar cautions in the early era of electronic commerce. As IoT approaches mature, this initial technical focus is likely to shift to considerations of interactivity and policy. In this paper, rather than considering the broader range of IoT applications in the military context, we focus on roles for IoT concepts and devices in future intelligence, surveillance and reconnaissance (ISR) tasks, drawing on experience in sensor-mission resourcing and human-computer collaboration (HCC) for ISR. We highlight the importance of low training overheads in the adoption of IoT approaches, and the need to balance proactivity and interactivity (push vs pull modes). As with sensing systems over the last decade, we emphasize that, to be valuable in ISR tasks, IoT devices will need a degree of mission-awareness in addition to an ability to self-manage their limited resources (power, memory, bandwidth, computation, etc). In coalition operations, the management and potential sharing of IoT devices and systems among partners (e.g., in cross-coalition tactical-edge ISR teams) becomes a key issue due heterogeneous factors such as language, policy, procedure and doctrine. Finally, we briefly outline a platform that we have developed in order to experiment with human-IoT teaming on ISR tasks, in both physical and virtual settings

Conversational Sensing

Recent developments in sensing technologies, mobile devices and context-aware user interfaces have made it possible to represent information fusion and situational awareness as a conversational process among actors - human and machine agents - at or near the tactical edges of a network. Motivated by use cases in the domain of security, policing and emergency response, this paper presents an approach to information collection, fusion and sense-making based on the use of natural language (NL) and controlled natural language (CNL) to support richer forms of human-machine interaction. The approach uses a conversational protocol to facilitate a flow of collaborative messages from NL to CNL and back again in support of interactions such as: turning eyewitness reports from human observers into actionable information (from both trained and untrained sources); fusing information from humans and physical sensors (with associated quality metadata); and assisting human analysts to make the best use of available sensing assets in an area of interest (governed by management and security policies). CNL is used as a common formal knowledge representation for both machine and human agents to support reasoning, semantic information fusion and generation of rationale for inferences, in ways that remain transparent to human users. Examples are provided of various alternative styles for user feedback, including NL, CNL and graphical feedback. A pilot experiment with human subjects shows that a prototype conversational agent is able to gather usable CNL information from untrained human subjects

Human Factors in Intelligence, Surveillance, and Reconnaissance: Gaps for Soldiers and Technology Recommendations

Abstract—We investigate the gaps for Soldiers in information collection and resource management for Intelligence, Surveillance, and Reconnaissance (ISR). ISR comprises the intelligence functions supporting military operations; we concentrate on ISR for physical sensors (air and ground platforms). To identify gaps, we use approaches from Human Factors (interactions between humans and technical systems to optimize human and system performance) at the level of Soldier functions/activities in ISR. Key gaps (e.g., the loud auditory signatures of some air assets, unofficial ISR requests, and unintended battlefield effects) are identified. These gaps illustrate that ISR is not purely a technical problem. Instead, interactions between technical systems, humans, and the environment result in unpredictability and adaptability in using technical systems. To mitigate these gaps, we provide technology recommendations. Keywords—intelligence, surveillance, and reconnaissance; ISR; human factors; human-systems integration; cognitive systems engineering; intelligenc

Provisioning robust and interpretable AI/ML-based service bundles

Coalition operations environments are characterised by the need to share intelligence, surveillance and reconnaissance services. Increasingly, such services are based on artificial intelligence (AI) and machine learning (ML) technologies. Two key issues in the exploitation of AI/ML services are robustness and interpretability. Employing a diverse portfolio of services can make a system robust to ‘unknown unknowns’. Interpretability — the need for services to offer explanation facilities to engender user trust — can be addressed by a variety of methods to generate either transparent or post hoc explanations according to users’ requirements. This paper shows how a service-provisioning framework for coalition operations can be extended to address specific requirements for robustness and interpretability, allowing automatic selection of service bundles for intelligence, surveillance and reconnaissance tasks. The approach is demonstrated in a case study on traffic monitoring featuring a diverse set of AI/ML services based on deep neural networks and heuristic reasoning approaches

Conversational homes

As devices proliferate, the ability for us to interact with them in an intuitive and meaningful way becomes increasingly challenging. In this paper we take the typical home as an experimental environment to investigate the challenges and potential solutions arising from ever-increasing device proliferation and complexity. We show a potential solution based on conversational interactions between “things” in the environment where those things can be either machine devices or human users. Our key innovation is the use of a Controlled Natural Language (CNL) technology as the underpinning information representation language for both machine and human agents, enabling humans and machines to trivially “read” the information being exchanged. The core CNL is augmented with a conversational protocol enabling different speech acts to be exchanged within the system. This conversational layer enables key contextual information to be conveyed, as well as providing a mechanism for translation from the core CNL to other forms, such as device specific API requests, or more easily consumable human representations. Our goal is to show that a single, uniform language can support machine- machine, machine-human, human-machine and human-human interaction in a dynamic environment that is able to rapidly evolve to accommodate new devices and capabilities as they are encountered

Unlocking Agility: Building Learning Capabilities Within A Consumer Healthcare Organization

Pharmaceutical companies are increasingly infusing the concept of agility to strive for continuous improvement. Significant exploration and research have focused on more technical-driven departments, like Information Technology and Research and Development. However, there has been little research with the focus on more process-driven functions, like Learning and Organizational Development. This action research study presents a case study of the implementation of a new training solution within a consumer healthcare organization from the lens of the project leader. Building upon the case study, this capstone includes a review of existing research and literature of agility with a focus on the healthcare sector, change management, adult learning, and organizational learning. The overall goal of this study is to explore the value of agility in building learning capacities within the pharmaceutical industry. Looking forward, the aim is to provide insights on how agility can be developed to facilitate an organization’s transformation to become a learning organization

Conversational homes: a uniform natural language approach for collaboration among humans and devices

As devices proliferate, the ability for us to interact with them in an intuitive and meaningful way becomes increasingly challenging. In this paper we take the typical home as an experimental environment to investigate the challenges and potential solutions arising from ever-increasing device proliferation and complexity. We describe and evaluate a potential solution based on conversational interactions between “things” in the environment where those things can be either machine devices or human users. Our key innovation is the use of a Controlled Natural Language (CNL) technology as the underpinning information representation language for both machine and human agents, enabling humans and machines to trivially “read” the information being exchanged. The core CNL is augmented with a conversational protocol enabling different speech acts to be exchanged within the system. This conversational layer enables key contextual information to be conveyed, as well as providing a mechanism for translation from the core CNL to other forms, such as device specific API (Application Programming Interface) requests, or more easily consumable human representations. Our goal is to show that a single, uniform language can support machine-machine, machine-human, human-machine and humanhuman interaction in a dynamic environment that is able to rapidly evolve to accommodate new devices and capabilities as they are encountered. We also report results from our first formal evaluation of a Conversational Homes prototype and demonstrate users with no previous experience of this environment are able to rapidly and effectively interact with simulated devices in a number of simple scenarios

Agilely assigning sensing assets to mission tasks in a coalition context

Due to copyright restrictions, the access to the full text of this article is only available via subscription.When managing intelligence, surveillance, and reconnaissance (ISR) operations in a coalition context, assigning available sensing assets to mission tasks can be challenging. The authors' approach to ISR asset assignment uses ontologies, allocation algorithms, and a service-oriented architecture.US Army Research Laboratory ; UK Ministry of Defenc