Multi-modal usability evaluation.

Research into the usability of multi-modal systems has tended to be device-led, with a resulting lack of theory about multi-modal interaction and how it might differ from more conventional interaction. This is compounded by a confusion over the precise definition of modality between the various disciplines within the HCI community, how modalities can be effectively classified, and their usability properties. There is a consequent lack of appropriate methodologies and notations to model such interactions and assess the usability implications of these interfaces. The role of expertise and craft skill in using HCI techniques is also poorly understood. This thesis proposes a new definition of modality, and goes on to identify issues of importance to multi-modal usability, culminating in the development of a new methodology to support the identification of such usability issues. It additionally explores the role of expertise and craft skill in using usability modelling techniques to assess usability issues. By analysing the problems inherent in current definitions and approaches, as well as issues relevant to cognitive science, a clear understanding of both the requirements for a suitable definition of modality and the salient usability issues are obtained. A novel definition of modality, based on the three elements of sense, information form and temporal nature is proposed. Further, an associated taxonomy is produced, which categorises modalities within the sensory dimension as visual, acoustic and haptic. This taxonomy classifies modalities within the information form dimension as lexical, symbolic or concrete, and classifies the temporal form dimension modalities as discrete, continuous, or dynamic. This results in a twenty-seven cell taxonomy, with each cell representing one taxon, indicating one particular type of modality. This is a faceted classification system, with the modality named after the intersection of the categories, building the category names into a compound modality name. The issues surrounding modality are examined and refined into the concepts of modality types, properties and clashes. Modalities are identified as belonging to either the system or the user, and being expressive or receptive in type. Various properties are described based on issues of granularity and redundancy. The five different types of clashes are described. Problems relating to the modelling of multi-modal interaction are examined by means of a motivating case study based on a portion of an interface for a robotic arm. The effectiveness of five modelling techniques, STN, CW, CPM-GOMS, PUM and Z, in representing multi-modal issues are assessed. From this, and using the collated definition, taxonomy and theory, a new methodology, Evaluating Multi-modal Usability (EMU), is developed. This is applied to a previous case study of the robotic arm to assess its application and coverage. Both the definition and EMU are used by students in a case study to test the definition and methodology's effectiveness, and to examine the leverage such an approach may give. The results shows that modalities can be successfully identified within an interactive context, and that usability issues can be described. Empirical video data of the robotic arm in use is used to confirm the issues identified by the previous analyses, and to identify new issues. A rational re-analysis of the six approaches (STN, CW, CPM-GOMS, PUM, Z and EMU) is conducted in order to distinguish between issues identified through craft skill, based on general HCI expertise and familiarity with the problem, and issues identified due to the core of the method for each approach. This is to gain a realistic understanding of the validity of claims made by each method, and to identify how else issues might be identified, and the consequent implications. Craft skill is found to have a wider role than anticipated, and the importance of expertise in using such approaches emphasised. From the case study and the re-analyses the implications for EMU are examined, and suggestions made for future refinement. The main contributions of this thesis are the new definition, taxonomy and theory, which significantly contribute to the theoretical understanding of multi-modal usability, helping to resolve existing confusion in this area. The new methodology, EMU, is a useful technique for examining interfaces for multi-modal usability issues, although some refinement is required. The importance of craft skill in the identification of usability issues has been explicitly explored, with implications for future work on usability modelling and the training of practitioners in such techniques

Research Agenda into Human-Intelligence/Machine-Intelligence Governance

Since the birth of modern artificial intelligence (AI) at the 1956 Dartmouth Conference, the AI community has pursued modeling and coding of human intelligence into AI reasoning processes (HI Þ MI). The Dartmouth Conference\u27s fundamental assertion was that every aspect of human learning and intelligence could be so precisely described that it could be simulated in AI. With the exception of knowledge specific areas (such as IBM\u27s Big Blue and a few others), sixty years later the AI community is not close to coding global human intelligence into AI. In parallel, the knowledge management (KM) community has pursued understanding of organizational knowledge creation, transfer, and management (HI Þ HI) over the last 40 years. Knowledge management evolved into an organized discipline in the early 1990\u27s through formal university courses and creation of the first chief knowledge officer organizational positions. Correspondingly, over the last 25 years there has been growing research into the transfer of intelligence and cooperation among computing systems and automated machines (MI Þ MI). In stark contrast to the AI community effort, there has been little research into transferring AI knowledge and machine intelligence into human intelligence (MI Þ HI) with a goal of improving human decision making. Most important, there has been no research into human-intelligence/machine-intelligence decision governance; that is, the policies and processes governing human-machine decision making toward systemic mission accomplishment. To address this gap, this paper reports on a research initiative and framework toward developing an HI-MI decision governance body of knowledge and discipline

Ubiquitous Technologies for Emotion Recognition

Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now, with the advent of wearable, mobile, and ubiquitous technologies, that we can aim to sense and recognize emotions, continuously and in real time. This book brings together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and the recognition of human emotions

An approach toward function allocation between humans and machines in space station activities

Basic guidelines and data to assist in the allocation of functions between humans and automated systems in a manned permanent space station are provided. Human capabilities and limitations are described. Criteria and guidelines for various levels of automation and human participation are described. A collection of human factors data is included

Cognitive Networking With Regards to NASA's Space Communication and Navigation Program

This report describes cognitive networking (CN) and its application to NASA's Space Communication and Networking (SCaN) Program. This report clarifies the terminology and framework of CN and provides some examples of cognitive systems. It then provides a methodology for developing and deploying CN techniques and technologies. Finally, the report attempts to answer specific questions regarding how CN could benefit SCaN. It also describes SCaN's current and target networks and proposes places where cognition could be deployed