Problem spotting in human-machine interaction

In human-human communication, dialogue participants are con-tinuously sending and receiving signals on the status of the inform-ation being exchanged. We claim that if spoken dialogue systems were able to detect such cues and change their strategy accordingly, the interaction between user and systemwould improve. Therefore, the goals of the present study are as follows: (i) to find out which positive and negative cues people actually use in human-machine interaction in response to explicit and implicit verification questions and (ii) to see which (combinations of) cues have the best predictive potential for spotting the presence or absence of problems. It was found that subjects systematically use negative/marked cues (more words, marked word order, more repetitions and corrections, less new information etc.) when there are communication problems. Using precision and recall matrices it was found that various combinations of cues are accurate problem spotters. This kind of information may turn out to be highly relevant for spoken dia-logue systems, e.g., by providing quantitative criteria for changing the dialogue strategy or speech recognition engine

Human Machine Interaction

In this book, the reader will find a set of papers divided into two sections. The first section presents different proposals focused on the human-machine interaction development process. The second section is devoted to different aspects of interaction, with a special emphasis on the physical interaction

Improving Human-Machine Interaction

This thesis studies human and machine interaction. For better interaction between humans and machines, this thesis aims to address three issues that remain unanswered in literature. Three objectives are proposed in this thesis to address the three issues, and the objectives are: (i) identification of the core capabilities of a Human Assistance System (HAS) and study of implementation strategy of the core capabilities; (ii) development of a framework for improving the accuracy of human mind state inference; (iii) study of the effect of representation of the machine’s state (which is represented in a “natural” way) on the user’s actions. By a natural way, it is meant a way that contains emotions known to be always present in humans (or human emotions in short). The study includes theoretical development, experimentation, and prototype implementation. This thesis has concluded: (1) the core capabilities to be addressed in designing a HAS are transparency, communication, rationale, cognition and task-sharing and they can be implemented with the existing technologies including fuzzy logics, Petri Net and ACT-R (Adaptive Control of Thought-Rational); (2) expert opinion elicitation technique is a promising method to construct a more general framework for integrating various algorithms on human state inference; (3) there is a significant effect of the representation of the machine’s state on the user’s actions. The main contributions of this thesis are: (1) provision of a case study for the proof-of-concept of HAS in the area of Computer Aided Design (CAD); (2) provision of an integrated framework for fatigue inference for improved accuracy, being readily generalized to inference of other mind states; (3) generation of a new knowledge regarding the effect of the natural representation of a machine’s states on the user’s actions. These contributions are significant in human-machine science and technology. The first contribution may lead to the development of a new generation CAD system in the near future. The second contribution provides a much powerful technology for human mind inference, which is a key capability in HAS, and the third contribution enriches the science of human-machine interaction and will give impact to the field of Artificial Intelligence (AI) as well. The application of the result of this thesis is rehabilitation, machine learning, etc

Visualization and Human-Machine Interaction

The digital age offers a lot of challenges in the eld of visualization. Visual imagery has been effectively used to communicate messages through the ages, to express both abstract and concrete ideas. Today, visualization has ever-expanding applications in science, engineering, education, medicine, entertainment and many other areas. Different areas of research contribute to the innovation in the eld of interactive visualization, such as data science, visual technology, Internet of things and many more. Among them, two areas of renowned importance are Augmented Reality and Visual Analytics. This thesis presents my research in the fields of visualization and human-machine interaction. The purpose of the proposed work is to investigate existing solutions in the area of Augmented Reality (AR) for maintenance. A smaller section of this thesis presents a minor research project on an equally important theme, Visual Analytics. Overall, the main goal is to identify the most important existing problems and then design and develop innovative solutions to address them. The maintenance application domain has been chosen since it is historically one of the first fields of application for Augmented Reality and it offers all the most common and important challenges that AR can arise, as described in chapter 2. Since one of the main problem in AR application deployment is reconfigurability of the application, a framework has been designed and developed that allows the user to create, deploy and update in real-time AR applications. Furthermore, the research focused on the problems related to hand-free interaction, thus investigating the area of speech-recognition interfaces and designing innovative solutions to address the problems of intuitiveness and robustness of the interface. On the other hand, the area of Visual Analytics has been investigated: among the different areas of research, multidimensional data visualization, similarly to AR, poses specific problems related to the interaction between the user and the machine. An analysis of the existing solutions has been carried out in order to identify their limitations and to point out possible improvements. Since this analysis delineates the scatterplot as a renowned visualization tool worthy of further research, different techniques for adapting its usage to multidimensional data are analyzed. A multidimensional scatterplot has been designed and developed in order to perform a comparison with another multidimensional visualization tool, the ScatterDice. The first chapters of my thesis describe my investigations in the area of Augmented Reality for maintenance. Chapter 1 provides definitions for the most important terms and an introduction to AR. The second chapter focuses on maintenance, depicting the motivations that led to choose this application domain. Moreover, the analysis concerning open problems and related works is described along with the methodology adopted to design and develop the proposed solutions. The third chapter illustrates how the adopted methodology has been applied in order to assess the problems described in the previous one. Chapter 4 describes the methodology adopted to carry out the tests and outlines the experimental results, whereas the fifth chapter illustrates the conclusions and points out possible future developments. Chapter 6 describes the analysis and research work performed in the eld of Visual Analytics, more specifically on multidimensional data visualizations. Overall, this thesis illustrates how the proposed solutions address common problems of visualization and human-machine interaction, such as interface de- sign, robustness of the interface and acceptance of new technology, whereas other problems are related to the specific research domain, such as pose tracking and reconfigurability of the procedure for the AR domain

Contact force regulation in physical human-machine interaction based on model predictive control

With increasing attention to physical human-machine interaction (pHMI), new control methods involving contact force regulation in collaborative and coexistence scenarios have spread in recent years. Thanks to its internal robustness, high dynamic performance, and capabilities to avoid constraint violations, a Model Predictive Control (MPC) action can pose a viable solution to manage the uncertainties involved in those applications. This paper uses an MPC-driven control method that aims to apply a well-defined and tunable force impulse on a human subject. After describing a general control design suitable to achieve this goal, a practical implementation of such a logic, based on an MPC controller, is shown. In particular, the physical interaction considered is the one occurring between the body of a patient and an external perturbation device in a dynamic posturography trial. The device prototype is presented in both its hardware architecture and software design. The MPC-based main control parameters are thus tuned inside hardware-in-the-loop and human-in-the-loop environments to get optimal behaviors. Finally, the device performance is analyzed to assess the MPC algorithm’s accuracy, repeatability, flexibility, and robustness concerning the several uncertainties due to the specific pHMI environment considered

PRESENCE: A human-inspired architecture for speech-based human-machine interaction

Recent years have seen steady improvements in the quality and performance of speech-based human-machine interaction driven by a significant convergence in the methods and techniques employed. However, the quantity of training data required to improve state-of-the-art systems seems to be growing exponentially and performance appears to be asymptotic to a level that may be inadequate for many real-world applications. This suggests that there may be a fundamental flaw in the underlying architecture of contemporary systems, as well as a failure to capitalize on the combinatorial properties of human spoken language. This paper addresses these issues and presents a novel architecture for speech-based human-machine interaction inspired by recent findings in the neurobiology of living systems. Called PRESENCE-"PREdictive SENsorimotor Control and Emulation" - this new architecture blurs the distinction between the core components of a traditional spoken language dialogue system and instead focuses on a recursive hierarchical feedback control structure. Cooperative and communicative behavior emerges as a by-product of an architecture that is founded on a model of interaction in which the system has in mind the needs and intentions of a user and a user has in mind the needs and intentions of the system

Human-Machine Interaction: Causal Dynamical Networks

The objective of this paper is to introduce a modified version of the Causal Dynamical Networks (CDN) algorithm for application in the human-machine interaction. It is demonstrated that an individual does not interact with one robot, but with a multitude of personalities stored in the robot. These personalities are independent of each other. A robot thus does not have a unique personality. In order for a robot to become a unique individual a new algorithm is proposed. The new algorithm is called the Causal Form Fluctuation Network (CEFN). It is shown that such an algorithm can help machines develop similar to human general intelligence capabilities such as interpretation, wisdom (acquiring knowledge), and prediction (intuition). Also to be able to make decisions, have ideas, and imaginations