Interaction design for multi-user virtual reality systems: An automotive case study

Virtual reality (VR) technology have become ever matured today. Various research and practice have demonstrated the potential benefits of using VR in different application area of manufacturing, such as in factory layout planning, product design, training, etc. However, along with the new possibilities brought by VR, comes with the new ways for users to communicate with the computer system. The human computer interaction design for these VR systems becomes pivotal to the smooth integration. In this paper, it reports the study that investigates interaction design strategies for the multi-user VR system used in manufacturing context though an automotive case study

External legibility and seamlessness in interface design

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2008.Includes bibliographical references (p. 97-108).This thesis presents External Legibility: a property of user interfaces that affects the ability of non-participating observers to understand the context of a user's actions. Claims of its value are supported with arguments from the social sciences and human-computer interaction literature; research in designing tangible user interfaces; and an experiment comparing the external legibility of four interaction techniques.by Jamie B. Zigelbaum.S.M

Affordance-map : learning hidden human context in 3D scenes through virtual human models

University of Technology Sydney. Faculty of Engineering and Information Technology.Ability to learn human context in an environment could be one of the most desired fundamental abilities that a robot should possess when sharing workspaces with human co-workers. Arguably, a robot with appropriate human context awareness could lead to a better human robot interaction. This thesis addresses the problem of learning human context in indoor environments by only looking at geometrics features of the environment. The novelty of this concept is, it does not require to observe real humans to learn human context. Instead, it uses virtual human models and their relationships with the environment to map hidden human affordances in 3D scenes. The problem of affordance mapping is formulated as a multi label classification problem with a binary classifier for each affordance type. The initial experiments proved that the SVM classifier is ideally suited for affordance mapping. However, SVM classifier recorded sub-optimum results when trained with imbalanced datasets. This imbalance occurs because in all 3D scenes in the dataset, the number of negative examples outnumbered positive examples by a great margin. As a solution to this, a number of SVM learners that are designed to tolerate class imbalance problem are tested for learning the affordance-map. These algorithms showed some tolerance to moderate class imbalances, but failed to perform well in some affordance types. To mitigate these drawbacks, this thesis proposes the use of Structured SVM (S-SVM) optimized for F1-score. This approach defines the affordance-map building problems as a structured learning problem and outputs the most optimum affordance-map for a given set of features (3D-Images). In addition, S-SVM can be learned efficiently even on a large extremely imbalanced dataset. Further, experimental results of the S-SVM method outperformed previously used classifiers for mapping affordances. Finally, this thesis presents two applications of the affordance-map. In the first application, affordance-map is used by a mobile robot to actively search for computer monitors in an office environment. The orientation and location information of humans models inferred by the affordance-map is used in this application to predict probable locations of computer monitors. The experimental results in a large office environment proved that the affordance-map concept simplifies the search strategy of the robot. In the second application, affordance-map is used for context aware path planning. In this application, human context information of the affordance-map is used by a service robot to plan paths with minimal distractions to office workers

Emotionally interactive agents

Models in language processing have researched how words are interpreted by humans. Many models presume the ability to correctly interpret the beliefs, motives and intentions underlying words. The interest relies also on how emotion motivates certain words or actions, inferences, and communicates information about mental state. As we will see below, some tutoring systems have explored this potential to inform user models. Likewise, dialogue systems, mixed-initiative planning systems, or systems that learn from observation could also benefit from such an approach. As these experimental data show, activating accessible constructs or attitudes through one set of stimuli can facilitate cognitive processing of other stimuli under certain circumstances, and can interfere with it under other circumstances. Some of the results support and converge on those centered on the constructs of current concern and emotional arousal. Future research has to take seriously into account this question: how to develop models where emotion interacts with cognitive processing. One example could be the work of Pitterman et al. (2010) where it is combined speech-based emotion recognition with adaptive human-computer modeling. With the robust recognition of emotions from speech signals as their goal, the authors analyze the effectiveness of using a plain emotion recognizer, a speech-emotion recognizer combining speech and emotion recognition, and multiple speech-emotion recognizers at the same time. The semi-stochastic dialogue model employed relates user emotion management to the corresponding dialogue interaction history and allows the device to adapt itself to the context, including altering the stylistic realization of its speech.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

An information assistant system for the prevention of tunnel vision in crisis management

In the crisis management environment, tunnel vision is a set of bias in decision makers’ cognitive process which often leads to incorrect understanding of the real crisis situation, biased perception of information, and improper decisions. The tunnel vision phenomenon is a consequence of both the challenges in the task and the natural limitation in a human being’s cognitive process. An information assistant system is proposed with the purpose of preventing tunnel vision. The system serves as a platform for monitoring the on-going crisis event. All information goes through the system before arrives at the user. The system enhances the data quality, reduces the data quantity and presents the crisis information in a manner that prevents or repairs the user’s cognitive overload. While working with such a system, the users (crisis managers) are expected to be more likely to stay aware of the actual situation, stay open minded to possibilities, and make proper decisions

Action Selection for Interaction Management: Opportunities and Lessons for Automated Planning

The central problem in automated planning---action selection---is also a primary topic in the dialogue systems research community, however, the nature of research in that community is significantly different from that of planning, with a focus on end-to-end systems and user evaluations. In particular, numerous toolkits are available for developing speech-based dialogue systems that include not only a method for representing states and actions, but also a mechanism for reasoning and selecting the actions, often combined with a technical framework designed to simplify the task of creating end-to-end systems. We contrast this situation with that of automated planning, and argue that the dialogue systems community could benefit from some of the directions adopted by the planning community, and that there also exist opportunities and lessons for automated planning

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Usability dimensions in collaborative GIS

Collaborative GIS requires careful consideration of the Human-Computer Interaction (HCI) and Usability aspects, given the variety of users that are expected to use these systems, and the need to ensure that users will find the system effective, efficient, and enjoyable. The chapter explains the link between collaborative GIS and usability engineering/HCI studies. The integration of usability considerations into collaborative GIS is demonstrated in two case studies of Web-based GIS implementation. In the first, the process of digitising an area on Web-based GIS is improved to enhance the user's experience, and to allow interaction over narrowband Internet connections. In the second, server-side rendering of 3D scenes allows users who are not equipped with powerful computers to request sophisticated visualisation without the need to download complex software. The chapter concludes by emphasising the need to understand the users' context and conditions within any collaborative GIS project. © 2006, Idea Group Inc

Temporal DINO: A Self-supervised Video Strategy to Enhance Action Prediction

The emerging field of action prediction plays a vital role in various computer vision applications such as autonomous driving, activity analysis and human-computer interaction. Despite significant advancements, accurately predicting future actions remains a challenging problem due to high dimensionality, complex dynamics and uncertainties inherent in video data. Traditional supervised approaches require large amounts of labelled data, which is expensive and time-consuming to obtain. This paper introduces a novel self-supervised video strategy for enhancing action prediction inspired by DINO (self-distillation with no labels). The Temporal-DINO approach employs two models; a 'student' processing past frames; and a 'teacher' processing both past and future frames, enabling a broader temporal context. During training, the teacher guides the student to learn future context by only observing past frames. The strategy is evaluated on ROAD dataset for the action prediction downstream task using 3D-ResNet, Transformer, and LSTM architectures. The experimental results showcase significant improvements in prediction performance across these architectures, with our method achieving an average enhancement of 9.9% Precision Points (PP), highlighting its effectiveness in enhancing the backbones' capabilities of capturing long-term dependencies. Furthermore, our approach demonstrates efficiency regarding the pretraining dataset size and the number of epochs required. This method overcomes limitations present in other approaches, including considering various backbone architectures, addressing multiple prediction horizons, reducing reliance on hand-crafted augmentations, and streamlining the pretraining process into a single stage. These findings highlight the potential of our approach in diverse video-based tasks such as activity recognition, motion planning, and scene understanding

Information support and interactive planning in the digital factory : approach and industry-driven evaluation

In the modern world we are continuously surrounded by information. The human brain has to analyse and interpret this information to transform into useable knowledge that is then used in decision making activities. The advent and implementation of Industry 4.0 will make it a requirement for systems within factories to interact and share large quantities of information with each other. This large volume of information will make it even more difficult for the human resources within the factory to sift through the large amount of information required since there is a limit to the information that our brains can cope with. Just in time information retrieval (JITIR) within the digital factory environment aims to provide support to the human stakeholders in the system by proactively yet non-intrusively providing the required information at the right time based on the users context. This paper will therefore provide an insight into the cognitive difficulties experienced by humans in the digital factory and how JITIR can tackle these challenges. By validating the JITIR concept, several industry scenarios have been evaluated: an exemplary model, concerning the machine tool industry, is presented in the paper. The results of this research are a set of guidelines for the development of a digital factory support tool.peer-reviewe