Multimodal human machine interactions in industrial environments

This chapter will present a review of Human Machine Interaction techniques for industrial applications. A set of recent HMI techniques will be provided with emphasis on multimodal interaction with industrial machines and robots. This list will include Natural Language Processing techniques and others that make use of various complementary interfaces: audio, visual, haptic or gestural, to achieve a more natural human-machine interaction. This chapter will also focus on providing examples and use cases in fields related to multimodal interaction in manufacturing, such as augmented reality. Accordingly, the chapter will present the use of Artificial Intelligence and Multimodal Human Machine Interaction in the context of STAR applications

AN EVALUATION OF AUGMENTED REALITY-BASED USER INTERFACES IN THE DESIGN PROCESS

The aim of this study is to evaluate the user interfaces that reflect different interaction layers in the context of Augmented Reality technology. Depending on the physical characteristics of human interaction with the computer, these layers were examined under four sections: Graphical User Interface (GUI), Tangible User Interface (TUI), Natural User Interface (NUI) and Spatial User Interface (SUI). In this context, a proposed Augmented Reality application interface has been developed to bridge the physical and digital environment. The use of AR-based applications in the design process provided a basis for evaluating the user interface in these interaction layers. In future studies, the interface and experience offered by this application have the potential to be supported by more comprehensive functions and a collaborative working environment

Exploring the use of smart glasses, gesture control, and environmental data in augmented reality games

Abstract. In the last decade, augmented reality has become a popular trend. Big corporations like Microsoft, Facebook, and Google started to invest in augmented reality because they saw the potential that it has especially with the rising of the consumer version of the head mounted displays such as Microsoft’s HoloLens and the ODG’s R7. However, there is a gap in the knowledge about the interaction with such devices since they are fairly new and an average consumer cannot yet afford them due to their relatively high prices. In this thesis, the Ghost Hunters game is described. The game is a mobile augmented reality pervasive game that uses the environment light data to charge the in-game “goggles”. The game has two different versions, a smartphone and smart glasses version. The Ghost Hunters game was implemented for exploring the use of two different types of interactions methods, buttons and natural hand gestures for both smartphones and smart glasses. In addition to that, the thesis sought to explore the use of ambient light in augmented reality games. First, the thesis defines the essential concepts related to games and augmented reality based on the literature and then describes the current state of the art of pervasive games and smart glasses. Second, both the design and implementation of the Ghost Hunters game are described in detail. Afterwards, the three rounds of field trials that were conducted to investigate the suitability of the two previously mentioned interaction methods are described and discussed. The findings suggest that smart glasses are more immersive than smartphones in context of pervasive AR games. Moreover, prior AR experience has a significant positive impact on the immersion of smart glasses users. Similarly, males were more immersed in the game than females. Hand gestures were proven to be more usable than the buttons on both devices. However, the interaction method did not affect the game engagement at all, but surprisingly it did affect the way users perceive the UI with smart glasses. Users that used the physical buttons were more likely to notice the UI elements than the users who used the hand gestures

From real spaces to virtual spaces: The metaverse and decentralized cinema

Developments in computer and communication technologies, which constitute the starting point of concepts such as decentralization, virtuality, simulation, augmented reality and metaverse, have also brought new forms of expression and designs in art to the agenda. In addition to the decentralized data architecture and metaverse areas that emerged in parallel with the development of network technologies, applications that increase the user's interaction and beleaguered experience such as virtual reality, augmented reality and mixed reality have increased their effectiveness in this field. The metaverse spaces that emerge with the cooperation of software, art and architecture offer their users a more similar life simulation of natural life through augmented reality vehicles or screens. Here, users can perform new experiences for artistic production and consumption as well as daily life practices such as socialization and communication. Metaverse spaces, which include the design of a three-dimensional virtual universe that can be supported by augmented reality, are free from all the constraints of the real world as a cinematic plateau. It is seen as a great advantage that the real film set can create a cinematic work without expensive equipment such as cameras, lights, and sound away from all the negativities of the natural shooting conditions. The fact that the production, distribution and screening of cinema works can be realized within this field brings a new understanding of decentralized cinema to the agenda. Decentralized cinema, which has begun to rise in the expanding virtual geography of the metaverse virtual space with its advantages such as virtual characters and scenes and creative space fictions, is an art form worth examining. This study focuses on the possible future transformations of cinema in terms of production and representation in the context of the relationship of virtual and augmented reality technologies with developing metaverse areas. The emergence of a new cinematic ecology; The opportunities and obstacles it provides to producers are examined with the philosophical criticism method through concepts such as virtual and augmented reality, web 3.0, metaverse in terms of audience experiences it offers for screening.  As a result of the study, it was concluded that the metaverse area has many advantages in terms of the production of cinema works, democratization of the production and distribution of works, digital privacy and security for metaverse artists, and recognition of ownership for digital works of art

Chatbot-Based Natural Language Interfaces for Data Visualisation: A Scoping Review

Rapid growth in the generation of data from various sources has made data visualisation a valuable tool for analysing data. However, visual analysis can be a challenging task, not only due to intricate dashboards but also when dealing with complex and multidimensional data. In this context, advances in Natural Language Processing technologies have led to the development of Visualisation-oriented Natural Language Interfaces (V-NLIs). In this paper, we carry out a scoping review that analyses synergies between the fields of Data Visualisation and Natural Language Interaction. Specifically, we focus on chatbot-based V-NLI approaches and explore and discuss three research questions. The first two research questions focus on studying how chatbot-based V-NLIs contribute to interactions with the Data and Visual Spaces of the visualisation pipeline, while the third seeks to know how chatbot-based V-NLIs enhance users' interaction with visualisations. Our findings show that the works in the literature put a strong focus on exploring tabular data with basic visualisations, with visual mapping primarily reliant on fixed layouts. Moreover, V-NLIs provide users with restricted guidance strategies, and few of them support high-level and follow-up queries. We identify challenges and possible research opportunities for the V-NLI community such as supporting high-level queries with complex data, integrating V-NLIs with more advanced systems such as Augmented Reality (AR) or Virtual Reality (VR), particularly for advanced visualisations, expanding guidance strategies beyond current limitations, adopting intelligent visual mapping techniques, and incorporating more sophisticated interaction methods

Creating and manipulating 3D paths with mixed reality spatial interfaces

Mixed reality offers unique opportunities to situate complex tasks within spatial environments. One such task is the creation and manipulation of intricate, three-dimensional paths, which remains a crucial challenge in many fields, including animation, architecture, and robotics. This paper presents an investigation into the possibilities of spatially situated path creation using new virtual and augmented reality technologies and examines how these technologies can be leveraged to afford more intuitive and natural path creation. We present a formative study (n = 20) evaluating an initial path planning interface situated in the context of augmented reality and human-robot interaction. Based on the findings of this study, we detail the development of two novel techniques for spatially situated path planning and manipulation that afford intuitive, expressive path creation at varying scales. We describe a comprehensive user study (n = 36) investigating the effectiveness, learnability, and efficiency of both techniques when paired with a range of canonical placement strategies. The results of this study confirm the usability of these interaction metaphors and provide further insight into how spatial interaction can be discreetly leveraged to enable interaction at scale. Overall, this work contributes to the development of 3DUIs that expand the possibilities for situating path-driven tasks in spatial environments

A brief review on augmented reality based google glass

Abstract—Augmented reality that is Wearable technology is fast becoming a part of our technological life. There are many devices, from dresses to headphones that can record your day-to- day life. The maturing field of wearable computing aims to inter- weave computing devices into everyday life. This report focuses on smart glasses, one of the categories of wearable computing devices which is present in the media. This paper provides a concise overview of the history and context of Glass based on AR technology. The intended purpose of smart Glass products would be the hands-free displaying of information currently available to most smartphone users, and allowing for interaction with the Internet via natural language voice commands

Modular 3D Interface Design for Accessible VR Applications

Designed with an accessible first design approach, the presented paper describes how exploiting humans proprioception ability in 3D space can result in a more natural interaction experience when using a 3D graphical user interface in a virtual environment. The modularity of the designed interface empowers the user to decide where they want to place interface elements in 3D space allowing for a highly customizable experience, both in the context of the player and the virtual space. Drawing inspiration from todays tangible interfaces used, such as those in aircraft cockpits, a modular interface is presented taking advantage of our natural understanding of interacting with 3D objects and exploiting capabilities that otherwise have not been used in 2D interaction. Additionally, the designed interface supports multimodal input mechanisms which also demonstrates the opportunity for the design to cross over to augmented reality applications. A focus group study was completed to better understand the usability and constraints of the designed 3D GUI.Comment: This preprint has not undergone peer review or any post-submission corrections. The Version of Record of this contribution will be published in Springer Nature Computer Science book series in Volume HCI International 202

Using Augmented Reality as a Medium to Assist Teaching in Higher Education

In this paper we describe the use of a high-level augmented reality (AR) interface for the construction of collaborative educational applications that can be used in practice to enhance current teaching methods. A combination of multimedia information including spatial three-dimensional models, images, textual information, video, animations and sound, can be superimposed in a student-friendly manner into the learning environment. In several case studies different learning scenarios have been carefully designed based on human-computer interaction principles so that meaningful virtual information is presented in an interactive and compelling way. Collaboration between the participants is achieved through use of a tangible AR interface that uses marker cards as well as an immersive AR environment which is based on software user interfaces (UIs) and hardware devices. The interactive AR interface has been piloted in the classroom at two UK universities in departments of Informatics and Information Science