Multimodality with Eye tracking and Haptics: A New Horizon for Serious Games?

The goal of this review is to illustrate the emerging use of multimodal virtual reality that can benefit learning-based games. The review begins with an introduction to multimodal virtual reality in serious games and we provide a brief discussion of why cognitive processes involved in learning and training are enhanced under immersive virtual environments. We initially outline studies that have used eye tracking and haptic feedback independently in serious games, and then review some innovative applications that have already combined eye tracking and haptic devices in order to provide applicable multimodal frameworks for learning-based games. Finally, some general conclusions are identified and clarified in order to advance current understanding in multimodal serious game production as well as exploring possible areas for new applications

Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe

Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe

Smart working technologies in industry 4.0 : contributions to different manufacturing activities and workers’ skills

A Indústria 4.0 é considerada a quarta revolução industrial porque utiliza uma ampla integração de tecnologias de informação e de operação na fabricação industrial. Apesar dessa perspectiva tecnológica, diversos estudos vêm evidenciando a importância de considerar o fator humano para o desenvolvimento de um sistema de manufatura inteligente. Nesse sentido, a dimensão denominada como Smart Working precisa ser melhor investigada, uma vez que entender como as tecnologias afetam os trabalhadores e as habilidades desses são cruciais para o bom desempenho das fábricas. Em razão disso, o objetivo desta dissertação foi entender como as Smart Working Technologies (SWT) podem contribuir para as atividades e as habilidades dos trabalhadores da manufatura. Para tanto, primeiramente foi realizada uma análise abrangente da literatura para identificar as SWT e seus impactos nas capacidades dos trabalhadores em suas atividades de manufatura. Deste modo, foram analisados 80 artigos que relacionam as SWT em oito atividades de manufatura. Posteriormente, foi selecionada uma das SWT mais relevantes conforme a literatura, os robôs colaborativos, para identificar os efeitos das tecnologias nas habilidades dos trabalhadores. Deste modo, foram analisados 138 casos de aplicação reportados por uma das empresas fornecedoras líderes mundiais, bem como três entrevistas com empresas adotantes da tecnologia. Os resultados apontam que existem 15 SWT que podem ser implementadas nas atividades de manufatura e relacionadas às capacidades dos trabalhadores. Além disso, os resultados também apontam que podem existir quatro efeitos das SWT nas habilidades dos trabalhadores. Estes achados demonstram que de acordo com a estratégia da empresa uma SWT pode impactar de diferentes formas os trabalhadores.Industry 4.0 is considered the fourth industrial revolution because it uses a broad integration of information and operating technologies in industrial manufacturing. Despite this technological perspective, several studies have highlighted the importance of considering the human factor to develop a smart manufacturing system. In this sense, the Smart Working dimension needs to be further investigated since understanding how technologies affect workers and their skills are crucial for factories' good performance. Therefore, the objective of this dissertation was to understand how Smart Working Technologies (SWT) can contribute to the activities and skills of manufacturing workers. To this end, firstly a systematic literature review was carried out to identify SWTs and their impacts on workers' capabilities in their manufacturing activities. Thus, 80 articles relating to SWT in eight manufacturing activities were analyzed. Subsequently, one of the most relevant SWTs according to the literature, collaborative robots, was selected to identify the effects of technologies on workers' skills. In this way, 138 application cases reported by one of the world's leading supplier companies were analyzed, as well as three interviews with companies that adopted the technology. The results show that there are 15 SWT that can be implemented in manufacturing activities and related to workers' capabilities. In addition, the results also point out that there may be four effects of SWT on workers' skills. According to the company's strategy, these findings demonstrate that an SWT can impact workers in different ways

Ubiquitous haptic feedback in human-computer interaction through electrical muscle stimulation

[no abstract