Understanding the impact of multimodal interaction using gaze informed mid-air gesture control in 3D virtual objects manipulation.

Multimodal interactions provide users with more natural ways to manipulate virtual 3D objects than using traditional input methods. An emerging approach is gaze modulated pointing, which enables users to perform object selection and manipulation in a virtual space conveniently through the use of a combination of gaze and other interaction techniques (e.g., mid-air gestures). As gaze modulated pointing uses different sensors to track and detect user behaviours, its performance relies on the user's perception on the exact spatial mapping between the virtual space and the physical space. An underexplored issue is, when the spatial mapping differs with the user's perception, manipulation errors (e.g., out of boundary errors, proximity errors) may occur. Therefore, in gaze modulated pointing, as gaze can introduce misalignment of the spatial mapping, it may lead to user's misperception of the virtual environment and consequently manipulation errors. This paper provides a clear definition of the problem through a thorough investigation on its causes and specifies the conditions when it occurs, which is further validated in the experiment. It also proposes three methods (Scaling, Magnet and Dual-gaze) to address the problem and examines them using a comparative study which involves 20 participants with 1040 runs. The results show that all three methods improved the manipulation performance with regard to the defined problem where Magnet and Dual-gaze delivered better performance than Scaling. This finding could be used to inform a more robust multimodal interface design supported by both eye tracking and mid-air gesture control without losing efficiency and stability

Talking about interaction*

© 2019 Elsevier Ltd Recent research has exposed disagreements over the nature and usefulness of what may (or may not) be Human–Computer Interaction's fundamental phenomenon: ‘interaction’. For some, HCI's theorising about interaction has been deficient, impacting its capacity to inform decisions in design, suggesting the need either to perform first-principles definition work or broader administrative clarification and formalisation of the multitude of formulations of the concepts of interaction and their particular uses. For others, there remain open questions over the continued relevance of certain ‘versions’ of interaction as a useful concept in HCI at all. We pursue a different perspective in this paper, reviewing how HCI treats interaction through examining its ‘conceptual pragmatics’ within HCI's discourse. We argue that articulations of the concepts of interaction can be a site of productive conflict for HCI that for many reasons may resist attempts of formalisation as well as attempts to dispense with them. The main contribution of this paper is in specifying how we might go about talking of interaction and the value of interaction language as promiscuous concepts

Review of three-dimensional human-computer interaction with focus on the leap motion controller

Modern hardware and software development has led to an evolution of user interfaces from command-line to natural user interfaces for virtual immersive environments. Gestures imitating real-world interaction tasks increasingly replace classical two-dimensional interfaces based on Windows/Icons/Menus/Pointers (WIMP) or touch metaphors. Thus, the purpose of this paper is to survey the state-of-the-art Human-Computer Interaction (HCI) techniques with a focus on the special field of three-dimensional interaction. This includes an overview of currently available interaction devices, their applications of usage and underlying methods for gesture design and recognition. Focus is on interfaces based on the Leap Motion Controller (LMC) and corresponding methods of gesture design and recognition. Further, a review of evaluation methods for the proposed natural user interfaces is given

U2XECS : avaliação de usabilidade e experiência de usuário de sistemas conversacionais

Orientadora: Natasha Malveira Costa ValentimDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 26/11/2020Inclui referências: p. 111-121Área de concentração: Ciência da ComputaçãoResumo: Devido ao aumento do uso das tecnologias nos ultimos anos, novas formas de interacao estao presentes no cotidiano da sociedade. A interacao baseada em voz, caracteristica dos Sistemas Conversacionais (SC), e um exemplo dessas novas formas de interagir. Amazon Alexa, Siri, Google Assistant, Amazon Frame, dispostivos Amazon Echo e Google Home sao exemplos de SCs que utilizam a voz do usuario para desempenhar tarefas. Os SCs tem despertado interesse tanto da industria como da academia, recebendo investimentos e fazendo parte de pesquisa em Interacao Humano-Computador (IHC) e Engenharia de Software (ES). Como qualquer outro sistema, e necessario que os SCs fornecam uma boa experiencia e que atendam as necessidades de seus usuarios. Nesse sentido, a avaliacao de Usabilidade e de Experiencia do Usuario (User eXperience - UX) e vista como etapa importante que contribui com a verificacao da qualidade dos SCs. Na avaliacao da Usabilidade, geralmente sao verificados atributos referentes as metas comportamentais do sistema, como a eficacia, eficiencia e satisfacao do usuario. Ja na avaliacao da UX, geralmente sao considerados os atributos ligados ao sentimento do usuarios, como emocao e motivacao. No entanto, atraves de dois Mapeamentos Sistematicos da Literatura (MSL), foi identificado que as tecnologias de avaliacao utilizadas para avaliar a Usabilidade e/ou UX dos SCs eram genericas, e poderiam avaliar qualquer tipo software. Alem disso, foram identificados alguns questionarios de avaliacao de Usabilidade ou UX de interfaces conversacionais. Entretanto, estas tecnologias consideram apenas um aspecto de qualidade, Usabilidade, ou UX. Os MSLs tambem identificaram que alguns pesquisadores utilizam questionarios que desenvolveram para seus proprios estudos, sem passar por um processo de avaliacao empirica. Sendo assim, o objetivo deste trabalho e fornecer uma tecnologia de avaliacao conjunta de Usabilidade e UX especifica para SCs, a U2XECS (Usability and User eXperience Evaluation of Conversational Systems). A U2XECS e uma tecnologia de avaliacao baseada em questionario que fornece afirmacoes de Usabilidade e UX especificas para avaliar SCs. O objetivo do U2XECS e orientar pesquisadores e desenvolvedores para identificar melhorias e percepcoes dos usuarios nestes sistemas. Alem dos MSLs e da proposicao da tecnologia, sao apresentados tambem tres estudos que foram realizados no processo de elaboracao e refinamento da tecnologia: um estudo exploratorio, um survey e um estudo de viabilidade. Os resultados evidenciaram pontos positivos da U2XECS relacionados a facilidade de uso, utilidade e intencoes de uso. Alem disso, foram identificadas oportunidades de melhoria, tais como afirmacoes ambiguas, mudanca na estrutura e no tamanho do questionario. Palavras-chave: Avaliacao de Usabilidade. Avaliacao de Experiencia de Usuario. Interacao Baseada em Voz. Sistemas Conversacionais.Abstract: Due to the increased use of technologies in recent years, new forms of interaction are present in society. The voice-based interaction, characteristic of Conversational Systems (CSs), is an example of these new interaction forms. Amazon Alexa, Siri, Google Assistant, Amazon Frame, Amazon Echo devices, and Google Home are examples of CSs that use the voice to perform tasks. The CSs have aroused interest from both industry and academia, receiving investments and being part of research in Human-Computer Interaction (HCI) and Software Engineering (SE). Like any other system, CSs must provide a good experience and meet the needs of their users. In this sense, the evaluation of Usability and User eXperience (UX) is seen as an essential step that contributes to verifying the quality of the CSs. In the Usability evaluation, attributes regarding the system's behavioral goals, such as effectiveness, efficiency, and user satisfaction, are usually verified. In the UX evaluation, attributes related to the user's feelings, such as emotion and motivation, are usually considered. However, through two Systematic Mapping Studies (SMS), it was identified that the evaluation technologies used to evaluate the Usability and/or UX of the CSs were generic and could evaluate any software. Besides, some Usability or UX evaluation questionnaires of conversational interfaces were identified. However, these technologies consider only one aspect of quality, Usability, or UX. SMSs also identified that some researchers use questionnaires that they developed for their studies without going through an empirical evaluation process. Therefore, this work aims to provide a CS-specific joint Usability and UX evaluation technology, the U2XECS (Usability and User eXperience Evaluation of Conversational Systems). U2XECS is a questionnaire-based evaluation technology that provides Usability and UX specific statements to evaluate CSs. The goal of U2XECS is to guide researchers and developers to identify improvements and user perceptions in these systems. Besides the SMSs and the technology proposition, three studies that were carried out in the process of elaboration and refinement of the technology are presented: an exploratory study, a survey, and a feasibility study. The results showed positive points of U2XECS related to ease of use, utility, and intentions of use. Besides, opportunities for improvement were identified, such as ambiguous statements, change in the structure and size of the questionnaire. Keywords: Usability Evaluation. User Experience Evaluation. Voice-Based Interaction. Conversational Systems