Enhancing User Immersion and Virtual Presence in Interactive Multiuser Virtual Environments through the Development and Integration of a Gesture-Centric Natural User Interface Developed from Existing Virtual Reality Technologies

Immersion, referring to the level of physical or psychological submergence of a user within a virtual space relative to that user's consciousness of the real-world environment, has predominantly been established as an indispensable part of interactive media designs. This is most prevalent in Virtual Reality (VR) platforms, as their applications are typically reliant on user believability. With a wide variation of possible methodologies for the enhancement of this feature, the collectively recognised paradigm lies on the emphasis of naturalism in the design of the virtual system [7]. Though widely used by some specialised VR applications [4] such concepts are yet to be fully explored in the more contemporary virtual systems such as Social Immersive Virtual Environment (SIVE). The focus of the study described in this paper are the techniques being developed to enhance user immersion, virtual presence and co-presence in a SIVE application, through the design and integration of a VR-based Natural User Interface (NUI) that allows users to naturally and intuitively interact with the virtual environment and other networked users through the utilisation of full body gesture controls. These gestural controls prioritise the emulation of the alternate equivalent of such real-wold interactions, whilst also providing an interface for the seamless and unobtrusive translation of the user's real-world physical state into the virtual environment through intuitive user to virtual avatar proprioceptive coordination. © Springer International Publishing Switzerland 2014

Systematic literature review of hand gestures used in human computer interaction interfaces

Gestures, widely accepted as a humans' natural mode of interaction with their surroundings, have been considered for use in human-computer based interfaces since the early 1980s. They have been explored and implemented, with a range of success and maturity levels, in a variety of fields, facilitated by a multitude of technologies. Underpinning gesture theory however focuses on gestures performed simultaneously with speech, and majority of gesture based interfaces are supported by other modes of interaction. This article reports the results of a systematic review undertaken to identify characteristics of touchless/in-air hand gestures used in interaction interfaces. 148 articles were reviewed reporting on gesture-based interaction interfaces, identified through searching engineering and science databases (Engineering Village, Pro Quest, Science Direct, Scopus and Web of Science). The goal of the review was to map the field of gesture-based interfaces, investigate the patterns in gesture use, and identify common combinations of gestures for different combinations of applications and technologies. From the review, the community seems disparate with little evidence of building upon prior work and a fundamental framework of gesture-based interaction is not evident. However, the findings can help inform future developments and provide valuable information about the benefits and drawbacks of different approaches. It was further found that the nature and appropriateness of gestures used was not a primary factor in gesture elicitation when designing gesture based systems, and that ease of technology implementation often took precedence

A novel user-based gesture vocabulary for conceptual design

Research into hand gestures for human computer interaction has been prolific recently, but within it research on hand gestures for conceptual design has either focused on gestures that were defined by the researchers rather than the users, or those that were heavily influenced by what can be achieved using currently available technology. This paper reports on the study performed to identify a user elicited vocabulary of gestures for conceptual design, disassociated from the currently available technology, and its subsequent evaluation. The study included 44 product design engineering students (3rd, 4th year and recent graduates) and identified 1772 gestures that were analysed to build a novel gesture consensus set of vocabulary of hand gestures for conceptual design. This set is then evaluated by 10 other professionals, in order to generalise this set for a wider range of users and possibly reduce the need for training. The evaluation has shown that majority of gestures added to the vocabulary were easy to perform and appropriate for the activities, but that at the implementation stage the vocabulary will require another round of evaluation to account for the technology capabilities. The aim of this work is to create a starting point for a potential future system that could adapt to individual designers and allow them to use non-prescribed gestures that will support rather than inhibit their conceptual design thinking processes, akin to the developments that happened in hand writing recognition or predictive texting

Avaliação comparativa de técnicas para reconhecimento de gestos estáticos e dinâmicos com foco em precisão e desempenho

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Ciência da Computação, Florianópolis, 2010É um comportamento comum aos seres humanos utilizar gestos como forma de expressão, como um complemento à fala ou como uma forma auto-contida de comunicação. No campo da Interação Humano-Computador, esse comportamento pode ser adotado na construção de interfaces alternativas, objetivando facilitar o relacionamento entre os elementos humano e computacional. Atualmente, várias técnicas para reconhecimento de gestos são descritas na literatura; porém, as validações dessas técnicas são executadas de maneira isolada, o que dificulta a comparação entre elas. Para reduzir essa lacuna, este trabalho apresenta uma comparação entre técnicas estabelecidas para o reconhecimento de gestos estáticos (posturas) e gestos dinâmicos (trajetórias). Essas técnicas são organizadas de forma a avaliar um conjunto de dados comum, adquirido por meio de uma luva instrumentada e um rastreador de movimento, gerando resultados em termos de precisão e desempenho. Especificamente para trajetórias, o processo de avaliação considera técnicas conhecidas (redes neurais e modelos ocultos de Markov) e uma nova heurística baseada em autômatos finitos determinísticos, idealizada e desenvolvida pelos autores. Os resultados obtidos mostram que o classificador baseado em uma SVM (Support Vector Machine) apresentou a melhor generalização, com as melhores taxas de reconhecimento para posturas. Para trajetórias, por sua vez, o classificador baseado em uma rede neural gerou os melhores resultados. Em termos de desempenho, todos os métodos apresentaram resultados suficientemente rápidos para serem usados de forma interativa. Finalmente, o presente trabalho identifica e discute um conjunto de critérios relevantes que deve ser observado nas etapas de construção, treinamento e avaliação dos classificadores, e sua relação com os resultados finais

User-based gesture vocabulary for form creation during a product design process

There are inconsistencies between the nature of the conceptual design and the functionalities of the computational systems supporting it, which disrupt the designers’ process, focusing on technology rather than designers’ needs. A need for elicitation of hand gestures appropriate for the requirements of the conceptual design, rather than those arbitrarily chosen or focusing on ease of implementation was identified.The aim of this thesis is to identify natural and intuitive hand gestures for conceptual design, performed by designers (3rd, 4th year product design engineering students and recent graduates) working on their own, without instruction and without limitations imposed by the facilitating technology. This was done via a user centred study including 44 participants. 1785 gestures were collected. Gestures were explored as a sole mean for shape creation and manipulation in virtual 3D space. Gestures were identified, described in writing, sketched, coded based on the taxonomy used, categorised based on hand form and the path travelled and variants identified. Then they were statistically analysed to ascertain agreement rates between the participants, significance of the agreement and the likelihood of number of repetitions for each category occurring by chance. The most frequently used and statistically significant gestures formed the consensus set of vocabulary for conceptual design. The effect of the shape of the manipulated object on the gesture performed, and if the sequence of the gestures participants proposed was different from the established CAD solid modelling practices were also observed.Vocabulary was evaluated by non-designer participants, and the outcomes have shown that the majority of gestures were appropriate and easy to perform. Evaluation was performed theoretically and in the VR environment. Participants selected their preferred gestures for each activity, and a variant of the vocabulary for conceptual design was created as an outcome, that aims to ensure that extensive training is not required, extending the ability to design beyond trained designers only.There are inconsistencies between the nature of the conceptual design and the functionalities of the computational systems supporting it, which disrupt the designers’ process, focusing on technology rather than designers’ needs. A need for elicitation of hand gestures appropriate for the requirements of the conceptual design, rather than those arbitrarily chosen or focusing on ease of implementation was identified.The aim of this thesis is to identify natural and intuitive hand gestures for conceptual design, performed by designers (3rd, 4th year product design engineering students and recent graduates) working on their own, without instruction and without limitations imposed by the facilitating technology. This was done via a user centred study including 44 participants. 1785 gestures were collected. Gestures were explored as a sole mean for shape creation and manipulation in virtual 3D space. Gestures were identified, described in writing, sketched, coded based on the taxonomy used, categorised based on hand form and the path travelled and variants identified. Then they were statistically analysed to ascertain agreement rates between the participants, significance of the agreement and the likelihood of number of repetitions for each category occurring by chance. The most frequently used and statistically significant gestures formed the consensus set of vocabulary for conceptual design. The effect of the shape of the manipulated object on the gesture performed, and if the sequence of the gestures participants proposed was different from the established CAD solid modelling practices were also observed.Vocabulary was evaluated by non-designer participants, and the outcomes have shown that the majority of gestures were appropriate and easy to perform. Evaluation was performed theoretically and in the VR environment. Participants selected their preferred gestures for each activity, and a variant of the vocabulary for conceptual design was created as an outcome, that aims to ensure that extensive training is not required, extending the ability to design beyond trained designers only