Gaze-based Interaction for Virtual Environments

Abstract We present an alternative interface that allows users to perceive new sensations in virtual environments. Gaze-based interaction in virtual environments creates the feeling of controlling objects with the mind, arguably translating into a more intense immersion sensation. Additionally, it is also free of some of the most cumbersome aspects of interacting in virtual worlds. By incorporating a real-time physics engine, the sensation of moving something real is further accentuated. We also describe various simple yet effective techniques that allow eyetracking devices to enhance the three-dimensional visualization capabilities of current displays. Some of these techniques have the additional advantage of freeing the mouse from most navigation tasks. This work focuses on the study of existing techniques, a detailed description of the implemented interface and the evaluation (both objective and subjective) of the interface. Given that appropriate filtering of the data from the eye tracker used is a key aspect for the correct functioning of the interface, we will also discuss that aspect in depth

Enhancing User Immersion and Virtual Presence in Interactive Multiuser Virtual Environments through the Development and Integration of a Gesture-Centric Natural User Interface Developed from Existing Virtual Reality Technologies

Immersion, referring to the level of physical or psychological submergence of a user within a virtual space relative to that user's consciousness of the real-world environment, has predominantly been established as an indispensable part of interactive media designs. This is most prevalent in Virtual Reality (VR) platforms, as their applications are typically reliant on user believability. With a wide variation of possible methodologies for the enhancement of this feature, the collectively recognised paradigm lies on the emphasis of naturalism in the design of the virtual system [7]. Though widely used by some specialised VR applications [4] such concepts are yet to be fully explored in the more contemporary virtual systems such as Social Immersive Virtual Environment (SIVE). The focus of the study described in this paper are the techniques being developed to enhance user immersion, virtual presence and co-presence in a SIVE application, through the design and integration of a VR-based Natural User Interface (NUI) that allows users to naturally and intuitively interact with the virtual environment and other networked users through the utilisation of full body gesture controls. These gestural controls prioritise the emulation of the alternate equivalent of such real-wold interactions, whilst also providing an interface for the seamless and unobtrusive translation of the user's real-world physical state into the virtual environment through intuitive user to virtual avatar proprioceptive coordination. © Springer International Publishing Switzerland 2014

A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms

In this paper a review is presented of the research on eye gaze estimation techniques and applications, that has progressed in diverse ways over the past two decades. Several generic eye gaze use-cases are identified: desktop, TV, head-mounted, automotive and handheld devices. Analysis of the literature leads to the identification of several platform specific factors that influence gaze tracking accuracy. A key outcome from this review is the realization of a need to develop standardized methodologies for performance evaluation of gaze tracking systems and achieve consistency in their specification and comparative evaluation. To address this need, the concept of a methodological framework for practical evaluation of different gaze tracking systems is proposed.Comment: 25 pages, 13 figures, Accepted for publication in IEEE Access in July 201

Investigating gaze interaction to support children’s gameplay

Gaze interaction has become an affordable option in the development of innovative interaction methods for user input. Gaze holds great promise as an input modality, offering increased immersion and opportunities for combined interactions (e.g., gaze and mouse, touch). However, the use of gaze as an input modality to support children’s gameplay has not been examined to unveil those opportunities. To investigate the potential of gaze interaction to support children’s gameplay, we designed and developed a game that enables children to utilize gaze interaction as an input modality. Then, we performed a between subjects research design study with 28 children using mouse as an input mechanism and 29 children using their gaze (8–14 years old). During the study, we collected children’s attitudes (via self-reported questionnaire) and actual usage behavior (using facial video, physiological data and computer logs). The results show no significant difference on children’s attitudes regarding the ease of use and enjoyment of the two conditions, as well as on the scores achieved and number of sessions played. Usage data from children’s facial video and physiological data show that sadness and stress are significantly higher in the mouse condition, while joy, surprise, physiological arousal and emotional arousal are significantly higher in the gaze condition. In addition, our findings highlight the benefits of using multimodal data to reveal children’s behavior while playing the game, by complementing self-reported measures. As well, we uncover a need for more studies to examine gaze as an input mechanism.peer-reviewe

Plataforma para a configuração de ambientes virtuais interativos

Mestrado em Sistemas de InformaçãoThis dissertation presents the creation of the Platform for Setting-up Interactive Virtual Environments (pSIVE). Bearing in mind the difficulty required to create virtual environments, the platform aims to allow non-specialists to benefit from virtual environments in applications such as virtual tours as marketing or training where one could interact with elements of the environment to extract contextual information. For this, several frameworks and technologies possible of been integrated into the platform are presented, as well as which ones are more suitable. The platform allows users, from a configuration tool, to create virtual environments and set up their aspects, modes of interaction and what hardware to use. The construction of the world is done by loading 3D models and associating multimedia information (videos, texts or PDF documents) to them. Alongside its development, a comparative study between two ray-tracing selection techniques was performed. Based on the results analysis, it is suggested which technique better fits the environments created with pSIVE. The study also demonstrates the flexibility of the platform, since it was adapted to serve as a test environment. A case of study is introduced where a step by step configuration of a virtual environment is shown, as well as its use within the PRODUTECH-PTI project. Finally, the conclusions are drawn, and suggestions for future work are presented.Este trabalho apresenta a criação da Plataforma para Configuração de Ambientes Virtuais Interativos (com o acrónimo em Inglês pSIVE). Tendo em mente a dificuldade necessária para a criação de ambientes virtuais, a plataforma tem como objectivo possibilitar a não especialistas tirarem proveito de ambientes virtuais, em aplicações genéricas, como por exemplo visitas virtuais que sirvam como publicidade ou treino onde seja possível interagir com elementos do ambiente para extrair informação contextualizada. Para isto apresenta-se um levantamento de tecnologias e frameworks passíveis de serem envolvidos no processo de criação e justifica-se a escolha dos mais adequados para integrar a plataforma. A plataforma permite que utilizadores, a partir de uma ferramenta de configuração, criem ambientes virtuais e seus aspectos, bem como modos de interação e indiquem o hardware a ser utilizado. Para a construção do mundo, é possível carregar modelos 3D associando-lhes informação multimédia (Vídeos, Textos ou Documentos PDF). Paralelamente ao desenvolvimento da plataforma, foi realizado um estudo comparativo entre duas técnicas de seleção por ray-tracing, que diferem quanto à origem do feixe. A análise dos resultados sugere qual técnica que melhor se adequa aos ambientes criados. O estudo também demonstra a flexibilidade da plataforma, uma vez que esta foi adaptada para servir como ambiente de teste. Apresenta-se ainda um caso de estudo, onde se mostra passo a passo a configuração de um ambiente virtual e a sua utilização no âmbito do projeto PRODUTECH-PTI. Por fim, são apresentadas conclusões e possíveis caminhos a serem seguidos para a evolução futura do trabalho

Language-Learner Computer Interactions

This book focuses on learner-computer interactions (LCI) in second language learning environments drawing largely on sociocultural theories of language development. It brings together a rich and varied range of theoretical discussions and applications in order to illustrate the way in which LCI can enrich our comprehension of technology-mediated communication, hence enhancing learners’ digital literacy skills. The book is based on the premise that, in order to fully understand the nature of language and literacy development in digital spaces, researchers and practitioners in linguistics, sciences and engineering need to borrow from each others’ theoretical and practical toolkits. In light of this premise, themes include such aspects as educational ergonomics, affordances, complex systems learning, learner personas and corpora, while also describing such data collecting tools as video screen capture devices, eye-tracking or intelligent learning tutoring systems

Moving out from the focus:Exploring gaze interaction design in games

Eye trackers have become an aordable and compelling input device for game interaction that is targeting the PC gaming community. The number of games adopting gaze input for in-game interaction has rapidly increased over the years with examples in mainstream game franchises. However, games have focused on integrating gaze input on top of fully functional games, utilising gaze as a pointing device and a tool for eciency; e.g. for the faster selection of game objects the player looks at to improve their performance. We deem this is limiting because the use of gaze is obvious, it does not harvest the full potential and richness of the eyes, and only considers that players look at game elements to interact with them. Accordingly, this thesis investigates new opportunities for gaze in games by exploring gaze concepts that challenge the interaction metaphor "what you look at is what you get" to propose adopting "not looking" gaze interactions that reflect what we can do with our eyes. Three playful concepts stem out from this principle: (1) playing with tension; (2) playing with peripheral vision; and (3) playing without looking. We operationalise each concept with game prototypes that pose different challenges based on visual attention, perception in the wider visual eld, and the ability to move the eyes with the eyelids closed. These demonstrate that ideas tested playfully can lead to useful solutions. Finally, we look across our work to distil guidelines to design with "not looking" interactions, the use of dramatisation to support the integration of gaze interaction in the game, and the exploration of interactive experiences only possible when taking input from the eyes. We aim to inspire the future of gaze-enabled games with new directions by proposing that there is more to the eyes than where players look

Une approche logicielle du traitement de la dyslexie : étude de modèles et applications

Neuropsychological disorders are widespread and generate real public health problems. In particular in our modern society, where written communication is ubiquitous, dyslexia can be extremely disabling. Nevertheless we can note that the diagnosis and remediation of this pathology are fastidious and lack of standardization. Unfortunately it seems inherent to the clinical characterization of dyslexia by exclusion, to the multitude of different practitioners involved in such treatment and to the lack of objectivity of some existing methods. In this respect, we decided to investigate the possibilities offered by modern computing to overcome these barriers. Indeed we have assumed that the democratization of computer systems and their computing power could make of them a perfect tool to alleviate the difficulties encountered in the treatment of dyslexia. This research has led us to study the techniques software as well as hardware, which can conduct to the development of an inexpensive and scalable system able to attend a beneficial and progressive changing of practices in this pathology field. With this project we put ourselves definitely in an innovative stream serving quality of care and aid provided to people with disabilities. Our work has been identifying different improvement areas that the use of computers enables. Then each of these areas could then be the subject of extensive research, modeling and prototype developments. We also considered the methodology for designing this kind of system as a whole. In particular our thoughts and these accomplishments have allowed us to define a software framework suitable for implementing a software platform that we called the PAMMA. This platform should theoretically have access to all the tools required for the flexible and efficient development of medical applications integrating business processes. In this way it is expected that this system allows the development of applications for caring dyslexic patients thus leading to a faster and more accurate diagnosis and a more appropriate and effective remediation. Of our innovation efforts emerge encouraging perspectives. However such initiatives can only be achieved within multidisciplinary collaborations with many functional, technical and financial means. Creating such a consortium seems to be the next required step to get a funding necessary for realizing a first functional prototype of the PAMMA, as well as its first applications. Some clinical studies may be conducted to prove undoubtedly the effectiveness of such an approach for treating dyslexia and eventually other neuropsychological disorders.Les troubles neuropsychologiques sont très répandus et posent de réels problèmes de santé publique. En particulier, dans notre société moderne où la communication écrite est omniprésente, la dyslexie peut s’avérer excessivement handicapante. On remarque néanmoins que le diagnostic et la remédiation de cette pathologie restent délicats et manquent d’uniformisation. Ceci semble malheureusement inhérent à la caractérisation clinique par exclusion de la dyslexie, à la multitude de praticiens différents impliqués dans une telle prise en charge ainsi qu’au manque d’objectivité de certaines méthodes existantes. A ce titre, nous avons décidé d’investiguer les possibilités offertes par l’informatique actuelle pour surmonter ces barrières. Effectivement, nous avons supposé que la démocratisation des systèmes informatiques et leur puissance de calcul pourraient en faire un outil de choix pour pallier les difficultés rencontrées lors de la prise en charge de la dyslexie. Cette recherche nous a ainsi mené à étudier les techniques, aussi bien logicielles que matérielles, pouvant conduire au développement d’un système bon marché et évolutif qui serait capable d’assister un changement bénéfique et progressif des pratiques qui entourent cette pathologie. Avec ce projet, nous nous plaçons définitivement dans un courant innovant au service de la qualité des soins et des aides apportées aux personnes souffrant d’un handicap. Notre travail a ainsi consisté à identifier différents axes d’amélioration que l’utilisation de l’outil informatique rend possible. Chacun de ces axes a alors pu faire l’objet de recherches exhaustives, de modélisations et de développements de prototypes. Nous avons également réfléchi à la méthodologie à mettre en œuvre pour concevoir un tel système dans sa globalité. En particulier, nos réflexions et ces différents accomplissements nous ont permis de définir un framework logiciel propice à l’implémentation d’une plate-forme logicielle que nous avons appelée la PAMMA. Cette plate-forme devrait théoriquement pouvoir disposer de tous les outils permettant le développement souple et efficace d’applications médicales intégrant des processus métiers. Il est ainsi attendu de ce système qu’il permette le développement d’applications, pour la prise en charges des patients dyslexiques, conduisant à un diagnostic plus rapide et plus précis ainsi qu’à une remédiation plus adaptée et plus efficace. De notre effort d’innovation ressortent des perspectives encourageantes. Cependant, ce type d’initiative ne peut se concrétiser qu’autour de collaborations pluridisciplinaires disposant de nombreux moyens fonctionnels, techniques et financiers. La constitution d’un tel consortium semble donc être la prochaine étape nécessaire à l’obtention des financements pour réaliser un premier prototype fonctionnel de la PAMMA, ainsi que de premières applications. Des études cliniques pourront être alors menées pour prouver indubitablement l’efficacité d’une telle approche dans le cadre de la prise en charge de la dyslexie, ainsi qu’éventuellement d’autres troubles neuropsychologiques

Gaze-based Interaction for Virtual Environments

Abstract We present an alternative interface that allows users to perceive new sensations in virtual environments. Gaze-based interaction in virtual environments creates the feeling of controlling objects with the mind, arguably translating into a more intense immersion sensation. Additionally, it is also free of some of the most cumbersome aspects of interacting in virtual worlds. By incorporating a real-time physics engine, the sensation of moving something real is further accentuated. We also describe various simple yet effective techniques that allow eyetracking devices to enhance the three-dimensional visualization capabilities of current displays. Some of these techniques have the additional advantage of freeing the mouse from most navigation tasks. This work focuses on the study of existing techniques, a detailed description of the implemented interface and the evaluation (both objective and subjective) of the interface. Given that appropriate filtering of the data from the eye tracker used is a key aspect for the correct functioning of the interface, we will also discuss that aspect in depth