Potential of consumer EEG for real-time interactions in immersive VR

Abstract. Virtual reality is an active research subject and has received a lot of attention over the last few years. We have seen multiple commercial VR devices, each improving upon the last iteration become available to the wider public. In addition, interest in brain-computer interface (BCI) devices has increased rapidly. As these devices are becoming more affordable and easy to use, we are presented with more accessible options to measure brain activity. In this study, our aim is to combine these two technologies to enhance the interaction within a virtual environment. In this study we sought to facilitate interaction in VR by using EEG signals. The EEG signals were used to estimate the volume of focus. By applying this concept with VR, we designed two use cases for further exploration. The methods of interactions explored in the study were telekinesis and teleportation. Telekinesis seemed an applicable option for this study since it allows the utilization of the EEG while maintaining a captivating and engaging user experience. With teleportation, the goal was the exploration of different options for locomotion in VR. To test our solution, we built a test environment by using Unity engine. We also invited several participants to gain feedback on the usability and accuracy of our methodology. For evaluation, 13 study participants were divided into two different groups. The other group tested our actual solution for the estimation of the focus. However, the other group used randomized values for the same purpose. Some key differences between the test groups were identified. We were able to create a working prototype where the users could interact with the environment by using their EEG signals. With some improvements, this could be expanded to a more refined solution with a better user experience. There is a lot of potential in combining the use of human brain signals with virtual environments to both enrich the interaction and increase the immersion of virtual reality.Kuluttaja-EEG laitteiden potentiaali reaaliaikaiseen vuorovaikutukseen immersiivisessä virtuaalitodellisuudessa. Tiivistelmä. Virtuaalitodellisuus (VR) on aktiivisen tutkimuksen kohde ja varsinkin viime vuosina herättänyt paljon huomiota. VR-laseissa on tapahtunut huomattavaa kehitystä ja niitä on saatavilla yhä laajemmalle käyttäjäkunnalle. Lisäksi kiinnostus aivo-tietokone -rajapintoihin (BCI) on kiihtymässä. Koska aivokäyrää mittaavat laitteet ovat yhä edullisempia ja kehittymässä helppokäyttöisemmiksi, monia uusia menetelmiä aivosignaalin mittamiseksi on saatavilla. Tässä työssä tavoitteemme oli yhdistää nämä kaksi teknologiaa parantaaksemme vuorovaikutusta virtuaalitodellisuudessa. Tässä tutkimuksessa käytimme aivosähkökäyrää VR-käyttäjäkokemuksen kehittämiseksi. Tätä tekniikkaa hyödyntäen arvioimme käyttäjän keskittymistä. Tutkimusta varten valitsimme kaksi vuorovaikutustapaa. Nämä tutkittavat tavat ovat telekinesia sekä teleportaatio. Telekinesia on mielenkiintoinen tapa hyödyntää aivosähkökäyrää luoden samalla mukaansatempaavan käyttäjäkokemuksen. Teleportaation päämääränä oli löytää uudenlaisia liikkumistapoja VR:ssä. Tutkimustamme varten, suunnittelimme testiympäristön Unity-pelimoottorilla. Kokosimme joukon testaajia, joiden avulla arvioimme työmme käyttökelpoisuutta sekä tarkkuutta. Saadaksemme luotettavampia testituloksia, jaoimme 13 testaajaa kahteen eri ryhmään. Toinen ryhmistä testasi varsinaista toteutustamme ja toinen ryhmä käytti satunnaistettuja keskittymisarvoja. Löysimme ratkaisevia eroja näiden kahden testiryhmän välillä. Onnistuimme kehittämään toimivan prototyypin, jossa käyttäjät kykenivät interaktioon virtuaaliympäristössä hyödyntäen aivosähkökäyrää. Jatkokehitystä tekemällä käyttäjäkokemusta olisi mahdollista parantaa entisestään. Integraatio aivosensoreiden ja virtuaalitodellisuuden välillä huokuu potentiaalia ja tarjoaa mahdollisuuksia tehdä virtuaalimaailmasta yhä immersiivisemmän

Anxiety reducing through a neurofeedback serious game with dynamic difficulty adjustment

Presently, society has to deal with a large number of mental issues. Anxiety disorder is a serious concern, affecting millions of people’s lives and, although methods to tackle the problem currently exist, these main treatments are being linked to some issues and improvements must be found. One of the alternatives is Neurofeedback, a biofeedback treatment, completely non-invasive and showing impressive results so far. It uses a neuroheadset equipment to read the neural activity of the brain, giving the user visual feedback about it. The purpose this, is to train the users’ brain in specific regions and frequencies, allowing the subjects to learn how to voluntarily control its neural activity, even outside of the session. Current applications using this method might be too simple, which can become tedious and disengaging. Serious games can help with these issues, since it can bring enjoyment and engagement while doing this type of treatment. The interest in games’ capabilities in education has been increasing over the past years, since it has been proved that games are an excellent tool for education and skill learning. Joining these concepts of game and neurofeedback, this project aims to create a serious game prototype, applying the current treatment knowledge. The development process of a new game with neuroheadset integration, capable of reading the neural activity of the user while playing and giving the appropriate feedback, will be described in the present document. Since studies proved that a good balance between challenge and skill increases the learning performance, a dynamic difficulty adjustment system is implemented within the game, allowing the game to adapt itself to each user’s skill individually, and keeping the user in a challenging, motivating zone. At the end of the document, the results of pilot test on a few subjects are shown.Na sociedade actual o número de problemas relacionados com perturbações mentais tem sido cada vez mais relevante, sendo esse o caso da ansiedade. O distúrbio de ansiedade é um problema que atinge milhões de pessoas e, embora existam métodos para combater este problema, estudos comprovam que estes têm algumas lacunas que podem trazer outros problemas associados, sendo portanto necessário procurar melhorias aos métodos actuais. Uma das alternativas tem apresentado excelentes resultados e denomina-se Neurofeedback. Este é um tratamento de biofeedback, nãoinvasivo e que utiliza um equipamento neuroheadset para capturar a actividade neuronal, apresentando indicações visuais sobre o comportamento do utilizador. Isto é feito com o objectivo de treinar o cérebro do utilizador, em regiões e frequências específicas, para que este seja capaz de controlar voluntariamente a sua actividade neuronal. As aplicações actualmente utilizadas com este intuito podem se tornar aborrecidas e monótonas devido à sua simplicidade. Um jogo sério pode ajudar com estes problemas, uma vez que é capaz de trazer divertimento e motivação para este tipo de tratamento. O crescente interesse nas capacidades educativas dos jogos sérios, tem identificado estes como excelentes ferramentas para a educação. Este projecto pretende portanto criar um protótipo de um jogo sério, aplicando os conceitos de neurofeedback. Neste documento, é apresentado o processo de desenvolvimento de um novo jogo com integração de um neuroheadset, capaz de identificar a actividade neuronal do jogador dando respostas adequadas. Uma vez que estudos comprovam que um bom balanço entre desafio apresentado e técnica do utilizador aumenta a capacidade de aprendizagem, foi implementado também um sistema de ajuste de dificuldade dinâmica, permitindo uma adaptação do jogo a cada indivíduo e mantendo este numa zona motivante de equilíbrio entre desafio e proficiência. No final serão apresentados os resultados de um teste piloto efectuado em alguns indivíduos

Exploiting physiological changes during the flow experience for assessing virtual-reality game design.

Immersive experiences are considered the principal attraction of video games. Achieving a healthy balance between the game's demands and the user's skills is a particularly challenging goal. However, it is a coveted outcome, as it gives rise to the flow experience – a mental state of deep concentration and game engagement. When this balance fractures, the player may experience considerable disinclination to continue playing, which may be a product of anxiety or boredom. Thus, being able to predict manifestations of these psychological states in video game players is essential for understanding player motivation and designing better games. To this end, we build on earlier work to evaluate flow dynamics from a physiological perspective using a custom video game. Although advancements in this area are growing, there has been little consideration given to the interpersonal characteristics that may influence the expression of the flow experience. In this thesis, two angles are introduced that remain poorly understood. First, the investigation is contextualized in the virtual reality domain, a technology that putatively amplifies affective experiences, yet is still insufficiently addressed in the flow literature. Second, a novel analysis setup is proposed, whereby the recorded physiological responses and psychometric self-ratings are combined to assess the effectiveness of our game's design in a series of experiments. The analysis workflow employed heart rate and eye blink variability, and electroencephalography (EEG) as objective assessment measures of the game's impact, and self-reports as subjective assessment measures. These inputs were submitted to a clustering method, cross-referencing the membership of the observations with self-report ratings of the players they originated from. Next, this information was used to effectively inform specialized decoders of the flow state from the physiological responses. This approach successfully enabled classifiers to operate at high accuracy rates in all our studies. Furthermore, we addressed the compression of medium-resolution EEG sensors to a minimal set required to decode flow. Overall, our findings suggest that the approaches employed in this thesis have wide applicability and potential for improving game designing practices

Models and Analysis of Vocal Emissions for Biomedical Applications

The Models and Analysis of Vocal Emissions with Biomedical Applications (MAVEBA) workshop came into being in 1999 from the particularly felt need of sharing know-how, objectives and results between areas that until then seemed quite distinct such as bioengineering, medicine and singing. MAVEBA deals with all aspects concerning the study of the human voice with applications ranging from the neonate to the adult and elderly. Over the years the initial issues have grown and spread also in other aspects of research such as occupational voice disorders, neurology, rehabilitation, image and video analysis. MAVEBA takes place every two years always in Firenze, Italy

Low-Cost Sensors and Biological Signals

Many sensors are currently available at prices lower than USD 100 and cover a wide range of biological signals: motion, muscle activity, heart rate, etc. Such low-cost sensors have metrological features allowing them to be used in everyday life and clinical applications, where gold-standard material is both too expensive and time-consuming to be used. The selected papers present current applications of low-cost sensors in domains such as physiotherapy, rehabilitation, and affective technologies. The results cover various aspects of low-cost sensor technology from hardware design to software optimization

Wireless Sensors for Brain Activity—A Survey

Over the last decade, the area of electroencephalography (EEG) witnessed a progressive move from high-end large measurement devices, relying on accurate construction and providing high sensitivity, to miniature hardware, more specifically wireless wearable EEG devices. While accurate, traditional EEG systems need a complex structure and long periods of application time, unwittingly causing discomfort and distress on the users. Given their size and price, aside from their lower sensitivity and narrower spectrum band(s), wearable EEG devices may be used regularly by individuals for continuous collection of user data from non-medical environments. This allows their usage for diverse, nontraditional, non-medical applications, including cognition, BCI, education, and gaming. Given the reduced need for standardization or accuracy, the area remains a rather incipient one, mostly driven by the emergence of new devices that represent the critical link of the innovation chain. In this context, the aim of this study is to provide a holistic assessment of the consumer-grade EEG devices for cognition, BCI, education, and gaming, based on the existing products, the success of their underlying technologies, as benchmarked by the undertaken studies, and their integration with current applications across the four areas. Beyond establishing a reference point, this review also provides the critical and necessary systematic guidance for non-medical EEG research and development efforts at the start of their investigation.</jats:p

Wireless Sensors for Brain Activity — A Survey

Over the last decade, the area of electroencephalography (EEG) witnessed a progressive move from high-end large measurement devices, relying on accurate construction and providing high sensitivity, to miniature hardware, more specifically wireless wearable EEG devices. While accurate, traditional EEG systems need a complex structure and long periods of application time, unwittingly causing discomfort and distress on the users. Given their size and price, aside from their lower sensitivity and narrower spectrum band(s), wearable EEG devices may be used regularly by individuals for continuous collection of user data from non-medical environments. This allows their usage for diverse, nontraditional, non-medical applications, including cognition, BCI, education, and gaming. Given the reduced need for standardization or accuracy, the area remains a rather incipient one, mostly driven by the emergence of new devices that represent the critical link of the innovation chain. In this context, the aim of this study is to provide a holistic assessment of the consumer-grade EEG devices for cognition, BCI, education, and gaming, based on the existing products, the success of their underlying technologies, as benchmarked by the undertaken studies, and their integration with current applications across the four areas. Beyond establishing a reference point, this review also provides the critical and necessary systematic guidance for non-medical EEG research and development efforts at the start of their investigation

Proceedings of the 7th international conference on disability, virtual reality and associated technologies, with ArtAbilitation (ICDVRAT 2008)

The proceedings of the conferenc