BCI-Based Navigation in Virtual and Real Environments

A Brain-Computer Interface (BCI) is a system that enables people to control an external device with their brain activity, without the need of any muscular activity. Researchers in the BCI field aim to develop applications to improve the quality of life of severely disabled patients, for whom a BCI can be a useful channel for interaction with their environment. Some of these systems are intended to control a mobile device (e. g. a wheelchair). Virtual Reality is a powerful tool that can provide the subjects with an opportunity to train and to test different applications in a safe environment. This technical review will focus on systems aimed at navigation, both in virtual and real environments.This work was partially supported by the Innovation, Science and Enterprise Council of the Junta de Andalucía (Spain), project P07-TIC-03310, the Spanish Ministry of Science and Innovation, project TEC 2011-26395 and by the European fund ERDF

Enhancing [Spatial] Creativity: Enhancing creativity of architects by applying unconventional virtual environments

Potentials of a virtual environment for enhancing the creativity of architects have shaped this research. There is no singular definition of creativity. In fact, there are more than 100 different definitions of creativity according to different contexts and disciplines. Nevertheless, it is possible to confine the boundaries of definitions and address creativity within a confined framework.  The first practical step was thus to perform an in-depth literature survey to define a boundary condition for the widespread topic of “creativity†and identify vital research questions pertaining to creativity. In this regard, personality and behaviour of creative people; mood, state, temper, intelligence vs. creativity, motivation and so forth were ignored. Instead, cognitive aspects of creativity such as thinking patterns, conceptual blending, idea expansion and tolerance of ambiguity have been focused upon. The second step was to test whether starting a design procedure with a 2 or a 3-dimensional mode of thinking has any correlation with creativity. An experiment pertaining to this test was designed in which participants were asked to perform the same design task once with 2D tools and environments (e.g. traditional pen and paper) and the next time by applying 3D tools and environments (e.g. 3D software). A jury of experts in the field of design subjectively compared the results and arrived at a conclusion that participates generated more creative ideas by implementing 3D environments/tools.  The third step involved the introduction of unconventional virtual environments (UVEs), which subsequently lead to the creation of a hypothesis. This hypothesis tries to connect navigation in UVEs with the enhancement of creativity.  Characteristics of UVEs and theoretical arguments around the hypothesis were also discussed.  The fourth step involved a discussion on two effective parameters of creativity: Tolerance of ambiguity Conceptual blending Attempts to verify these parameters, lead to the formulation of two separate experiments. The conclusions of these experiments were as follows: Tolerance of ambiguity has a direct relationship with creativity. Architects deal with multiple parameters during a typical design process. The ability to meaningfully process the relationships between such a multitude of parameters has also trained architects to possess a higher level of tolerance of ambiguity aiding them in postponing making hasty judgements. By combining different ideas, one can formulate newer and much novel ideas. Ideas tend to become creative when more remote ideas are combined and synthesised. In order to prevent repetitive and self-similar ideas, the unconscious mind needs to be fed with more ideas, which the mind has not been exposed to. These new ideas/experiences can be generated by exposing one’s self to UVEs since UVEs can expose one to situations and experiences which the brain cannot experience in the physical world. The fifth step aims at understanding how the brain perceives different environments. Three different environments were chosen for an experimental study pertaining to the same: 1- Abstract environment 2- Semi-designed environment and 3- Fully designed environment. Participants were asked to provide a feedback by answering a questionnaire after navigating each of these environments. Simultaneously, their brain activity patterns were recorded via a professional neuropsychology apparatus. After analysing the brain activities, in conjunction with the questionnaire, it became clear that perception of an abstract environment is completely different from the perception of a Semi-designed or Fully designed environment.  This experiment consolidated the hypothesis that UVEs as abstract designed environments activate parts of the brain that are correlated with creativity. The sixth step involved putting the hypothesis to a final test. A conclusive experiment was thus designed. The experiment won the Visionair fund of FP7 and was conducted in Italy. A UVE was designed using the software 3D max and was converted to the 3D stereoscopic mode using a specific software: GIOVE; developed in ITIA-CNR, Italy. Using a 3D goggle and 6 axis mouse, a group of 20 participants were asked to navigate the UVE. They were asked to provide a written feedback pertaining to their feelings, expectations, the strategy of navigation and in general, their experience. Their responses were collected and analysed. The final step involved answering the formulated research questions and discussing the final results

Enhancing [Spatial] Creativity – Enhancing creativity of architects by applying unconventional virtual environments (UVEs)

Potentials of virtual environment for enhancing creativity of architects have shaped this research. There is no singular definition of creativity. In fact, there are more than 100 different definitions for creativity according to different contexts and disciplines. Nevertheless, it is possible to confine the boundaries of definitions and address creativity within a confined framework. The first practical step was thus to perform an in-depth literature survey to define a boundary condition for the widespread topic of “creativity” and identify vital research questions pertaining to creativity. In this regard, personality and behavior of creative people; mood, state, temper, intelligence vs. creativity, motivation and so forth were ignored. Instead, cognitive aspects of creativity such as thinking patterns, conceptual blending, idea expansion and tolerance of ambiguity have been focused upon.The second step was to test whether starting a design procedure with a 2 or a 3-dimensional mode of thinking has any correlation with creativity. An experiment pertaining to this test was designed in which participants were asked to perform the same design task once with 2D tools and environments (e.g. traditional pen and paper) and the next time by applying 3D tools and environments (e.g. 3D software). A jury of experts in the field of design subjectively compared the results and arrived at a conclusion that participates generated more creative ideas by implementing 3D environments/tools. The third step, involved the introduction of unconventional virtual environments (UVEs), which subsequently lead to the creation of a hypothesis. This hypothesis tries to connect navigation in UVEs with the enhancement of creativity.  Characteristics of UVEs and theoretical arguments around the hypothesis were also discussed. The fourth step, involved a discussion on two effective parameters of creativity: 1- Tolerance of ambiguity2- Conceptual blendingAttempts to verify these parameters, lead to the formulation of two separate experiments. The conclusions of these experiments were as follows:1-  Tolerance of ambiguity has a direct relationship with creativity. Architects deal with multiple parameters during a typical design process. The ability to meaningfully process the relationships between such a multitude of parameters has also trained architects to possess a higher level of tolerance of ambiguity aiding them in postponing making hasty judgements.2-  By combining different ideas, one can formulate newer and much novel ideas. Ideas tend to become creative when more remote ideas are combined and synthesized. In order to prevent repetitive and self-similar ideas, the unconscious mind needs to be fed with more ideas, which the mind has not been exposed to. These new ideas/experiences can be generated by exposing one’s self to UVEs, since UVEs can expose one to situations and experiences which the brain cannot experience in the physical world.The fifth step, aims at understanding how the brain perceives different environments. Three different environments were chosen for an experimental study pertaining to the same: 1- Abstract environment 2- Semi-designed environment and 3- Fully designed environment. Participants were asked to provide a feedback by answering a questionnaire after navigating each of these environments. Simultaneously, their brain activity patterns were recorded via a professional neuropsychology apparatus. After analyzing the brain activities, in conjunction with the questionnaire, it became clear that perception of an abstract environment is completely different from the perception of a Semi-designed or Fully designed environment.  This experiment consolidated the hypothesis that UVEs as abstract designed environments activate parts of the brain that are correlate with creativity.The sixth step, involved putting the hypothesis to a final test. A conclusive experiment was thus designed. The experiment won the Visionair fund of FP7 and was conducted in Italy. A UVE was designed using the software 3D max and was converted to 3D stereoscopic mode using a specific software: GIOVE; developed in ITIA-CNR, Italy. Using a 3D goggle and 6 axis mouse, a group of 20 participants were asked to navigate the UVE. They were asked to provide a written feedback pertaining to their feelings, expectations, strategy of navigation and in general, their experience. Their responses were collected and analyzed.The final step, involved answering the formulated research questions and discussing the final results

Sample UVE

By applying virtual environments in design processes, architects can expect a variety of solutions, as compared to conventional methods despite the problems with visual perception or mental workload or frequency of iteration between creation and modification (Schnabel & Kvan, 2003). Designers use different tools to communicate and express their thoughts. Although some architects visualize their design decisions through large-scale models, modelling is always limited by the overall dimensions, scale, resource constraints and material restrictions. To overcome these constraints, architects implement virtual environments (Bertol & Foell, 1997). Although virtual environments can be easily found everywhere, immersive virtual environments (IVE) are typically found in research-laboratories or universities. Recent progression in technology provided the ground for IVEs to be implemented in the consumer electronics sector such as the gaming industry (Leach, 2002). The story is the same in the discipline of architecture and opportunities for applying VEs in architectural design are still developing (Stuart, 2001). VEs are employed successfully to study, communicate, collaborate, and present architectural designs but are rarely used for the actual act of creation and form-finding in the field of architecture (Maze, 2002). In this chapter another application of the IVEs is proposed: to create an immersive UVE. This application is not developed for the purpose of visualization or form finding, but specifically for enhancing creativity

A brain-computer interface for navigation in virtual reality

L'interface cerveau-ordinateur (ICO) décode les signaux électriques du cerveau requise par l’électroencéphalographie et transforme ces signaux en commande pour contrôler un appareil ou un logiciel. Un nombre limité de tâches mentales ont été détectés et classifier par différents groupes de recherche. D’autres types de contrôle, par exemple l’exécution d'un mouvement du pied, réel ou imaginaire, peut modifier les ondes cérébrales du cortex moteur. Nous avons utilisé un ICO pour déterminer si nous pouvions faire une classification entre la navigation de type marche avant et arrière, en temps réel et en temps différé, en utilisant différentes méthodes. Dix personnes en bonne santé ont participé à l’expérience sur les ICO dans un tunnel virtuel. L’expérience fut a était divisé en deux séances (48 min chaque). Chaque séance comprenait 320 essais. On a demandé au sujets d’imaginer un déplacement avant ou arrière dans le tunnel virtuel de façon aléatoire d’après une commande écrite sur l'écran. Les essais ont été menés avec feedback. Trois électrodes ont été montées sur le scalp, vis-à-vis du cortex moteur. Durant la 1re séance, la classification des deux taches (navigation avant et arrière) a été réalisée par les méthodes de puissance de bande, de représentation temporel-fréquence, des modèles autorégressifs et des rapports d’asymétrie du rythme β avec classificateurs d’analyse discriminante linéaire et SVM. Les seuils ont été calculés en temps différé pour former des signaux de contrôle qui ont été utilisés en temps réel durant la 2e séance afin d’initier, par les ondes cérébrales de l'utilisateur, le déplacement du tunnel virtuel dans le sens demandé. Après 96 min d'entrainement, la méthode « online biofeedback » de la puissance de bande a atteint une précision de classification moyenne de 76 %, et la classification en temps différé avec les rapports d’asymétrie et puissance de bande, a atteint une précision de classification d’environ 80 %.A Brain-Computer Interface (BCI) decodes the brain signals representing a desire to do something, and transforms those signals into a control command. However, only a limited number of mental tasks have been previously detected and classified. Performing a real or imaginary navigation movement can similarly change the brainwaves over the motor cortex. We used an ERS-BCI to see if we can classify between movements in forward and backward direction offline and then online using different methods. Ten healthy people participated in BCI experiments comprised two-sessions (48 min each) in a virtual environment tunnel. Each session consisted of 320 trials where subjects were asked to imagine themselves moving in the tunnel in a forward or backward motion after a randomly presented (forward versus backward) command on the screen. Three EEG electrodes were mounted bilaterally on the scalp over the motor cortex. Trials were conducted with feedback. In session 1, Band Power method, Time-frequency representation, Autoregressive models and asymmetry ratio were used in the β rhythm range with a Linear-Discriminant-analysis classifier and a Support Vector Machine classifier to discriminate between the two mental tasks. Thresholds for both tasks were computed offline and then used to form control signals that were used online in session 2 to trigger the virtual tunnel to move in the direction requested by the user's brain signals. After 96 min of training, the online band-power biofeedback training achieved an average classification precision of 76 %, whereas the offline classification with asymmetrical ratio and band-power achieved an average classification precision of 80%

Is it worthwhile going immersive? : evaluating the performance of virtual simulated stores for shopper research : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Marketing at Massey University, Albany, New Zealand

Listed in 2020 Dean's List of Exceptional ThesesAdvances in simulation technology offer the possibility of more authentic shopper environments for virtual store experiments. Criticisms of subjective measures of consumer behavior previously led to the use of test markets or simulated stores for consumer experimental research. As cost implications made such experiments unavailable to the wider market research community, virtual simulated stores (VSSs) were developed as an alternative. However, the adoption of VSSs has been slow as traditional desktop-operated VSSs do not provide an authentic multicategory shopper experience. New simulation technologies offer the opportunity for more immersive and authentic VSS environments. Yet there has been little research on how authenticity of VSSs is impacted by newly available technology such as head-mounted displays, motion tracking, force feedback controllers, and application of place and plausibility cues. Thus, this dissertation asks whether immersive technologies have potential to provide highly authentic VSS environments. Of the many factors that may determine authenticity, this dissertation examines three; participants’ sense of telepresence, the realism of shopper behaviour, and the effects of shopper locomotion alternatives. An immersive VSS incorporating new virtual technologies was specifically designed and built for this research. Three studies were undertaken. The first compared perceived telepresence and usability between a desktop-operated VSS and an equivalent immersive walk-around VSS. The second examined the authenticity of shopper behaviour in the immersive walk-around VSS by comparing observed shopping patterns to those previously reported in the marketing literature. The third tested whether walk-around locomotion was necessary for authenticity, or whether a simpler teleportation method would result in equivalent shopper behaviour and emotions. Results showed that immersive VSS systems are preferable to traditional desktop-operated systems with regards to telepresence and usability. Further, authentic behavioural patterns can be found in immersive walk-around store experiments, including plausibility of private label shares, pack inspection times, shelf-height effects and impulse purchases. Lastly, there were no differences in shopper emotions and purchase behaviour between walk-around locomotion and controller-based instant teleportation, implying that the teleportation technique can be used, thereby reducing the required physical footprint for immersive VSS simulations. Collectively, the findings imply that marketers who study in-store shopper behavior can be confident using immersive VSS for their research as opposed to outdated desktop VSS technology

Development of a neurofeedback-based virtual reality environment

Recent technology has continuously expanded the reaching spectre of psychotherapy. In the latest years, the development of digital environments, coupled with the evolution of sensorial hardware, has demonstrated usefulness and effectiveness in some areas of psychotherapy such as phobia treatment and attention deficit hyperactivity disorder management through neurofeedback training. However, the generality of these equipments is very expensive. In this project, an audiovisual stimuli virtual reality environment was developed, capable of displaying signals provided by an electroencephalography-based brain-computer interface. This environment has the objective of providing its user with neurofeedback training and being suited for affordable hardware equipments. Development of the aforementioned environment took place in the Unity3D ® game engine version 5.3.0f4, using C# scripting developed in Microsoft ® Visual Studio 2015 TM. As for the virtual reality display, an Oculus Rift ® development kit 1 was used for testing, together with the Oculus runtime for Windows ®, version 0.8.0.0. The used brain-computer interface was Neurosky’s Mindband TM, a research tool with a single electroencephalography channel, mediated through the ThinkGear Connector, version 3.1.8.0. The creation of this environment as an application directed towards neurofeedback training and compatible with affordable equipments is a contribution towards a reality where virtual reality is more synchronized with our society