Eliciting steady-state visual evoked potentials by means of stereoscopic displays

Brain-Computer Interfaces (BCIs) provide users communication and control capabilities by analyzing their brain activity. A technique to implement BCIs, used recently also in Virtual Reality (VR) environments, is based on the Steady State Visual Evoked Potentials (SSVEPs) detection. Exploiting the SSVEP response, BCIs could be implemented showing targets flickering at different frequencies and detecting which is gazed by the observer analyzing her/his electroencephalographic (EEG) signals. In this work, we evaluate the use of stereoscopic displays for the presentation of SSVEP eliciting stimuli, comparing their effectiveness between monoscopic and stereoscopic stimuli. Moreover we propose a novel method to elicit SSVEP responses exploiting the stereoscopic displays capability of presenting dichoptic stimuli. We have created an experimental scene to present flickering stimuli on an active stereoscopic display, obtaining reliable control of the targets' frequency independently for the two stereo views. Using an EEG acquisition device, we analyzed the SSVEP responses from a group of subjects. From the preliminary results, we got evidence that stereoscopic displays represent valid devices for the presentation of SSVEP stimuli. Moreover, the use of different flickering frequencies for the two views of a single stimulus proved to elicit non-linear interactions between the stimulation frequencies, clearly visible in the EEG signal. This suggests interesting applications for SSVEP-based BCIs in VR environments able to overcome some limitations imposed by the refresh frequency of standard displays, but also the use of commodity stereoscopic displays to implement binocular rivalry experiments

The Berlin Brain-Computer Interface: Progress Beyond Communication and Control

The combined effect of fundamental results about neurocognitive processes and advancements in decoding mental states from ongoing brain signals has brought forth a whole range of potential neurotechnological applications. In this article, we review our developments in this area and put them into perspective. These examples cover a wide range of maturity levels with respect to their applicability. While we assume we are still a long way away from integrating Brain-Computer Interface (BCI) technology in general interaction with computers, or from implementing neurotechnological measures in safety-critical workplaces, results have already now been obtained involving a BCI as research tool. In this article, we discuss the reasons why, in some of the prospective application domains, considerable effort is still required to make the systems ready to deal with the full complexity of the real world.EC/FP7/611570/EU/Symbiotic Mind Computer Interaction for Information Seeking/MindSeeEC/FP7/625991/EU/Hyperscanning 2.0 Analyses of Multimodal Neuroimaging Data: Concept, Methods and Applications/HYPERSCANNING 2.0DFG, 103586207, GRK 1589: Verarbeitung sensorischer Informationen in neuronalen Systeme

The role of neural oscillations in the visual system and their relation to conscious perception

Neural oscillations are intrinsically linked with attention, vigilance and featural sensitivity and therefore often associated with visual perception. However, the neural oscillation literature remains conflicted on several issues. Here, I describe four experiments investigating these conflicts using a variety of experimental and analysis techniques. We first explored the relationship between the inhibitory neurotransmitter GABA and gamma frequency oscillations in the rodent visual cortex. We found no evidence that synaptic and extrasynaptic GABA concentration altered gamma oscillations, suggesting that GABAergic inhibition cannot be linked directly to GABA concentration and instead depend on postsynaptic receptor kinetics. The second chapter examined how spontaneous alpha activity related to performance in an orientation discrimination task. Alpha amplitude was a significant predictor of reaction time but not task accuracy. The results suggested that alpha can modulate visual perception through top-down mechanisms. Interestingly, we also found that the relationship between alpha activity and task accuracy was determined by the subject’s task expertise. The third chapter examined how exogenous rhythms (generated by chromatic gratings) within visual cortex may interact with ongoing endogenous oscillations. Univariate analysis of single EEG channels revealed significantly higher endogenous power during chromatic than achromatic stimulation. An additional multivariate classifier showed distinct patterns of activity at very high frequencies, suggesting phase coupling between exogenous and endogenous signals. This finding was extended in the final chapter, which examined the neural correlates of rapid chromatic stimulation. Robust BOLD responses were found even when stimuli flickered above the consciously perceptible frequency, indicating that the temporal filtering stage limiting perception is later than V4. Additionally, chromatic preference in ‘colour area’ V4 was strongly dependent on stimulus frequency

Mechanisms of visual feature binding

Visual feature binding is the method by which coherent objects and scenes are perceived. Advances in the science of perception have indicated that visual features such as colour, motion, and orientation are to some extent, processed separately in primate early visual cortex. However, the mechanism by which these features are integrated remains unclear. Phenomenologically, the process of binding features to form objects appears to be an efficient and automatic process. Some research also shows a high temporal resolution for binding features together, in addition to populations of neurons that jointly code for features. However, dominant theories of feature binding and the majority of the binding literature indicate that the feature binding process is severely limited by a relatively low temporal resolution, especially when compared to other perceptual properties such as feature detection. To identify and resolve the discrepancy in the feature binding literature, I investigate the feature binding process and its inter-relationship with perceptual surface segregation. Surface segregation has been postulated as the method by which features can be rapidly bound together, giving them impression of a high temporal resolution. In Chapter 2, displays are used that alternate between two arrays of differently coloured, oppositely moving dots. The alternation frequency is modified in order to gauge the temporal resolution of binding. This is combined with surface segregation cues such as coherent motion, consistency of dot configuration, and colour. In Chapter 3, coloured, oriented gratings are used to investigate colour-orientation binding. Angular separation, spatial and temporal coincidence, and stimulus presentation duration are varied. Across these experiments, a number of these surface segregation cues are manipulated in order to measure the corresponding effects on feature binding, perceptual interpretation of the stimulus, and its neural representation. The results of the psychophysical experiments indicate that feature binding, surface segregation, and temporal integration are inextricably linked. These findings are reinforced by data gathered through functional magnetic resonance imaging (fMRI) of human subjects. Both surface segregation and feature pairs were found to modulate neural activity in early visual cortex, providing evidence that similar neural substrates are recruited for both feature binding and surface segregation. Overall, the two complementary sets of experiments using stimulus conjunctions of colour-motion and colour-orientation stimuli provide converging evidence and insight into the dynamics of the underlying binding mechanisms. A discussion of the implications of the research follows, concluding that rapidly formed surface representations can be maintained across presentation intervals by temporal integration. Attentional selection of one feature (e.g. orientation) can then be used to boost the response to the paired feature (colour) in order to identify and extract the correct feature pairing. Based on the known properties of the visual system, several potential neural mechanisms are proposed that are consistent with both the psychophysical and neural data, in addition to suggested future directions for the study of visual feature binding

Perceptual models for high-refresh-rate rendering

Rendering realistic images requires substantial computational power. With new high-refresh-rate displays as well as the renaissance of virtual reality (VR) and augmented reality (AR), one cannot expect that GPU performance will scale fast enough to meet the requirements of immersive photo-realistic rendering with current rendering techniques. In this dissertation, I follow the dual of the well-known computer vision approach: vision is inverse graphics: to improve graphical algorithms, I consider the operation of the human visual system. I propose to model and exploit the limitations of the visual system in the context of novel high-refresh-rate displays; specifically, I focus on spatio-temporal perception, a topic that has received remarkably less attention than spatial-only perception so far. I present three main contributions. First, I demonstrate the validity of the perceptual approach by presenting a conceptually simple rendering technique motivated by our eyes' limited sensitivity to high spatio-temporal change which reduces the rendering load and transmission requirement of current-generation VR headsets without introducing perceivable visual artefacts. Second, I present two visual models related to motion perception: (a) a metric for detecting flicker; and (b) a comprehensive visual model to predict perceived motion quality on monitors with arbitrary refresh rates and monitor resolutions. Third, I propose an adaptive rendering algorithm that utilises the proposed models. All algorithms operate on physical colorimetric units (instead of display-referenced pixel values), for which I provide the appropriate display measurements and models. All proposed algorithms and visual models are calibrated and validated with psychophysical experiments

TOWARDS STEADY-STATE VISUALLY EVOKED POTENTIALS BRAIN-COMPUTER INTERFACES FOR VIRTUAL REALITY ENVIRONMENTS EXPLICIT AND IMPLICIT INTERACTION

In the last two decades, Brain-Computer Interfaces (BCIs) have been investigated mainly for the purpose of implementing assistive technologies able to provide new channels for communication and control for people with severe disabilities. Nevertheless, more recently, thanks to technical and scientific advances in the different research fields involved, BCIs are gaining greater attention also for their adoption by healthy users, as new interaction devices. This thesis is dedicated to to the latter goal and in particular will deal with BCIs based on the Steady State Visual Evoked Potential (SSVEP), which in previous works demonstrated to be one of the most flexible and reliable approaches. SSVEP based BCIs could find applications in different contexts, but one which is particularly interesting for healthy users, is their adoption as new interaction devices for Virtual Reality (VR) environments and Computer Games. Although being investigated since several years, BCIs still poses several limitations in terms of speed, reliability and usability with respect to ordinary interaction devices. Despite of this, they may provide additional, more direct and intuitive, explicit interaction modalities, as well as implicit interaction modalities otherwise impossible with ordinary devices. This thesis, after a comprehensive review of the different research fields being the basis of a BCI exploiting the SSVEP modality, present a state-of-the-art open source implementation using a mix of pre-existing and custom software tools. The proposed implementation, mainly aimed to the interaction with VR environments and Computer Games, has then been used to perform several experiments which are hereby described as well. Initially performed experiments aim to stress the validity of the provided implementation, as well as to show its usability with a commodity bio-signal acquisition device, orders of magnitude less expensive than commonly used ones, representing a step forward in the direction of practical BCIs for end users applications. The proposed implementation, thanks to its flexibility, is used also to perform novel experiments aimed to investigate the exploitation of stereoscopic displays to overcome a known limitation of ordinary displays in the context of SSVEP based BCIs. Eventually, novel experiments are presented investigating the use of the SSVEP modality to provide also implicit interaction. In this context, a first proof of concept Passive BCI based on the SSVEP response is presented and demonstrated to provide information exploitable for prospective applications

Touch-sensitive : cybernetic images and replicant bodies in the post-industrial age

This thesis uses Deleuzian cybernetics to advance upon post-modern accounts of the contemporary image economy. It begins with the hypothesis that the schizophrenic behaviours of late capitalism have induced an irreparable crisis in the inherited `specular economy' (Irigaray). This is manifested as the breakdown of the laws of generalised equivalence between truth, value and meaning and the end of a stable signifier-signified relationship - theorised as the escape of reality into 'hyperreality', or the world become simulation according to Baudrillard. It will expose the insufficiency of post-modern accounts which theorise this crisis in representation via methods which fail to escape their own always already representational terms and it will then rigorously follow through the implications of an image economy which is constituted by simulations which are `genuinely' sourceless, which do not imitate a prior reality but which rather synthesise forces and relations. To escape the closed loop of representationalism, it will divert attention away from the signifier and will concentrate on the sub-representational power of images to re-engineer reality and to re-invent the limits of the body. Using the theory and practice of Deleuze, Spinoza, Bergson, Benjamin and Virilio, it will treat images as planes of corporeal becoming - as material entities, virtual avatars, possessional states and conductors of pre-personal affect. Post-modem accounts which cite the overwhelming predominance of images sit uncomfortably with the theories of French anti-ocularcentrism - accessed here via Irigaray and Lyotard - which mark the demise of vision and its attached representational order. This paradox requires that a new perceptual relation be mapped - figured here as entirely corporeal, as tactile and synesthetic (Mcluhan) and therefore immersive. Both 'affect' and 'intensity', as modes of pre-personal perception, will be treated as tactile interactions for these responses to images demand that a body be always 'in touch' with its environment, always anorganically altering its perceptual capacities by rules of feedback. It will be argued that in this reality studio, the body no longer perceives via a specular light source, solid form and assumed phallocentric meaning. The proposed synthesis between cybernetic imaging technologies, immanent perceptual criteria and the ever-changing state of the body requires an engagement with the female since she bears a privileged relation to this scenario. In the specular economy, women have been assumed, like faithful images, to secondarily reproduce an underlying, phallocentric truth. However, it will be shown that just as images can work nonrepresentationally, so too can female bodies; on the one hand appearing representational but on the other conducting radically subversive effects. Where bodies and images are such simulatory becomings it will be shown how the female is neither representationally ordered (social constructivism) nor essentially defined (biological reductivism) but is rather cybernetically engineered. Throughout, her privileged access to the virtual realm beyond language will be used to substantiate the major claim of this thesis that cybernetic simulation is more concerned with the material alteration of an environment rather than with the implementation of linguistic obligation

Blickpunktabhängige Computergraphik

Contemporary digital displays feature multi-million pixels at ever-increasing refresh rates. Reality, on the other hand, provides us with a view of the world that is continuous in space and time. The discrepancy between viewing the physical world and its sampled depiction on digital displays gives rise to perceptual quality degradations. By measuring or estimating where we look, gaze-contingent algorithms aim at exploiting the way we visually perceive to remedy visible artifacts. This dissertation presents a variety of novel gaze-contingent algorithms and respective perceptual studies. Chapter 4 and 5 present methods to boost perceived visual quality of conventional video footage when viewed on commodity monitors or projectors. In Chapter 6 a novel head-mounted display with real-time gaze tracking is described. The device enables a large variety of applications in the context of Virtual Reality and Augmented Reality. Using the gaze-tracking VR headset, a novel gaze-contingent render method is described in Chapter 7. The gaze-aware approach greatly reduces computational efforts for shading virtual worlds. The described methods and studies show that gaze-contingent algorithms are able to improve the quality of displayed images and videos or reduce the computational effort for image generation, while display quality perceived by the user does not change.Moderne digitale Bildschirme ermöglichen immer höhere Auflösungen bei ebenfalls steigenden Bildwiederholraten. Die Realität hingegen ist in Raum und Zeit kontinuierlich. Diese Grundverschiedenheit führt beim Betrachter zu perzeptuellen Unterschieden. Die Verfolgung der Aug-Blickrichtung ermöglicht blickpunktabhängige Darstellungsmethoden, die sichtbare Artefakte verhindern können. Diese Dissertation trägt zu vier Bereichen blickpunktabhängiger und wahrnehmungstreuer Darstellungsmethoden bei. Die Verfahren in Kapitel 4 und 5 haben zum Ziel, die wahrgenommene visuelle Qualität von Videos für den Betrachter zu erhöhen, wobei die Videos auf gewöhnlicher Ausgabehardware wie z.B. einem Fernseher oder Projektor dargestellt werden. Kapitel 6 beschreibt die Entwicklung eines neuartigen Head-mounted Displays mit Unterstützung zur Erfassung der Blickrichtung in Echtzeit. Die Kombination der Funktionen ermöglicht eine Reihe interessanter Anwendungen in Bezug auf Virtuelle Realität (VR) und Erweiterte Realität (AR). Das vierte und abschließende Verfahren in Kapitel 7 dieser Dissertation beschreibt einen neuen Algorithmus, der das entwickelte Eye-Tracking Head-mounted Display zum blickpunktabhängigen Rendern nutzt. Die Qualität des Shadings wird hierbei auf Basis eines Wahrnehmungsmodells für jeden Bildpixel in Echtzeit analysiert und angepasst. Das Verfahren hat das Potenzial den Berechnungsaufwand für das Shading einer virtuellen Szene auf ein Bruchteil zu reduzieren. Die in dieser Dissertation beschriebenen Verfahren und Untersuchungen zeigen, dass blickpunktabhängige Algorithmen die Darstellungsqualität von Bildern und Videos wirksam verbessern können, beziehungsweise sich bei gleichbleibender Bildqualität der Berechnungsaufwand des bildgebenden Verfahrens erheblich verringern lässt

Image and Evidence: The Study of Attention through the Combined Lenses of Neuroscience and Art

: Levy, EK 2012, ‘An artistic exploration of inattention blindness’, in Frontiers Hum Neurosci, vol. 5, ISSN=1662-5161.Full version unavailable due to 3rd party copyright restrictions.This study proposed that new insights about attention, including its phenomenon and pathology, would be provided by combining perspectives of the neurobiological discourse about attention with analyses of artworks that exploit the constraints of the attentional system. To advance the central argument that art offers a training ground for the attentional system, a wide range of contemporary art was analysed in light of specific tasks invoked. The kinds of cognitive tasks these works initiate with respect to the attentional system have been particularly critical to this research. Attention was explored within the context of transdisciplinary art practices, varied circumstances of viewing, new neuroscientific findings, and new approaches towards learning. Research for this dissertation required practical investigations in a gallery setting, and this original work was contextualised and correlated with pertinent neuroscientific approaches. It was also concluded that art can enhance public awareness of attention disorders and assist the public in discriminating between medical and social factors through questioning how norms of behaviour are defined and measured. This territory was examined through the comparative analysis of several diagnostic tests for attention deficit hyperactivity disorder (ADHD), through the adaptation of a methodology from economics involving patent citation in order to show market incentives, and through examples of data visualisation. The construction of an installation and collaborative animation allowed participants to experience first-hand the constraints on the attentional system, provoking awareness of our own “normal” physiological limitations. The embodied knowledge of images, emotion, and social context that are deeply embedded in art practices appeared to be capable of supplementing neuroscience’s understanding of attention and its disorders