Brain-Computer Interfaces

Die vorliegende Dissertation beschreibt Ergebnisse der Arbeit, durchgeführt am Fraunhofer Institut für Rechnerarchitektur und Softwaretechnik (FIRST), insbesondere bei der Forschungsgruppe für Intelligente Datenanalyse (IDA), im Rahmen des Projektes "Brain-Computer Interface" (BCI). Das angestrebte Ziel des aktuell laufenden Projektes ist es, ein Hardware- und Software-System zu entwerfen und zu entwickeln, das in der Lage ist elektroencephalographische (EEG)Signale(gewonnen auf eine nicht-invasive Art, mit Hilfe der Oberflächenelektroden, die über dem Kopf des Benutzers angebracht sind), in Echtzeit in spezielle Kommandos umzuwandeln, so dass für den Probanden eine verlässliche Steuerung einer Computeranwendung, bzw. eines Gerätes ermöglicht wird. Die Steuerung einer Computeranwendung soll im Rahmen dieser Arbeit in Form von einfachen Computerspielen (Ping-Pong, Pacman, Tetris) repräsentiert werden, im Weiteren bezeichnet als "Brain-Gaming". Hierzu sind fundierte Überlegungen zum Entwurf und Realisierung einer Kommunikationsschnittstelle und des zugehörigen Protokolls angestellt worden. Ein weiterer wichtiger Bestandteil jeder Steuerung ist deren Strategie und der Befehlssatz der Anwendung. So wurden mehrere Strategien entwickelt, implementiert und in verschiedenen Szenarien experimentell erprobt. Die Flexibilität der Steuerungsschnittstelle stellte sich als einer der wichtigsten Aspekte beim Entwurf und der Entwicklung von Rückkopplungsanwendungen. Bei der Steuerung eines Gerätes kann es sich um das Lenken und Bewegen eines Rollstuhls, z.B. für querschnittsgelehmte Patienten, oder um das Bewegen einer Arm-, bzw. Beinprothese für Patienten mit amputierten Extremitäten handeln. Dies wurde vorerst als Simulation einer Extremität (Arm) auf dem Computer-Bildschirm realisiert, so dass es in zukünftigen Experimenten an bedürftigen Patienten getestet werden kann. Im Gegensatz zu Brain-Gaming Experimenten, bei denen der Spieler mit einem zusätzlichen Kommunikationskanal (der unabhängig von anderen normalen Kanälen des menschlichen neuromuskulären Systems ist) ausgestattet wird, stellen die Experimente an Patienten keine Anforderung an die ultra-schnelle Erkennung der Bewegungsabsicht; So kann das Steuersignal zur Ausführung einer simulierten Bewegung auch nach dem Auslösen der eigentlichen Phantombewegung, sogar nach deren Ausführung, erkannt werden. In Experimenten mit Feedback-Szenarien, die kompetitiven Spielen ähneln, können verschiedene Aspekte der ultraschnellen Erkennung einer Bewegungsintension mit Hilfe von Reaktionstests untersucht werden. Dieses eröffnet neue Perspektiven bei der Ausführung von Präventivmaßnahmen in zeitkritischen Anwendungen. Diese Dissertation, sowie die Entwicklung und Implementierung des Prototyps basiert auf fundierten Erkenntnissen der Neurophysiologie; Ein ausgewähltes Kapitel dieser Dissertation verschafft deshalb tieferen Einblick in die menschliche Neurophysiologie. Ferner, beschreibt ein gesondertes Kapitel dieser Dissertation die Entwicklung und Implementierung eines online Prototyps " des BBCI-Systems (Berliner Brain-Computer Interface) " in dessen Einzelkomponenten und der Gesamtheit aus dem Sichtpunkt der Softwaretechnik. Im Rahmen der Arbeit wurden mehrere Rückkopplungsmodule (Bio-Feedback), sowohl spielerischen Charakters, als auch zu Rehabilitationszwecken entwickelt, die hier im Detail vorgestellt werden. Mit besonderer Aufmerksamkeit wurde der Einfluss des online Bio-Feedbacks auf den Probanden untersucht.This dissertation aims to describe work carried out at the Fraunhofer Institute for Computer Architecture and Software Technology (FhG-FIRST), in particular at the research group for Intelligent Data Analysis (IDA), within the project "Brain-Computer Interface" (BCI). The goal of that project is to design and develop a hardware and software system that is capable of transforming, in real-time, electroencephalographic (EEG) signals (signals retrieved in a non-invasive way from surface electrodes placed over the user's head) into specific commands such that the user gains reliable control over a computer application or a device. In this dissertation, control over a computer application will be represented with "Brain-Gaming", i.e. simple computer games like Ping-Pong, Pacman or Tetris. To this end, substantial considerations were made on the design and realization of a communication interface and its corresponding protocol. A further important component of every control operation is its strategy and the control alphabet, i.e. the command set. For this purpose, several control strategies were developed, implemented and proved experimentally in different scenarios. The most important aspect of the design and development of these interfaces turned out to be their flexibility. Control over a device could include the steering and moving of a wheel-chair for paralyzed patients or the gaining of control over an arm or foot prosthesis for patients with amputated limbs. The latter was realized here as a computer-based simulation of a virtual limb (e.g. arm), such that it can be tested in future experiments on patients with amputated limbs. In contrast to the "Brain-Gaming" experiments, where the player was equipped with an additional communication channel (one that exists independently of normal communication channels of the human neuromuscular system), the experiments with patients did not require an ultra-fast recognition of the intended movement; i.e. the command signal for a simulated movement can be recognized after the phantom movement is initialized and performed. In experiments with feedback scenarios, which can be resembled as competitive games, several aspects of ultra-fast movement detection could be investigated with reaction tests. This opens new perspectives for the execution of preventive actions in time-critical applications. This dissertation, and the development and implementation of the prototype, is based on well-founded insights into human neurophysiology; one chapter will deal exclusively with these insights. Moreover, a special chapter of this dissertation will also describe the development and implementation of an online prototype of the BBCI system (Berlin Brain-Computer Interface) from the software engineering viewpoint. A number of bio-feedback modules, for gaming and for rehabilitation purposes, were developed within this work and will be presented in detail. Special attention was paid to the influence of the online bio-feedback on the user's behavior

A New Algorithm for the Interrogation of 3D Holographic Particle Tracking Velocimetry Data Based on Deterministic Annealing and EM-Optimization

Recently we have presented a new particle tracking algorithm for the interrogation of PTV data [Kuzmanowski et al., 1998], [Stellmacher and Obermayer, 1999]. This procedure is based on an algorithm recently proposed by [Gold et al., 1998] for solving point matching problems in statistical pattern recognition, which estimates correspondences between particles in two PTV frames and the local flow field parameter simultaneously. The new method has two advantages: (1) It allows to determine not only local velocity, but also other local components of the flow field like rotation and shear, and (2) it allows to reliably determine flow-field parameters also in regions of high velocity gradients (e.g. vortices or shear flow). In this contribution we extend this algorithm to be applied to the interrogation of 3D holographic PIV data. Benchmarks with cross-correlation and nearest neighbor methods show, that the algorithm remains the superior performance which we have observed for the 2D case. Be..

Berlin Brain–Computer Interface -- The HCI communication channel for discovery

The investigation of innovative Human-Computer Interfaces (HCI) provides a challenge for future interaction research and development. Brain-Computer Interfaces (BCIs) exploit the ability of human communication and control bypassing the classical neuromuscular communication channels. In general, BCIs offer a possibility of communication for people with severe neuromuscular disorders, such as amyotrophic lateral sclerosis (ALS) or complete paralysis of all extremities due to high spinal cord injury. Beyond medical applications, a BCI conjunction with exciting multimedia applications, e.g., a dexterity discovery, could define a new level of control possibilities also for healthy customers decoding information directly from the user's brain, as reflected in EEG signals which are recorded non-invasively from the scalp. This contribution introduces the Berlin Brain-Computer Interface (BBCI) and presents set-ups where the user is provided with intuitive control strategies in plausible interactive bio-feedback applications. Yet at its beginning, BBCI thus adds a new dimension in HCI research by offering the user an additional and independent communication channel based on brain activity only. Successful experiments already yielded inspiring proofs-of-concept. A diversity of interactive application models, say computer games, and their specific intuitive control strategies are now open for BCI research aiming at a further speed up of user adaptation and increase of learning success and transfer bit rates. BBCI is a complex distributed software system that can be run on several communicating computers responsible for (i) the signal acquisition, (ii) the data processing and (iii) the feedback application. Developing a BCI system, special attention must be paid to the design of the feedback application that serves as the HCI unit. This should provide the user with the information about her/his brain activity in a way that is intuitively intelligible. Exciting discovery applications qualify perfectly for this role. However, most of these applications incorporate control strategies that are developed especially for the control with haptic devices, e.g., joystick, keyboard or mouse. Therefore, novel control strategies should be developed for this purpose that (i) allow the user to incorporate additional information for the control of animated objects and (ii) do not frustrate the user in the case of a misclassification of the decoded brain signal. BCIs are able to decode different information types from the user's brain activity, such as sensory perception or motor intentions and imaginations, movement preparations, levels of stress, workload or task-related idling. All of these diverse brain signals can be incorporated in an exciting discovery scenario. Modern HCI research and development technologies can provide BCI researchers with the know-how about interactive feedback applications and corresponding control strategies

Boosting Bit Rates and Error Detection for the Classification of Fast-paced Motor Commands Based on Single-trial EEG Analysis

Brain-Computer-Interfaces (BCI) involve two coupled adapting systems: the human subject and the computer. In developing our BCI, our goal was to minimize the need for subject training and to impose the major learning load on the computer. To this end, we use behavioral paradigms that exploit single-trial EEG potentials preceding voluntary nger movements. Here, we report recent results on the basic physiology of such pre-movement event-related potentials (ERP): 1) We predict the laterality of imminent left vs. right hand nger movements in a natural keyboard typing condition and demonstrate that a single-trial classi- cation based on the lateralized Bereitschaftspotential (BP) achieves good accuracies even at a pace as fast as 2 taps per second. Results for 4 out of 8 subjects reached a peak information transfer rate of more than 15 bits per minute (bpm); the 4 other subjects reached 6-10 bpm. 2) We detect cerebral error potentials from single false-response trials in a forced-choice task, reecting the subject's recognition of an erroneous response. Based on a specically tailored classi cation procedure that limits the rate of false positives at, e.g. 2 %, the algorithm manages to detect 85 % of error trials in 7/8 subjects. Thus, concatenating a primary single-trial BP-paradigm involving nger classication feedback with such secondary error detection could serve as an ecient on-line conrmation/correction tool for improvement of bit rates in a future BCI setting. As the present variant of the Berlin BCI (BBCI) is designed to achieve fast classications in normally behaving subjects, it opens a new perspective for assistance of action control in time-critical behavioral contexts; the potential transfer to paralysed patients will require further study