EOG-Based Eye Movement Classification and Application on HCI Baseball Game

© 2013 IEEE. Electrooculography (EOG) is considered as the most stable physiological signal in the development of human-computer interface (HCI) for detecting eye-movement variations. EOG signal classification has gained more traction in recent years to overcome physical inconvenience in paralyzed patients. In this paper, a robust classification technique, such as eight directional movements is investigated by introducing a concept of buffer along with a variation of the slope to avoid misclassification effects in EOG signals. Blinking detection becomes complicated when the magnitude of the signals are considered. Hence, a correction technique is introduced to avoid misclassification for oblique eye movements. Meanwhile, a case study has been considered to apply these correction techniques to HCI baseball game to learn eye-movements

Auxilio: A Sensor-Based Wireless Head-Mounted Mouse for People with Upper Limb Disability

Upper limb disability may be caused either due to accidents, neurological disorders, or even birth defects, imposing limitations and restrictions on the interaction with a computer for the concerned individuals using a generic optical mouse. Our work proposes the design and development of a working prototype of a sensor-based wireless head-mounted Assistive Mouse Controller (AMC), Auxilio, facilitating interaction with a computer for people with upper limb disability. Combining commercially available, low-cost motion and infrared sensors, Auxilio solely utilizes head and cheek movements for mouse control. Its performance has been juxtaposed with that of a generic optical mouse in different pointing tasks as well as in typing tasks, using a virtual keyboard. Furthermore, our work also analyzes the usability of Auxilio, featuring the System Usability Scale. The results of different experiments reveal the practicality and effectiveness of Auxilio as a head-mounted AMC for empowering the upper limb disabled community.Comment: 28 pages, 9 figures, 5 table

Control of Light Disc Based on Move of Eyes

Import 03/11/2016Tématem této bakalářské práce je návrh a realizace světelného terče, tvořeného LED diodami, který je ovládán pomocí pohybů očních bulbů. Oční pohyby jsou snímány pomocí metody EOG. Pro měření jsou použity klasické Ag/AgCl elektrody. Teoretická část rozebírá problematiku elektrookulografie se zaměřením na vznik signálů. Součástí práce je také rešerše, která popisuje různá zařízení, která pro ovládání používají metodu EOG. Spojení mezi EOG přístrojem, světelným terčem a PC zajištuje přístroj NI ELVIS II. Jako vizualizační program bylo zvoleno grafické vývojové prostředí LabVIEW, které obsahuje knihovny pro přímou komunikaci s NI ELVIS II. V poslední části bylo nutno zhodnotit naměřené výsledky.The topic of this thesis is the design and implementation of the light target, which is formed by the LEDs and controlled by moving eyeballs. Eyeballs movements are scanned by using the method EOG. For measurements are used Ag/AgCl electrodes. The theoretical part analyzes electrooculography and focuse on the emergence of the signals. This thesis also includes research that describes various devices which use the EOG method to control. The connection between EOG, the light target and PC provides NI ELVIS II. Graphical development environment of LabView was chosen as a visualization program, which contains libraries for direct communication with NI ELVIS II. In the last section, it was necessary to evaluate the measured results.450 - Katedra kybernetiky a biomedicínského inženýrstvívýborn

Applications of the electric potential sensor for healthcare and assistive technologies

The work discussed in this thesis explores the possibility of employing the Electric Potential Sensor for use in healthcare and assistive technology applications with the same and in some cases better degrees of accuracy than those of conventional technologies. The Electric Potential Sensor is a generic and versatile sensing technology capable of working in both contact and non-contact (remote) modes. New versions of the active sensor were developed for specific surface electrophysiological signal measurements. The requirements in terms of frequency range, electrode size and gain varied with the type of signal measured for each application. Real-time applications based on electrooculography, electroretinography and electromyography are discussed, as well as an application based on human movement. A three sensor electrooculography eye tracking system was developed which is of interest to eye controlled assistive technologies. The system described achieved an accuracy at least as good as conventional wet gel electrodes for both horizontal and vertical eye movements. Surface recording of the electroretinogram, used to monitor eye health and diagnose degenerative diseases of the retina, was achieved and correlated with both corneal fibre and wet gel surface electrodes. The main signal components of electromyography lie in a higher bandwidth and surface signals of the deltoid muscle were recorded over the course of rehabilitation of a subject with an injured arm. Surface electromyography signals of the bicep were also recorded and correlated with the joint dynamics of the elbow. A related non-contact application of interest to assistive technologies was also developed. Hand movement within a defined area was mapped and used to control a mouse cursor and a predictive text interface

Methods and metrics for the improvement of the interaction and the rehabilitation of cerebral palsy through inertial technology

Cerebral palsy (CP) is one of the most limiting disabilities in childhood, with 2.2 cases per 1000 1-year survivors. It is a disorder of movement and posture due to a defect or lesion of the immature brain during the pregnancy or the birth. These motor limitations appear frequently in combination with sensory and cognitive alterations generally result in great difficulties for some people with CP to manipulate objects, communicate and interact with their environment, as well as limiting their mobility. Over the last decades, instruments such as personal computers have become a popular tool to overcome some of the motor limitations and promote neural plasticity, especially during childhood. According to some estimations, 65% of youths with CP that present severely limited manipulation skills cannot use standard mice nor keyboards. Unfortunately, even when people with CP use assistive technology for computer access, they face barriers that lead to the use of typical mice, track balls or touch screens for practical reasons. Nevertheless, with the proper customization, novel developments of alternative input devices such as head mice or eye trackers can be a valuable solution for these individuals. This thesis presents a collection of novel mapping functions and facilitation algorithms that were proposed and designed to ease the act of pointing to graphical elements on the screen—the most elemental task in human-computer interaction—to individuals with CP. These developments were implemented to be used with any head mouse, although they were all tested with the ENLAZA, an inertial interface. The development of such techniques required the following approach: Developing a methodology to evaluate the performance of individuals with CP in pointing tasks, which are usually described as two sequential subtasks: navigation and targeting. Identifying the main motor abnormalities that are present in individuals with CP as well as assessing the compliance of these people with standard motor behaviour models such as Fitts’ law. Designing and validating three novel pointing facilitation techniques to be implemented in a head mouse. They were conceived for users with CP and muscle weakness that have great difficulties to maintain their heads in a stable position. The first two algorithms consist in two novel mapping functions that aim to facilitate the navigation phase, whereas the third technique is based in gravity wells and was specially developed to facilitate the selection of elements in the screen. In parallel with the development of the facilitation techniques for the interaction process, we evaluated the feasibility of use inertial technology for the control of serious videogames as a complement to traditional rehabilitation therapies of posture and balance. The experimental validation here presented confirms that this concept could be implemented in clinical practice with good results. In summary, the works here presented prove the suitability of using inertial technology for the development of an alternative pointing device—and pointing algorithms—based on movements of the head for individuals with CP and severely limited manipulation skills and new rehabilitation therapies for the improvement of posture and balance. All the contributions were validated in collaboration with several centres specialized in CP and similar disorders and users with disability recruited in those centres.La parálisis cerebral (PC) es una de las deficiencias más limitantes de la infancia, con un incidencia de 2.2 casos por cada 1000 supervivientes tras un año de vida. La PC se manifiesta principalmente como una alteración del movimiento y la postura y es consecuencia de un defecto o lesión en el cerebro inmaduro durante el embarazo o el parto. Las limitaciones motrices suelen aparecer además en compañía de alteraciones sensoriales y cognitivas, lo que provoca por lo general grandes dificultades de movilidad, de manipulación, de relación y de interacción con el entorno. En las últimas décadas, el ordenador personal se ha extendido como herramienta para la compensación de parte de estas limitaciones motoras y como medio de promoción de la neuroplasticidad, especialmente durante la infancia. Desafortunadamente, cerca de un 65% de las personas PC que son diagnosticadas con limitaciones severas de manipulación son incapaces de utilizar ratones o teclados convencionales. A veces, ni siquiera la tecnología asistencial les resulta de utilidad ya que se encuentran con impedimentos que hacen que opten por usar dispositivos tradicionales aun sin dominar su manejo. Para estas personas, los desarrollos recientes de ratones operados a través de movimientos residuales con la cabeza o la mirada podrían ser una solución válida, siempre y cuando se personalice su manejo. Esta tesis presenta un conjunto de novedosas funciones de mapeo y algoritmos de facilitaci ón que se han propuesto y diseñado con el ánimo de ayudar a personas con PC en las tareas de apuntamiento de objetos en la pantalla —las más elementales dentro de la interacción con el ordenador. Aunque todas las contribuciones se evaluaron con la interfaz inercial ENLAZA, desarrollada igualmente en nuestro grupo, podrían ser aplicadas a cualquier ratón basado en movimientos de cabeza. El desarrollo de los trabajos se resume en las siguientes tareas abordadas: Desarrollo de una metodología para la evaluación de la habilidad de usuarios con PC en tareas de apuntamiento, que se contemplan como el encadenamiento de dos sub-tareas: navegación (alcance) y selección (clic). Identificación de los tipos de alteraciones motrices presentes en individuos con PC y el grado de ajuste de éstos a modelos estándares de comportamiento motriz como puede ser la ley de Fitts. Propuesta y validación de tres técnicas de facilitación del alcance para ser implementadas en un ratón basado en movimientos de cabeza. La facilitación se ha centrado en personas que presentan debilidad muscular y dificultades para mantener la posición de la cabeza. Mientras que los dos primeros algoritmos se centraron en facilitar la navegación, el tercero tuvo como objetivo ayudar en la selección a través de una técnica basada en pozos gravitatorios de proximidad. En paralelo al desarrollo de estos algoritmos de facilitación de la interacción, evaluamos la posibilidad de utilizar tecnología inercial para el control de videojuegos en rehabilitación. Nuestra validación experimental demostró que este concepto puede implementarse en la práctica clínica como complemento a terapias tradicionales de rehabilitación de la postura y el equilibrio. Como conclusión, los trabajos desarrollados en esta tesis vienen a constatar la idoneidad de utilizar sensores inerciales para el desarrollo de interfaces de accesso alternativo al ordenador basados en movimientos residuales de la cabeza para personas con limitaciones severas de manipulación. Esta solución se complementa con algoritmos de facilitación del alcance. Por otra parte, estas soluciones tecnológicas de interfaz con el ordenador representan igualmente un complemento de terapias tradicionales de rehabilitación de la postura y el equilibrio. Todas las contribuciones se validaron en colaboración con una serie de centros especializados en parálisis cerebral y trastornos afines contando con usuarios con discapacidad reclutados en dichos centros.This thesis was completed in the Group of Neural and Cognitive Engineering (gNEC) of the CAR UPM-CSIC with the financial support of the FP7 Framework EU Research Project ABC (EU-2012-287774), the IVANPACE Project (funded by Obra Social de Caja Cantabria, 2012-2013), and the Spanish Ministry of Economy and Competitiveness in the framework of two projects: the Interplay Project (RTC-2014-1812-1) and most recently the InterAAC Project (RTC-2015-4327-1)Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Juan Manuel Belda Lois.- Secretario: María Dolores Blanco Rojas.- Vocal: Luis Fernando Sánchez Sante

Development of a practical and mobile brain-computer communication device for profoundly paralyzed individuals

Thesis (Ph.D.)--Boston UniversityBrain-computer interface (BCI) technology has seen tremendous growth over the past several decades, with numerous groundbreaking research studies demonstrating technical viability (Sellers et al., 2010; Silvoni et al., 2011). Despite this progress, BCIs have remained primarily in controlled laboratory settings. This dissertation proffers a blueprint for translating research-grade BCI systems into real-world applications that are noninvasive and fully portable, and that employ intelligent user interfaces for communication. The proposed architecture is designed to be used by severely motor-impaired individuals, such as those with locked-in syndrome, while reducing the effort and cognitive load needed to communicate. Such a system requires the merging of two primary research fields: 1) electroencephalography (EEG)-based BCIs and 2) intelligent user interface design. The EEG-based BCI portion of this dissertation provides a history of the field, details of our software and hardware implementation, and results from an experimental study aimed at verifying the utility of a BCI based on the steady-state visual evoked potential (SSVEP), a robust brain response to visual stimulation at controlled frequencies. The visual stimulation, feature extraction, and classification algorithms for the BCI were specially designed to achieve successful real-time performance on a laptop computer. Also, the BCI was developed in Python, an open-source programming language that combines programming ease with effective handling of hardware and software requirements. The result of this work was The Unlock Project app software for BCI development. Using it, a four-choice SSVEP BCI setup was implemented and tested with five severely motor-impaired and fourteen control participants. The system showed a wide range of usability across participants, with classification rates ranging from 25-95%. The second portion of the dissertation discusses the viability of intelligent user interface design as a method for obtaining a more user-focused vocal output communication aid tailored to motor-impaired individuals. A proposed blueprint of this communication "app" was developed in this dissertation. It would make use of readily available laptop sensors to perform facial recognition, speech-to-text decoding, and geo-location. The ultimate goal is to couple sensor information with natural language processing to construct an intelligent user interface that shapes communication in a practical SSVEP-based BCI

Toward Simulation-Based Training Validation Protocols: Exploring 3d Stereo with Incremental Rehearsal and Partial Occlusion to Instigate and Modulate Smooth Pursuit and Saccade Responses in Baseball Batting

“Keeping your eye on the ball” is a long-standing tenet in baseball batting. And yet, there are no protocols for objectively conditioning, measuring, and/or evaluating eye-on-ball coordination performance relative to baseball-pitch trajectories. Although video games and other virtual simulation technologies offer alternatives for training and obtaining objective measures, baseball batting instruction has relied on traditional eye-pitch coordination exercises with qualitative “face validation”, statistics of whole-task batting performance, and/or subjective batter-interrogation methods, rather than on direct, quantitative eye-movement performance evaluations. Further, protocols for validating transfer-of-training (ToT) for video games and other simulation-based training have not been established in general ― or for eye-movement training, specifically. An exploratory research study was conducted to consider the ecological and ToT validity of a part-task, virtual-fastball simulator implemented in 3D stereo along with a rotary pitching machine standing as proxy for the live-pitch referent. The virtual-fastball and live-pitch simulation couple was designed to facilitate objective eye-movement response measures to live and virtual stimuli. The objective measures 1) served to assess the ecological validity of virtual fastballs, 2) informed the characterization and comparison of eye-movement strategies employed by expert and novice batters, 3) enabled a treatment protocol relying on repurposed incremental-rehearsal and partial-occlusion methods intended to instigate and modulate strategic eye movements, and 4) revealed whether the simulation-based treatment resulted in positive (or negative) ToT in the real task. Results indicated that live fastballs consistently elicited different saccade onset time responses than virtual fastballs. Saccade onset times for live fastballs were consistent with catch-up saccades that follow the smooth-pursuit maximum velocity threshold of approximately 40-70˚/sec while saccade onset times for virtual fastballs lagged in the order of 13%. More experienced batters employed more deliberate and timely combinations of smooth pursuit and catch-up saccades than less experienced batters, enabling them to position their eye to meet the ball near the front edge of home plate. Smooth pursuit and saccade modulation from treatment was inconclusive from virtual-pitch pre- and post-treatment comparisons, but comparisons of live-pitch pre- and post-treatment indicate ToT improvements. Lagging saccade onset times from virtual-pitch suggest possible accommodative-vergence impairment due to accommodation-vergence conflict inherent to 3D stereo displays

Brain processes underlying the perception of audiovisual content

Programa de Doctorado en NeurocienciasLínea de Investigación: Procesos neuronales que subyacen al aprendizaje y la memoria en el animal despierto. Substratos neuronales de los comportamientos apetitivos y exploratoriosClave Programa: DNFCódigo Línea: 90La percepción audiovisual ha sido tradicionalmente abordada desde perspectivas sociales, psicológicas, semióticas y antropológicas. En los últimos años ha surgido un enfoque novedoso desde la neurociencia. En el campo de la neurocinemática, las técnicas de la neurociencia se utilizan para comprender los procesos cerebrales que subyacen a la percepción del contenido audiovisual. Esta Tesis Doctoral estudia estos procesos bajo tres perspectivas principales: (1) encontrar diferencias en la percepción de un contenido narrativo en una representación real o en una pantalla; (2) aprender cómo el estilo y el contenido de las obras audiovisuales afectan a la percepción de los espectadores; y (3) determinar si la profesionalización en los medios tiene un impacto en la percepción visual. Con técnicas electroencefalográficas y electrooculográficas, se registraron los parpadeos y la actividad eléctrica cerebral de los participantes. Los principales hallazgos de estos trabajos son los siguientes. En cuanto a la primera perspectiva, se encontraron diferencias en la percepción de la realidad y el audiovisual. Un contenido narrativo en un video provoca una menor tasa de parpadeo de los espectadores en comparación con el mismo contenido dentro de una actuación real. En cuanto a la segunda perspectiva relacionada con el estilo y el contenido de las obras audiovisuales, se encontró que: (i) la tasa de parpadeo de los espectadores cambia según el estilo de montaje de las obras audiovisuales, cuanto más caótico el estilo, menor la tasa de parpadeo; (ii) los cortes de plano inhiben los parpadeos inmediatamente posteriores a ellos, evitando la posibilidad de trabajar en una sincronización de cortes con parpadeos para evitar la pérdida de información visual realizada por ambos procesos; (iii) los cortes de plano afectan la percepción de los medios y provocan una posible propagación de la actividad desde las áreas occipitales hacia las áreas frontales del cerebro alrededor de 200 ms después del corte; (iv) el estilo de edición donde se insertan los cortes tiene un impacto en la decodificación de los espectadores y los audiovisuales caóticos y rápidos aumentan el alcance atencional pero disminuyen el procesamiento cognitivo superior; (v) existen contenidos narrativos específicos que sincronizan aumentos y disminuciones de la tasa de parpadeo de los espectadores, independientemente del estilo de edición en que se presenten; y (vi) los cortes en los audiovisuales no provocan ninguna actividad cerebral asimétrica específica en la banda alfa en los espectadores, lo que sugiere que la asimetría cerebral al ver audiovisuales puede estar más relacionada con el contenido narrativo que con el estilo formal. Finalmente, con respecto a la tercera perspectiva relacionada con la profesionalización de los medios, se encontró que: (i) los profesionales de los medios prestan más atención tanto a las pantallas como al mundo real que los que no son profesionales audiovisuales; (ii) los profesionales de los medios disminuyen más su tasa de parpadeo después de los cortes de plano, lo que sugiere que pueden manejar mejor la pérdida de información visual que implican los parpadeos evitándolos cuando se presenta nueva información visual; y (iii) la conectividad cerebral efectiva ocurre de una manera más organizada en los profesionales del audiovisual. En conclusión, los audiovisuales modulan la percepción visual de los espectadores, en función de varios parámetros como el estilo, el contenido y la profesionalización. Conocer los procesos cerebrales que subyacen a la percepción de contenidos audiovisuales puede ser útil en múltiples ámbitos como el comunicativo, el clínico o el formativo, entre otros.Universidad Pablo de Olavide de Sevilla. Departamento de Fisiología, Anatomía y Biología Celula

Leveraging Eye Structure and Motion to Build a Low-Power Wearable Gaze Tracking System

Clinical studies have shown that features of a person\u27s eyes can function as an effective proxy for cognitive state and neurological function. Technological advances in recent decades have allowed us to deepen this understanding and discover that the actions of the eyes are in fact very tightly coupled to the operation of the brain. Researchers have used camera-based eye monitoring technology to exploit this connection and analyze mental state across across many different metrics of interest. These range from simple things like attention and scene processing, to impairments such as a fatigue or substance use, and even significant mental disorders such as Parkinson\u27s, autism, and schizophrenia. While there is a wealth of knowledge and social benefit to be gained from eye tracking, the field has historically been restricted to laboratory use by crippling technological limitations - most notably, device size and power consumption. These issues primarily stem from the use of high-resolution cameras and heavyweight video-processing algorithms, both of which induce extremely high performance overhead on the eye tracker. To address this problem, we have constructed a lightweight, ultra-low-power eye monitoring device in the form factor of a pair of eyeglasses. The key guiding design principle for its construction was saliency-aware resource minimization. Specifically, our design leverages the fact that close-up images of the eye are characterized by large salient features which provide a high degree of redundant information; we exploit this to heavily subsample the eye image and reduce resource utilization while performing effective eye tracking. In the first part of this thesis, we present an initial design of a wearable system to enable ubiquitous eye tracking. By exploiting the fact that the eye has several large, visually redundant features such as the iris and pupil, we were able to develop a neural-network-based adaptive-sampling algorithm for predicting the gaze point while sampling a minimal number of pixels from the image. This enabled us to realize a power savings using specialized imaging hardware that would sample only those most salient pixels, which proportionally reduced the power and time cost of reading images for eye tracking. With these optimizations we were able to build a first-of-of its kind wearable eye tracker that consumed 40 mW of power and demonstrated a gaze tracking error of only 3 degrees across multiple subjects. We refer to this device as the iShadow platform. The second contribution and section of this thesis is a significant improvement upon the original iShadow design for the purpose of improving both power utilization and eye tracking performance. We constructed a new pupil-tracking algorithm based on lightweight computer vision features, which leverages the smoothness of the eye\u27s motion to reduce even further the amount of camera sampling needed. To guard against very infrequent discontinuities resulting from blinks or reflections off the eye, we integrated this model with the previously-used one-shot neural network algorithm. Because the common case (smooth, uninterrupted eye motion) occurs 90% of the time, we were able to realize a dramatic increase in performance due to the efficiency of the smooth tracking algorithm. The new and improved system, labeled CIDER, enabled much more accurate eye tracking - 0.4 degree error - with power consumption as low as 7 mW. This design also enabled a tradeoff between power consumption and eye tracking rate, in which it was also possible to draw higher power of ~30 mW in order to do eye tracking at rates of up to 240 frames per second. The final contribution of this thesis is a re-designed version of the iShadow glasses hardware that is suitable for ``in-the-wild\u27\u27 studies on subjects in their daily living environment. A wearable device, especially one that is worn on the head, must be minimally obtrusive in order to be accepted and used in the field by subjects. This design goal conflicts with the ideal placement of cameras that is needed for achieving consistent eye tracking fidelity. We present multiple possible methods we explored for addressing these competing design challenges, and discuss the reasons that many proved infeasible. To conclude, we present a working design solution that appears to optimally trade off user comfort and convenience and against the technical requirements of the system