3D gaze cursor: continuous calibration and end-point grasp control of robotic actuators

© 2016 IEEE.Eye movements are closely related to motor actions, and hence can be used to infer motor intentions. Additionally, eye movements are in some cases the only means of communication and interaction with the environment for paralysed and impaired patients with severe motor deficiencies. Despite this, eye-tracking technology still has a very limited use as a human-robot control interface and its applicability is highly restricted to 2D simple tasks that operate on screen based interfaces and do not suffice for natural physical interaction with the environment. We propose that decoding the gaze position in 3D space rather than in 2D results into a much richer spatial cursor signal that allows users to perform everyday tasks such as grasping and moving objects via gaze-based robotic teleoperation. Eye tracking in 3D calibration is usually slow - we demonstrate here that by using a full 3D trajectory for system calibration generated by a robotic arm rather than a simple grid of discrete points, gaze calibration in the 3 dimensions can be successfully achieved in short time and with high accuracy. We perform the non-linear regression from eye-image to 3D-end point using Gaussian Process regressors, which allows us to handle uncertainty in end-point estimates gracefully. Our telerobotic system uses a multi-joint robot arm with a gripper and is integrated with our in-house GT3D binocular eye tracker. This prototype system has been evaluated and assessed in a test environment with 7 users, yielding gaze-estimation errors of less than 1cm in the horizontal, vertical and depth dimensions, and less than 2cm in the overall 3D Euclidean space. Users reported intuitive, low-cognitive load, control of the system right from their first trial and were straightaway able to simply look at an object and command through a wink to grasp this object with the robot gripper

Eye Movement and Pupil Measures: A Review

Our subjective visual experiences involve complex interaction between our eyes, our brain, and the surrounding world. It gives us the sense of sight, color, stereopsis, distance, pattern recognition, motor coordination, and more. The increasing ubiquity of gaze-aware technology brings with it the ability to track gaze and pupil measures with varying degrees of fidelity. With this in mind, a review that considers the various gaze measures becomes increasingly relevant, especially considering our ability to make sense of these signals given different spatio-temporal sampling capacities. In this paper, we selectively review prior work on eye movements and pupil measures. We first describe the main oculomotor events studied in the literature, and their characteristics exploited by different measures. Next, we review various eye movement and pupil measures from prior literature. Finally, we discuss our observations based on applications of these measures, the benefits and practical challenges involving these measures, and our recommendations on future eye-tracking research directions

Study of non-invasive cognitive tasks and feature extraction techniques for brain-computer interface (BCI) applications

A brain-computer interface (BCI) provides an important alternative for disabled people that enables the non-muscular communication pathway among individual thoughts and different assistive appliances. A BCI technology essentially consists of data acquisition, pre-processing, feature extraction, classification and device command. Indeed, despite the valuable and promising achievements already obtained in every component of BCI, the BCI field is still a relatively young research field and there is still much to do in order to make BCI become a mature technology. To mitigate the impediments concerning BCI, the study of cognitive task together with the EEG feature and classification framework have been investigated. There are four distinct experiments have been conducted to determine the optimum solution to those specific issues. In the first experiment, three cognitive tasks namely quick math solving, relaxed and playing games have been investigated. The features have been extracted using power spectral density (PSD), logenergy entropy, and spectral centroid and the extracted feature has been classified through the support vector machine (SVM), K-nearest neighbor (K-NN), and linear discriminant analysis (LDA). In this experiment, the best classification accuracy for single channel and five channel datasets were 86% and 91.66% respectively that have been obtained by the PSD-SVM approach. The wink based facial expressions namely left wink, right wink and no wink have been studied through fast Fourier transform (FFT) and sample range feature and then the extracted features have been classified using SVM, K-NN, and LDA. The best accuracy (98.6%) has been achieved by the sample range-SVM based approach. The eye blinking based facial expression has been investigated following the same methodology as the study of wink based facial expression. Moreover, the peak detection approach has also been employed to compute the number of blinks. The optimum accuracy of 99% has been achieved using the peak detection approach. Additionally, twoclass motor imagery hand movement has been classified using SVM, K-NN, and LDA where the feature has been extracted through PSD, spectral centroid and continuous wavelet transform (CWT). The optimum 74.7% accuracy has been achieved by the PSDSVM approach. Finally, two device command prototypes have been designed to translate the classifier output. One prototype can translate four types of cognitive tasks in terms of 5 watts four different colored bulbs, whereas, another prototype may able to control DC motor utilizing cognitive tasks. This study has delineated the implementation of every BCI component to facilitate the application of brainwave assisted assistive appliances. Finally, this thesis comes to the end by drawing the future direction regarding the current issues of BCI technology and these directions may significantly enhance usability for the implementation of commercial applications not only for the disabled but also for a significant number of healthy users

Machine Understanding of Human Behavior

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior

Design of a gesture detection system at real time

The project I have realized consist in developing a gesture detection system to work at real time situations. In particular, it has the aim to detect a wink of an eye and activate a flag when that happens. There are some actual projects and systems that already do that, but they are focused on voice detection. This project follows the same principles but it uses an input of video instead of a sound. The creation of the pipeline was made in different parts. First of all, a convolutional neural network was created to detect the gesture in a sequence of images and it had to be trained to do so. Secondly, a convolutional neural network for face detection was used as background subtraction, in order to select the main part of the image. Finally, different methods of optimization were taken into account, so as to make the processing operations work faster. A code was implemented to prove the background susbstraction of the image in order to reduce the processing time. Using this code, results were obtained about the accuracy and the processing time using Python. However we only obtained results from the part of background subtraction because the part of detecting the gesture was finally proposed as future work according to the lack of time and resources. All in all, the results obtained were about the simplification of the image doing the background subtraction using a face detection method. We obtained that the time to detect the zone of the face took an average of 0.65 second. Knowing that the system needs to take an image with the camera, do the background subtraction and process a convolutional neural network several times to detect a gesture, we deduce that the time that lasts the face detection makes that imposible. It is needed to improve it more to make it work at real time