Gaze-Based Control of Robot Arm in Three-Dimensional Space

Eye tracking technology has opened up a new communication channel for people with very restricted body movements. These devices had already been successfully applied as a human computer interface, e.g. for writing a text, or to control different devices like a wheelchair. This thesis proposes a Human Robot Interface (HRI) that enables the user to control a robot arm in 3-Dimensional space using only 2-Dimensional gaze direction and the states of the eyes. The introduced interface provides all required commands to translate, rotate, open or close the gripper with the definition of different control modes. In each mode, different commands are provided and direct gaze direction of the user is applied to generate continuous robot commands. To distinguish between natural inspection eye movements and the eye movements that intent to control the robot arm, dynamic command areas are proposed. The dynamic command areas are defined around the robot gripper and are updated with its movements. To provide a direct interaction of the user, gaze gestures and states of the eyes are used to switch between different control modes. For the purpose of this thesis, two versions of the above-introduced HRI were developed. In the first version of the HRI, only two simple gaze gestures and two states of the eye (closed eyes and eye winking) are used for switching. In the second version, instead of the two simple gestures, four complex gaze gestures were applied and the positions of the dynamic command areas were optimized. The complex gaze gestures enable the user to switch directly from initial mode to the desired control mode. These gestures are flexible and can be generated directly in the robot environments. For the recognition of complex gaze gestures, a novel algorithm based on Dynamic Time Warping (DTW) is proposed. The results of the studies conducted with both HRIs confirmed their feasibility and showed the high potential of the proposed interfaces as hands-free interfaces. Furthermore, the results of subjective and objective measurements showed that the usability of the interface with simple gaze gestures had been improved with the integration of complex gaze gestures and the new positions of the dynamic command areas

Design And Development Of A Social Robotic Head - Dorothy

Master'sMASTER OF ENGINEERIN

Automatic 3D Facial Performance Acquisition and Animation using Monocular Videos

Facial performance capture and animation is an essential component of many applications such as movies, video games, and virtual environments. Video-based facial performance capture is particularly appealing as it offers the lowest cost and the potential use of legacy sources and uncontrolled videos. However, it is also challenging because of complex facial movements at different scales, ambiguity caused by the loss of depth information, and a lack of discernible features on most facial regions. Unknown lighting conditions and camera parameters further complicate the problem. This dissertation explores the video-based 3D facial performance capture systems that use a single video camera, overcome the challenges aforementioned, and produce accurate and robust reconstruction results. We first develop a novel automatic facial feature detection/tracking algorithm that accurately locates important facial features across the entire video sequence, which are then used for 3D pose and facial shape reconstruction. The key idea is to combine the respective powers of local detection, spatial priors for facial feature locations, Active Appearance Models (AAMs), and temporal coherence for facial feature detection. The algorithm runs in realtime and is robust to large pose and expression variations and occlusions. We then present an automatic high-fidelity facial performance capture system that works on monocular videos. It uses the detected facial features along with multilinear facial models to reconstruct 3D head poses and large-scale facial deformation, and uses per-pixel shading cues to add fine-scale surface details such as emerging or disappearing wrinkles and folds. We iterate the reconstruction procedure on large-scale facial geometry and fine-scale facial details to improve the accuracy of facial reconstruction. We further improve the accuracy and efficiency of the large-scale facial performance capture by introducing a local binary feature based 2D feature regression and a convolutional neural network based pose and expression regression, and complement it with an efficient 3D eye gaze tracker to achieve realtime 3D eye gaze animation. We have tested our systems on various monocular videos, demonstrating the accuracy and robustness under a variety of uncontrolled lighting conditions and overcoming significant shape differences across individuals

Eye Tracking: A Perceptual Interface for Content Based Image Retrieval

In this thesis visual search experiments are devised to explore the feasibility of an eye gaze driven search mechanism. The thesis first explores gaze behaviour on images possessing different levels of saliency. Eye behaviour was predominantly attracted by salient locations, but appears to also require frequent reference to non-salient background regions which indicated that information from scan paths might prove useful for image search. The thesis then specifically investigates the benefits of eye tracking as an image retrieval interface in terms of speed relative to selection by mouse, and in terms of the efficiency of eye tracking mechanisms in the task of retrieving target images. Results are analysed using ANOVA and significant findings are discussed. Results show that eye selection was faster than a computer mouse and experience gained during visual tasks carried out using a mouse would benefit users if they were subsequently transferred to an eye tracking system. Results on the image retrieval experiments show that users are able to navigate to a target image within a database confirming the feasibility of an eye gaze driven search mechanism. Additional histogram analysis of the fixations, saccades and pupil diameters in the human eye movement data revealed a new method of extracting intentions from gaze behaviour for image search, of which the user was not aware and promises even quicker search performances. The research has two implications for Content Based Image Retrieval: (i) improvements in query formulation for visual search and (ii) new methods for visual search using attentional weighting. Futhermore it was demonstrated that users are able to find target images at sufficient speeds indicating that pre-attentive activity is playing a role in visual search. A current review of eye tracking technology, current applications, visual perception research, and models of visual attention is discussed. A review of the potential of the technology for commercial exploitation is also presented

Perception and Orientation in Minimally Invasive Surgery

During the last two decades, we have seen a revolution in the way that we perform abdominal surgery with increased reliance on minimally invasive techniques. This paradigm shift has come at a rapid pace, with laparoscopic surgery now representing the gold standard for many surgical procedures and further minimisation of invasiveness being seen with the recent clinical introduction of novel techniques such as single-incision laparoscopic surgery and natural orifice translumenal endoscopic surgery. Despite the obvious benefits conferred on the patient in terms of morbidity, length of hospital stay and post-operative pain, this paradigm shift comes at a significantly higher demand on the surgeon, in terms of both perception and manual dexterity. The issues involved include degradation of sensory input to the operator compared to conventional open surgery owing to a loss of three-dimensional vision through the use of the two-dimensional operative interface, and decreased haptic feedback from the instruments. These changes have led to a much higher cognitive load on the surgeon and a greater risk of operator disorientation leading to potential surgical errors. This thesis represents a detailed investigation of disorientation in minimally invasive surgery. In this thesis, eye tracking methodology is identified as the method of choice for evaluating behavioural patterns during orientation. An analysis framework is proposed to profile orientation behaviour using eye tracking data validated in a laboratory model. This framework is used to characterise and quantify successful orientation strategies at critical stages of laparoscopic cholecystectomy and furthermore use these strategies to prove that focused teaching of this behaviour in novices can significantly increase performance in this task. Orientation strategies are then characterised for common clinical scenarios in natural orifice translumenal endoscopic surgery and the concept of image saliency is introduced to further investigate the importance of specific visual cues associated with effective orientation. Profiling of behavioural patterns is related to performance in orientation and implications on education and construction of smart surgical robots are drawn. Finally, a method for potentially decreasing operator disorientation is investigated in the form of endoscopic horizon stabilization in a simulated operative model for transgastric surgery. The major original contributions of this thesis include: Validation of a profiling methodology/framework to characterise orientation behaviour Identification of high performance orientation strategies in specific clinical scenarios including laparoscopic cholecystectomy and natural orifice translumenal endoscopic surgery Evaluation of the efficacy of teaching orientation strategies Evaluation of automatic endoscopic horizon stabilization in natural orifice translumenal endoscopic surgery The impact of the results presented in this thesis, as well as the potential for further high impact research is discussed in the context of both eye tracking as an evaluation tool in minimally invasive surgery as well as implementation of means to combat operator disorientation in a surgical platform. The work also provides further insight into the practical implementation of computer-assistance and technological innovation in future flexible access surgical platforms

Improving sports vision diagnostics using wireless electro-oculography

Includes bibliographical references (leaves 91-94).Traditional techniques such as eye hand coordination tests have difficulty isolating the pure eye movement data as their results are reliant on subjects performing physical responses to visual stimuli. What is needed in this field is diagnostic equipment that can give information on eye motion alone. Electro-Oculography was investigated as a form of eye tracking as it is an inexpensive, simple, underdeveloped and primary method of eye tracking that required low data transmission rates

NASA patent abstracts bibliography: A continuing bibliography. Section 1: Abstracts (supplement 39)

Abstracts are provided for 154 patents and patent applications entered into the NASA scientific and technical information systems during the period Jan. 1991 through Jun. 1991. Each entry consists of a citation, an abstract, and in most cases, a key illustration selected from the patent or patent application

Cerebral Circulation

Diagnostics and diseases related to the cerebrovascular system are constantly evolving and updating. 3D augmented reality or quantification of cerebral perfusion are becoming important diagnostic tools in daily practice and the role of the cerebral venous system is being constantly revised considering new theories such as that of “the glymphatic system.” This book provides updates on models, diagnosis, and treatment of diseases of the cerebrovascular system

Aerospace medicine and biology. A continuing bibliography, supplement 207

This bibliography lists 184 reports, articles, and other documents introduced into the NASA scientific and technical information system in May 1980