Pupil Position by an Improved Technique of YOLO Network for Eye Tracking Application

This Eye gaze following is the real-time collection of information about a person's eye movements and the direction of their look. Eye gaze trackers are devices that measure the locations of the pupils to detect and track changes in the direction of the user's gaze. There are numerous applications for analyzing eye movements, from psychological studies to human-computer interaction-based systems and interactive robotics controls. Real-time eye gaze monitoring requires an accurate and reliable iris center localization technique. Deep learning technology is used to construct a pupil tracking approach for wearable eye trackers in this study. This pupil tracking method uses deep-learning You Only Look Once (YOLO) model to accurately estimate and anticipate the pupil's central location under conditions of bright, natural light (visible to the naked eye). Testing pupil tracking performance with the upgraded YOLOv7 results in an accuracy rate of 98.50% and a precision rate close to 96.34% using PyTorch

A Novel Authentication Method Using Multi-Factor Eye Gaze

A method for novel, rapid and robust one-step multi-factor authentication of a user is presented, employing multi-factor eye gaze. The mobile environment presents challenges that render the conventional password model obsolete. The primary goal is to offer an authentication method that competitively replaces the password, while offering improved security and usability. This method and apparatus combine the smooth operation of biometric authentication with the protection of knowledge based authentication to robustly authenticate a user and secure information on a mobile device in a manner that is easily used and requires no external hardware. This work demonstrates a solution comprised of a pupil segmentation algorithm, gaze estimation, and an innovative application that allows a user to authenticate oneself using gaze as the interaction medium

A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms

In this paper a review is presented of the research on eye gaze estimation techniques and applications, that has progressed in diverse ways over the past two decades. Several generic eye gaze use-cases are identified: desktop, TV, head-mounted, automotive and handheld devices. Analysis of the literature leads to the identification of several platform specific factors that influence gaze tracking accuracy. A key outcome from this review is the realization of a need to develop standardized methodologies for performance evaluation of gaze tracking systems and achieve consistency in their specification and comparative evaluation. To address this need, the concept of a methodological framework for practical evaluation of different gaze tracking systems is proposed.Comment: 25 pages, 13 figures, Accepted for publication in IEEE Access in July 201

Real-time eye gaze tracking for gaming design and consumer electronics systems

Real time face detection combined with eyegaze tracking can provide a means of user input into a gaming environment. Game and CE system designers can use facial and eye-gaze information in various ways to enhance UI design providing smarter modes of gameplay interaction and UI modalities that are sensitive to a users behaviors and mood. Here we review earlier approaches, using wearable sensors, or enhanced IR illumination. Our technique only requires video feed from a low-resolution user-facing camera. The algorithm is described and some comparative results on a range of embedded hardware are provided. The potential for using eye-gaze as a means of direct user input and improving the accuracy of estimation accordingly is also discussed.peer-reviewe

Real-time eye gaze tracking for gaming design and consumer electronics systems

Real time face detection combined with eyegaze tracking can provide a means of user input into a gaming environment. Game and CE system designers can use facial and eye-gaze information in various ways to enhance UI design providing smarter modes of gameplay interaction and UI modalities that are sensitive to a users behaviors and mood. Here we review earlier approaches, using wearable sensors, or enhanced IR illumination. Our technique only requires video feed from a low-resolution user-facing camera. The algorithm is described and some comparative results on a range of embedded hardware are provided. The potential for using eye-gaze as a means of direct user input and improving the accuracy of estimation accordingly is also discussed

Real-Time Eye Gaze Tracking for Gaming Design and Consumer Electronics Systems

Real time face detection combined with eye-gaze tracking can provide a means of user input into a gaming environment. Game and CE system designers can use facial and eye-gaze information in various ways to enhance UI design providing smarter modes of gameplay interaction and UI modalities that are sensitive to a users behaviors and mood. Here we review earlier approaches, using wearable sensors, or enhanced IR illumination. Our technique only requires video feed from a low-resolution user-facing camera. The algorithm is described and some comparative results on a range of embedded hardware are provided. The potential for using eye-gaze as a means of direct user input and improving the accuracy of estimation accordingly is also discussed

Natural interactions: an application for gestural hands recognition

Dissertação de Mestrado em Desenvolvimento de Software e Sistemas Interativos apresentada à Escola Superior de Tecnologia do Instituto Politécnico de Castelo Branco.Este trabalho apresenta um sistema para o desenvolvimento de novas interfaces homem-máquina com foco no reconhecimento de gestos estáticos de mãos humanas. A proposta é auxiliar o acesso a certos objetos para o ocupante de uma cadeira de rodas inteligente, a fim de facilitar a sua vida diária. A metodologia proposta baseia-se no uso de processos computacionais simples e de hardware de baixo custo. O seu desenvolvimento envolve uma abordagem completa aos problemas de visão computacional, com base nas etapas da captura de imagem de vídeo, segmentação de imagens, extração de características, reconhecimento e classificação de padrões. A importância deste trabalho relaciona-se com a necessidade de construir novos modelos de interação que permitam, de uma forma natural e intuitiva, a simplificação da vida quotidiana de uma pessoa com dificuldades motoras.Abstract: This thesis presents a system for the development of new human-machine interfaces focused on static gestures recognition of human hands. The proposal is to give access to certain objects to the occupant of an intelligent wheelchair in order to facilitate their daily life. The proposed methodology relies on the use of simple computational processes and low-cost hardware. Its development involves a comprehensive approach to the problems of computer vision, based on the steps of the video image capture, image segmentation, feature extraction, pattern recognition and classification. The importance of this work relates to the need to build new models of interaction that allow, in a natural and intuitive way, to simplify the daily life of a disable person

Automatic Age Estimation From Real-World And Wild Face Images By Using Deep Neural Networks

Automatic age estimation from real-world and wild face images is a challenging task and has an increasing importance due to its wide range of applications in current and future lifestyles. As a result of increasing age specific human-computer interactions, it is expected that computerized systems should be capable of estimating the age from face images and respond accordingly. Over the past decade, many research studies have been conducted on automatic age estimation from face images. In this research, new approaches for enhancing age classification of a person from face images based on deep neural networks (DNNs) are proposed. The work shows that pre-trained CNNs which were trained on large benchmarks for different purposes can be retrained and fine-tuned for age estimation from unconstrained face images. Furthermore, an algorithm to reduce the dimension of the output of the last convolutional layer in pre-trained CNNs to improve the performance is developed. Moreover, two new jointly fine-tuned DNNs frameworks are proposed. The first framework fine-tunes tow DNNs with two different feature sets based on the element-wise summation of their last hidden layer outputs. While the second framework fine-tunes two DNNs based on a new cost function. For both frameworks, each has two DNNs, the first DNN is trained by using facial appearance features that are extracted by a well-trained model on face recognition, while the second DNN is trained on features that are based on the superpixels depth and their relationships. Furthermore, a new method for selecting robust features based on the power of DNN and ??21-norm is proposed. This method is mainly based on a new cost function relating the DNN and the L21 norm in one unified framework. To learn and train this unified framework, the analysis and the proof for the convergence of the new objective function to solve minimization problem are studied. Finally, the performance of the proposed jointly fine-tuned networks and the proposed robust features are used to improve the age estimation from the facial images. The facial features concatenated with their corresponding robust features are fed to the first part of both networks and the superpixels features concatenated with their robust features are fed to the second part of the network. Experimental results on a public database show the effectiveness of the proposed methods and achieved the state-of-art performance on a public database

Automatic 3D Facial Performance Acquisition and Animation using Monocular Videos

Facial performance capture and animation is an essential component of many applications such as movies, video games, and virtual environments. Video-based facial performance capture is particularly appealing as it offers the lowest cost and the potential use of legacy sources and uncontrolled videos. However, it is also challenging because of complex facial movements at different scales, ambiguity caused by the loss of depth information, and a lack of discernible features on most facial regions. Unknown lighting conditions and camera parameters further complicate the problem. This dissertation explores the video-based 3D facial performance capture systems that use a single video camera, overcome the challenges aforementioned, and produce accurate and robust reconstruction results. We first develop a novel automatic facial feature detection/tracking algorithm that accurately locates important facial features across the entire video sequence, which are then used for 3D pose and facial shape reconstruction. The key idea is to combine the respective powers of local detection, spatial priors for facial feature locations, Active Appearance Models (AAMs), and temporal coherence for facial feature detection. The algorithm runs in realtime and is robust to large pose and expression variations and occlusions. We then present an automatic high-fidelity facial performance capture system that works on monocular videos. It uses the detected facial features along with multilinear facial models to reconstruct 3D head poses and large-scale facial deformation, and uses per-pixel shading cues to add fine-scale surface details such as emerging or disappearing wrinkles and folds. We iterate the reconstruction procedure on large-scale facial geometry and fine-scale facial details to improve the accuracy of facial reconstruction. We further improve the accuracy and efficiency of the large-scale facial performance capture by introducing a local binary feature based 2D feature regression and a convolutional neural network based pose and expression regression, and complement it with an efficient 3D eye gaze tracker to achieve realtime 3D eye gaze animation. We have tested our systems on various monocular videos, demonstrating the accuracy and robustness under a variety of uncontrolled lighting conditions and overcoming significant shape differences across individuals