A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms

In this paper a review is presented of the research on eye gaze estimation techniques and applications, that has progressed in diverse ways over the past two decades. Several generic eye gaze use-cases are identified: desktop, TV, head-mounted, automotive and handheld devices. Analysis of the literature leads to the identification of several platform specific factors that influence gaze tracking accuracy. A key outcome from this review is the realization of a need to develop standardized methodologies for performance evaluation of gaze tracking systems and achieve consistency in their specification and comparative evaluation. To address this need, the concept of a methodological framework for practical evaluation of different gaze tracking systems is proposed.Comment: 25 pages, 13 figures, Accepted for publication in IEEE Access in July 201

Quick Models for Saccade Amplitude Prediction

This paper presents a new saccade amplitude prediction model. The model is based on a Kalman filter and regression analysis. The aim of the model is to predict a saccade’s am-plitude extremely quickly, i.e., within two eye position samples at the onset of a saccade. Specifically, the paper explores saccade amplitude prediction considering one or two sam-ples at the onset of a saccade. The models’ prediction performance was tested with 35 subjects. The amplitude accuracy results yielded approximately 5.26° prediction error, while the error for direction prediction was 5.3% for the first sample model and 1.5% for the two samples model. The practical use of the proposed model lays in the area of real-time gaze-contingent compression and extreme eye-gaze aware interaction applications. The paper provides theoretical evaluation of the benefits of saccade amplitude prediction to the gaze-contingent multimedia compression, estimating a 21% improvement in com-pression for short network delays

A Novel Authentication Method Using Multi-Factor Eye Gaze

A method for novel, rapid and robust one-step multi-factor authentication of a user is presented, employing multi-factor eye gaze. The mobile environment presents challenges that render the conventional password model obsolete. The primary goal is to offer an authentication method that competitively replaces the password, while offering improved security and usability. This method and apparatus combine the smooth operation of biometric authentication with the protection of knowledge based authentication to robustly authenticate a user and secure information on a mobile device in a manner that is easily used and requires no external hardware. This work demonstrates a solution comprised of a pupil segmentation algorithm, gaze estimation, and an innovative application that allows a user to authenticate oneself using gaze as the interaction medium

편광 다중화를 이용하여 향상된 기능을 제공하는 도파관 기반의 근안 디스플레이

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2021. 2. 이병호.This dissertation presents the studies on the optical design method that enhances the display performance of see-through waveguide-based near-eye displays (WNEDs) using the polarization multiplexing technique. The studies focus on the strategies to improve the crucial display performances without compromising a small form factor, the most attractive merit of the WNEDs. To achieve this goal, thin and lightweight polarization-dependent optical elements are devised and employed in the WNED structure. The polarization-dependent devices can allow multiple optical functions or optical paths depending on the polarization state of the input beam, which can break through the limitation of the waveguide system with the polarization multiplexing. To realize the function-selective eyepiece for AR applications, the proposed devices should operate as an optically transparent window for the real scene while performing specific optical functions for the virtual image. The proposed devices are manufactured in a combination structure in which polarization-dependent optical elements are stacked. The total thickness of the stacked structure is about 1 mm, and it can be attached to the waveguide surface without conspicuously increasing the form factor of the optical system. Using the proposed polarization-dependent devices, the author proposes three types of novel WNED systems with enhanced performance. First, the author suggests a compact WNED with dual focal planes. Conventional WNEDs have an inherent limitation that the focal plane of the virtual image is at an infinite distance because they extract a stream of collimated light at the out-coupler. By using the polarization-dependent eyepiece lens, an additional focal plane can be generated with the polarization multiplexing in addition to infinity depth. The proposed configuration can provide comfortable AR environments by alleviating visual fatigue caused by vergence-accommodation conflict. Second, the novel WNED configuration with extended field-of-view (FOV) is presented. In the WNEDs, the maximum allowable FOV is determined by the material properties of the diffraction optics and the substrate. By using the polarization-dependent steering combiner, the FOV can be extended up to two times, which can provide more immersive AR experiences. In addition, this dissertation demonstrates that the distortion for the real scene caused by the stacked structure cannot severely disturb the image quality, considering the acuity of human vision. Lastly, the author presents a retinal projection-based WNED with switchable viewpoints by simultaneously adopting the polarization-dependent lens and grating. The proposed system can convert the viewpoint according to the position of the eye pupil without mechanical movement. The polarization-dependent viewpoint switching can resolve the inherent problem of a narrow eyebox in retinal projection displays without employing the bulky optics for mechanical movement. In conclusion, the dissertation presents the practical optical design and detailed analysis for enhanced WNED based on the polarization multiplexing technique through various simulations and experiments. The proposed approaches are expected to be utilized as an innovative solution for compact wearable displays.본 박사학위 논문에서는 편광 다중화 기법을 이용하여 도파관 기반의 증강현실 근안 디스플레이의 성능을 향상시키는 광학 설계 및 이에 대한 분석에 대해 논의한다. 본 연구는 도파관 기반 근안 디스플레이의 가장 큰 장점인 소형 폼 팩터를 유지하면서 디스플레이 성능을 개선하는 것에 중점을 둔다. 이를 위해 기존 광학 소자에 비해 매우 가볍고 얇은 편광 의존형 결합기 광학 소자가 새롭게 제안되며, 이는 입사광의 편광 상태에 따라 독립적인 광 경로 제어를 가능케 하여 편광 다중화를 통해 향상된 성능을 제공 할 수 있다. 또한 실제 영상의 빛은 그대로 투과 시킴으로써 증강현실을 위한 소자로 활용 가능하다. 본 연구에서 제안하는 편광 의존형 결합기 광학 소자는 기하학적 위상(geometric phase, GP)에 기반하여 동작한다. GP 기반의 광학소자가 서로 직교하는 원형 편광 입사광에 대해 상보적인 기능을 수행하는 것을 이용하여, 두 개 이상의 GP 소자와 편광 제어를 위한 광학 필름들을 중첩 시킴으로써 증강현실 결합기 광학 소자를 구현할 수 있다. 이들 광학소자는 매우 얇기 때문에, 본 연구에서 제작된 편광 의존형 결합기 광학 소자의 총 두께는 1 mm 수준으로 폼 팩터 제약으로부터 자유롭다. 또한 평평한 필름 형태이므로, 평판형 도파관에 부착하기 쉽다는 이점을 지닌다. 고안된 편광 의존형 결합기 광학 소자를 사용하여 세 가지 유형의 새로운 도파관 기반의 근안 디스플레이 구조를 제안한다. 첫 번째는 입사광의 편광 상태에 따라 투명 광학 창 또는 오목 렌즈로 작동하는 편광 의존형 결합기 렌즈를 적용하여 가상 영상에 대해 이중 초점면을 제공하는 시스템이다. 제안된 구조는 기존의 도파관 기반 근안 디스플레이가 무한대 위치에 단일 초점면을 제공함으로써 발생하는 시각적 피로 및 흐릿한 증강현실 영상의 문제를 완화할 수 있다. 두 번째로는 입사광의 편광 상태에 따라 광 경로를 좌측 또는 우측으로 제어할 수 있는 편광 격자를 활용하여 가상 영상의 시야각을 기존보다 최대 2배까지 확장할 수 있는 시스템을 제안한다. 이는 단일 도파관 기반 근안 디스플레이에서 영상 결합기 (imaging combiner)로 활용되는 회절 소자의 설계 변수에 의해 제한되는 시야각 한계점을 돌파할 수 있는 구조로 컴팩트한 폼 팩터로 더욱 몰입감 있는 대화면 증강현실 영상을 제공할 수 있다. 마지막으로 위에서 제안된 두 가지 편광 의존형 광학 소자를 모두 사용하여 시점 전환이 가능한 도파관 기반의 망막 투사형 디스플레이 구조를 제안한다. 편광 다중화를 통해 다중 초점들을 선택적으로 활성화함으로써, 확장된 시청영역을 제공하는 동시에 동공 크기 변화 또는 움직임에 의한 이중 영상 문제를 완화할 수 있다. 또한 기계적 움직임 없이 시점 간의 고속 전환이 가능하다는 장점을 지니고 있다. 본 박사학위 논문에서 제시한 편광 다중화를 활용한 새로운 결합기 광학 소자 및 광학 구조들은 도파관 기반 근안 디스플레이의 향상된 성능을 제공하는 해결책 및 새로운 가능성으로 제시할 수 있을 것이라 기대된다.Abstract i Contents iii List of Tables vi List of Figures vii Chapter. 1 Introduction 1 1.1 Augmented reality near-eye display 1 1.2 Key performance parameters of near-eye displays 4 1.3 Basic scheme of waveguide-based near-eye displays 22 1.4 Motivation and purpose of this dissertation 33 1.5 Scope and organization 37 Chapter 2 Dual-focal waveguide-based near-eye display using polarization-dependent combiner lens 39 2.1 Introduction 39 2.2 Optical design for polarization-dependent combiner lens 42 2.2.1 Design and principle of polarization-dependent combiner lens 42 2.2.2 Prototype implementation 48 2.3 Waveguide-based augmented reality near-eye display with dual-focal plane using polarization-dependent combiner lens 51 2.3.1 Implementation of the prototype and experimental results 51 2.3.2 Performance analysis and discussion 57 2.4 Conclusion 69 Chapter 3 Extended-field-of-view waveguide-based near-eye display via polarization-dependent steering combiner 70 3.1 Introduction 70 3.2 Optical design for polarization-dependent steering combiner 73 3.2.1 Principle of polarization grating 73 3.2.2 Principle of polarization-dependent steering combiner 76 3.2.3 Analysis and verification experiment for real-scene distortion 77 3.3 Waveguide-based augmented reality near-eye display with extended-field-of-view 81 3.3.1 Field-of-view for volume grating based waveguide technique 81 3.3.2 Implementation of the prototype and experimental results 84 3.3.3 Performances analysis and discussion 87 3.4 Conclusion 92 Chapter 4 Viewpoint switchable retinal-projection-based near-eye display with waveguide configuration 93 4.1 Introduction 93 4.2 Polarization-dependent switchable eyebox 97 4.2.1 Optical devices for polarization-dependent switching of viewpoints 97 4.2.2 System configuration for proposed method 100 4.2.3 Design of waveguide and imaging combiner 105 4.3 Compact retinal projection-based near-eye display with switchable viewpoints via waveguide configuration 114 4.3.1 Implementation of the prototype and experimental results 114 4.3.2 Performance analysis and discussion 118 4.4 Conclusion 122 Chapter 5. Conclusion 123 Bibliography 127 Appendix 135Docto