편광 다중화를 이용하여 향상된 기능을 제공하는 도파관 기반의 근안 디스플레이

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2021. 2. 이병호.This dissertation presents the studies on the optical design method that enhances the display performance of see-through waveguide-based near-eye displays (WNEDs) using the polarization multiplexing technique. The studies focus on the strategies to improve the crucial display performances without compromising a small form factor, the most attractive merit of the WNEDs. To achieve this goal, thin and lightweight polarization-dependent optical elements are devised and employed in the WNED structure. The polarization-dependent devices can allow multiple optical functions or optical paths depending on the polarization state of the input beam, which can break through the limitation of the waveguide system with the polarization multiplexing. To realize the function-selective eyepiece for AR applications, the proposed devices should operate as an optically transparent window for the real scene while performing specific optical functions for the virtual image. The proposed devices are manufactured in a combination structure in which polarization-dependent optical elements are stacked. The total thickness of the stacked structure is about 1 mm, and it can be attached to the waveguide surface without conspicuously increasing the form factor of the optical system. Using the proposed polarization-dependent devices, the author proposes three types of novel WNED systems with enhanced performance. First, the author suggests a compact WNED with dual focal planes. Conventional WNEDs have an inherent limitation that the focal plane of the virtual image is at an infinite distance because they extract a stream of collimated light at the out-coupler. By using the polarization-dependent eyepiece lens, an additional focal plane can be generated with the polarization multiplexing in addition to infinity depth. The proposed configuration can provide comfortable AR environments by alleviating visual fatigue caused by vergence-accommodation conflict. Second, the novel WNED configuration with extended field-of-view (FOV) is presented. In the WNEDs, the maximum allowable FOV is determined by the material properties of the diffraction optics and the substrate. By using the polarization-dependent steering combiner, the FOV can be extended up to two times, which can provide more immersive AR experiences. In addition, this dissertation demonstrates that the distortion for the real scene caused by the stacked structure cannot severely disturb the image quality, considering the acuity of human vision. Lastly, the author presents a retinal projection-based WNED with switchable viewpoints by simultaneously adopting the polarization-dependent lens and grating. The proposed system can convert the viewpoint according to the position of the eye pupil without mechanical movement. The polarization-dependent viewpoint switching can resolve the inherent problem of a narrow eyebox in retinal projection displays without employing the bulky optics for mechanical movement. In conclusion, the dissertation presents the practical optical design and detailed analysis for enhanced WNED based on the polarization multiplexing technique through various simulations and experiments. The proposed approaches are expected to be utilized as an innovative solution for compact wearable displays.본 박사학위 논문에서는 편광 다중화 기법을 이용하여 도파관 기반의 증강현실 근안 디스플레이의 성능을 향상시키는 광학 설계 및 이에 대한 분석에 대해 논의한다. 본 연구는 도파관 기반 근안 디스플레이의 가장 큰 장점인 소형 폼 팩터를 유지하면서 디스플레이 성능을 개선하는 것에 중점을 둔다. 이를 위해 기존 광학 소자에 비해 매우 가볍고 얇은 편광 의존형 결합기 광학 소자가 새롭게 제안되며, 이는 입사광의 편광 상태에 따라 독립적인 광 경로 제어를 가능케 하여 편광 다중화를 통해 향상된 성능을 제공 할 수 있다. 또한 실제 영상의 빛은 그대로 투과 시킴으로써 증강현실을 위한 소자로 활용 가능하다. 본 연구에서 제안하는 편광 의존형 결합기 광학 소자는 기하학적 위상(geometric phase, GP)에 기반하여 동작한다. GP 기반의 광학소자가 서로 직교하는 원형 편광 입사광에 대해 상보적인 기능을 수행하는 것을 이용하여, 두 개 이상의 GP 소자와 편광 제어를 위한 광학 필름들을 중첩 시킴으로써 증강현실 결합기 광학 소자를 구현할 수 있다. 이들 광학소자는 매우 얇기 때문에, 본 연구에서 제작된 편광 의존형 결합기 광학 소자의 총 두께는 1 mm 수준으로 폼 팩터 제약으로부터 자유롭다. 또한 평평한 필름 형태이므로, 평판형 도파관에 부착하기 쉽다는 이점을 지닌다. 고안된 편광 의존형 결합기 광학 소자를 사용하여 세 가지 유형의 새로운 도파관 기반의 근안 디스플레이 구조를 제안한다. 첫 번째는 입사광의 편광 상태에 따라 투명 광학 창 또는 오목 렌즈로 작동하는 편광 의존형 결합기 렌즈를 적용하여 가상 영상에 대해 이중 초점면을 제공하는 시스템이다. 제안된 구조는 기존의 도파관 기반 근안 디스플레이가 무한대 위치에 단일 초점면을 제공함으로써 발생하는 시각적 피로 및 흐릿한 증강현실 영상의 문제를 완화할 수 있다. 두 번째로는 입사광의 편광 상태에 따라 광 경로를 좌측 또는 우측으로 제어할 수 있는 편광 격자를 활용하여 가상 영상의 시야각을 기존보다 최대 2배까지 확장할 수 있는 시스템을 제안한다. 이는 단일 도파관 기반 근안 디스플레이에서 영상 결합기 (imaging combiner)로 활용되는 회절 소자의 설계 변수에 의해 제한되는 시야각 한계점을 돌파할 수 있는 구조로 컴팩트한 폼 팩터로 더욱 몰입감 있는 대화면 증강현실 영상을 제공할 수 있다. 마지막으로 위에서 제안된 두 가지 편광 의존형 광학 소자를 모두 사용하여 시점 전환이 가능한 도파관 기반의 망막 투사형 디스플레이 구조를 제안한다. 편광 다중화를 통해 다중 초점들을 선택적으로 활성화함으로써, 확장된 시청영역을 제공하는 동시에 동공 크기 변화 또는 움직임에 의한 이중 영상 문제를 완화할 수 있다. 또한 기계적 움직임 없이 시점 간의 고속 전환이 가능하다는 장점을 지니고 있다. 본 박사학위 논문에서 제시한 편광 다중화를 활용한 새로운 결합기 광학 소자 및 광학 구조들은 도파관 기반 근안 디스플레이의 향상된 성능을 제공하는 해결책 및 새로운 가능성으로 제시할 수 있을 것이라 기대된다.Abstract i Contents iii List of Tables vi List of Figures vii Chapter. 1 Introduction 1 1.1 Augmented reality near-eye display 1 1.2 Key performance parameters of near-eye displays 4 1.3 Basic scheme of waveguide-based near-eye displays 22 1.4 Motivation and purpose of this dissertation 33 1.5 Scope and organization 37 Chapter 2 Dual-focal waveguide-based near-eye display using polarization-dependent combiner lens 39 2.1 Introduction 39 2.2 Optical design for polarization-dependent combiner lens 42 2.2.1 Design and principle of polarization-dependent combiner lens 42 2.2.2 Prototype implementation 48 2.3 Waveguide-based augmented reality near-eye display with dual-focal plane using polarization-dependent combiner lens 51 2.3.1 Implementation of the prototype and experimental results 51 2.3.2 Performance analysis and discussion 57 2.4 Conclusion 69 Chapter 3 Extended-field-of-view waveguide-based near-eye display via polarization-dependent steering combiner 70 3.1 Introduction 70 3.2 Optical design for polarization-dependent steering combiner 73 3.2.1 Principle of polarization grating 73 3.2.2 Principle of polarization-dependent steering combiner 76 3.2.3 Analysis and verification experiment for real-scene distortion 77 3.3 Waveguide-based augmented reality near-eye display with extended-field-of-view 81 3.3.1 Field-of-view for volume grating based waveguide technique 81 3.3.2 Implementation of the prototype and experimental results 84 3.3.3 Performances analysis and discussion 87 3.4 Conclusion 92 Chapter 4 Viewpoint switchable retinal-projection-based near-eye display with waveguide configuration 93 4.1 Introduction 93 4.2 Polarization-dependent switchable eyebox 97 4.2.1 Optical devices for polarization-dependent switching of viewpoints 97 4.2.2 System configuration for proposed method 100 4.2.3 Design of waveguide and imaging combiner 105 4.3 Compact retinal projection-based near-eye display with switchable viewpoints via waveguide configuration 114 4.3.1 Implementation of the prototype and experimental results 114 4.3.2 Performance analysis and discussion 118 4.4 Conclusion 122 Chapter 5. Conclusion 123 Bibliography 127 Appendix 135Docto

비등방성 광학 소자를 이용한 광 시야각 근안 디스플레이

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 전기·컴퓨터공학부, 2019. 2. 이병호.Near-eye display is considered as a promising display technique to realize augmented reality by virtue of its high sense of immersion and user-friendly interface. Among the important performances of near-eye display, a field of view is the most crucial factor for providing a seamless and immersive experience for augmented reality. In this dissertation, a transmissive eyepiece is devised instead of a conventional reflective eyepiece and it is discussed how to widen the field of view without loss of additional system performance. In order to realize the transmissive eyepiece, the eyepiece should operate lens to virtual information and glass to real-world scene. Polarization multiplexing technique is used to implement the multi-functional optical element, and anisotropic optical elements are used as material for multi-functional optical element. To demonstrate the proposed idea, an index-matched anisotropic crystal lens has been presented that reacts differently depending on polarization. With the combination of isotropic material and anisotropic crystal, the index-matched anisotropic crystal lens can be the transmissive eyepiece and achieve the large field of view. Despite the large field of view by the index-matched anisotropic crystal lens, many problems including form factor still remain to be solved. In order to overcome the limitations of conventional optics, a metasurface is adopted to the augmented reality application. With a stunning optical performance of the metasurface, a see-through metasurface lens is proposed and designed for implementing wide field of view near-eye display. The proposed novel eyepieces are expected to be an initiative study not only improving the specification of the existing near-eye display but opening the way for a next generation near-eye display.근안 디스플레이는 높은 몰입감과 사용자 친화적인 인터페이스로 인해 증강 현실을 구현하는 가장 효과적인 기술로 최근 활발한 연구가 계속되고 있다. 이러한 근안 디스플레이의 중요한 성능 중 시야각은 매끄럽고 몰입감 있는 경험을 사용자에게 전해줌으로써 가장 중요한 광학적 평가지표 중에 하나이다. 본 논문에서는 기존의 반사형 아이피스 (eyepiece) 를 대신하는 투과형 아이피스를 제안한다. 이러한 투과형 아이피스를 구현하기 위해서는 외부 정보에 대해서는 투명한 유리와 같이 투과시키며, 동시에 가상 정보는 렌즈로 작동하여 먼 거리에 띄울 수 있는 광학소자를 개발하여야 한다. 이러한 투과형 아이피스를 구현하기 위해서 편광에 따라 다르게 반응하는 굴절률 정합 이방성 결정 렌즈 (index-matched anisotropic crystal lens) 를 제안하였다. 이방성 결정 구조 (anisotropic crystal)로 이루어진 렌즈와 이를 둘러싼 등방성 물질 (isotropic crytal) 로 이루어진 굴절률 정합 이방성 결정 렌즈는 편광에 따라 다르게 작동한다. 이러한 투과형 아이피스는 기존의 근안 디스플레이에 비해 넓은 시야각을 제공할 수 있지만 이방성 결정 구조의 낮은 굴절률 차이로 인해 시스템의 크기가 커지는 단점을 가지고 있다. 본 논문에서는 이러한 단점을 개선하기 위해 메타 표면을 증강 현실 디스플레이 분야에 적용하였다. 메타 표면의 기존 광학 소자를 능가하는 놀라운 광학 성능을 이용하여 넓은 시야각을 가지는 근안 디스플레이를 구현하기 위해 투명 메타 렌즈를 제안하였다. 편광에 따라 다르게 반응하는 투명 메타렌즈는 넓은 시야각과 경량화 시스템 구현이 가능하며 이를 입증하기 위해 투명 메타렌즈의 설계 방법 뿐 아니라 실제 구현을 통한 가능성을 입증하였다. 이러한 새로운 아이피스에 대한 개념은 기존의 근안 디스플레이의 사양 개선에 유용하게 사용될 뿐 아니라 차세대 근안 디스플레이를 위한 선도적인 역할을 할 것으로 기대된다.Abstract Contents List of Tables List of Figures Near-eye displays with wide field of view using anisotropic optical elements Chapter 1 Introduction 1.1 Near-eye displays for augmented reality 1.2 Optical performances of near-eye display 1.3 State-of-the-arts of near-eye display 1.4 Motivation and contribution of this dissertation Chapter 2 Transmissive eyepiece for wide field of view near-eye display 2.1 Transmissive eyepiece for near-eye display Chapter 3 Near-eye display using index-matched anisotropic crystal lens 3.1 Introduction 3.2 Index-matched anisotropic crystal lens 3.2.1 Principle of the index-matched anisotropic crystal lens 3.2.2 Aberration analysis of index-matched anisotropic crystal lens 3.2.3 Implementation 3.3 Near-eye displays using index-matched anisotropic crystal lens 3.3.1 Near-eye display using index-matched anisotropic crystal lens 3.3.2 Flat panel type near-eye display using IMACL 3.3.3 Polarization property of transparent screen 3.4 Conclusion Chapter 4 Near-eye display using metasurface lens 4.1 Introduction 4.2 See-through metasurface lens 4.2.1 Metasurface lens 4.3 Full-color near-eye display using metasurface lens 4.3.1 Full-color near-eye display using metasurface lens 4.3.2 Holographic near-eye display using metasurface lens for aberration compensation 4.4 Conclusion Chapter 5 Conclusion Bibliography AppendixDocto

Deformable Beamsplitters: Enhancing Perception with Wide Field of View, Varifocal Augmented Reality Displays

An augmented reality head-mounted display with full environmental awareness could present data in new ways and provide a new type of experience, allowing seamless transitions between real life and virtual content. However, creating a light-weight, optical see-through display providing both focus support and wide field of view remains a challenge. This dissertation describes a new dynamic optical element, the deformable beamsplitter, and its applications for wide field of view, varifocal, augmented reality displays. Deformable beamsplitters combine a traditional deformable membrane mirror and a beamsplitter into a single element, allowing reflected light to be manipulated by the deforming membrane mirror, while transmitted light remains unchanged. This research enables both single element optical design and correct focus while maintaining a wide field of view, as demonstrated by the description and analysis of two prototype hardware display systems which incorporate deformable beamsplitters. As a user changes the depth of their gaze when looking through these displays, the focus of virtual content can quickly be altered to match the real world by simply modulating air pressure in a chamber behind the deformable beamsplitter; thus ameliorating vergence–accommodation conflict. Two user studies verify the display prototypes’ capabilities and show the potential of the display in enhancing human performance at quickly perceiving visual stimuli. This work shows that near-eye displays built with deformable beamsplitters allow for simple optical designs that enable wide field of view and comfortable viewing experiences with the potential to enhance user perception.Doctor of Philosoph

Design and demonstration of a vari-focal optical see-through head-mounted display using freeform Alvarez lenses

Alvarez lenses offer accurate and high-speed, dynamic tuning of optical power through a lateral shifting of two lens elements, making them an appealing solution to eliminate the inherent decoupling of accommodation and convergence seen in conventional stereoscopic displays. In this paper, we present a design of a compact eyepiece coupled with two lateral-shifting freeform Alvarez lenses to enable a compact. high-resolution, optical see-through head-mounted display (HMD). The proposed design is able to tune its focal depth from 0 to 3 diopters, rendering near-accurate focus cues with high image quality and a large undistorted see-through field of view (FOV). Our design utilizes an 1920x1080 color resolution organic light-emitting diode (OLED) microdisplay to achieve a >30 degree virtual diagonal FOV, with an angular resolution of 0.4 contrast over the full field. We also experimentally demonstrate a fully functional benchtop prototype using mostly off-the-shelf optics. (C) 2019 Optical Society of America under the terms of the OSA Open Access Publishing AgreementOpen access journalThis item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at [email protected]

Geometric model and calibration method for a solid-state LiDAR

This paper presents a novel calibration method for solid-state LiDAR devices based on a geometrical description of their scanning system, which has variable angular resolution. Determining this distortion across the entire Field-of-View of the system yields accurate and precise measurements which enable it to be combined with other sensors. On the one hand, the geometrical model is formulated using the well-known Snell’s law and the intrinsic optical assembly of the system, whereas on the other hand the proposed method describes the scanned scenario with an intuitive camera-like approach relating pixel locations with scanning directions. Simulations and experimental results show that the model fits with real devices and the calibration procedure accurately maps their variant resolution so undistorted representations of the observed scenario can be provided. Thus, the calibration method proposed during this work is applicable and valid for existing scanning systems improving their precision and accuracy in an order of magnitude.Peer ReviewedPostprint (published version

Roadmap on digital holography [Invited]

This Roadmap article on digital holography provides an overview of a vast array of research activities in the field of digital holography. The paper consists of a series of 25 sections from the prominent experts in digital holography presenting various aspects of the field on sensing, 3D imaging and displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the vision of its author to describe the significant progress, potential impact, important developments, and challenging issues in the field of digital holography

Bio-Inspired Multi-Spectral Image Sensor and Augmented Reality Display for Near-Infrared Fluorescence Image-Guided Surgery

Background: Cancer remains a major public health problem worldwide and poses a huge economic burden. Near-infrared (NIR) fluorescence image-guided surgery (IGS) utilizes molecular markers and imaging instruments to identify and locate tumors during surgical resection. Unfortunately, current state-of-the-art NIR fluorescence imaging systems are bulky, costly, and lack both fluorescence sensitivity under surgical illumination and co-registration accuracy between multimodal images. Additionally, the monitor-based display units are disruptive to the surgical workflow and are suboptimal at indicating the 3-dimensional position of labeled tumors. These major obstacles have prevented the wide acceptance of NIR fluorescence imaging as the standard of care for cancer surgery. The goal of this dissertation is to enhance cancer treatment by developing novel image sensors and presenting the information using holographic augmented reality (AR) display to the physician in intraoperative settings. Method: By mimicking the visual system of the Morpho butterfly, several single-chip, color-NIR fluorescence image sensors and systems were developed with CMOS technologies and pixelated interference filters. Using a holographic AR goggle platform, an NIR fluorescence IGS display system was developed. Optoelectronic evaluation was performed on the prototypes to evaluate the performance of each component, and small animal models and large animal models were used to verify the overall effectiveness of the integrated systems at cancer detection. Result: The single-chip bio-inspired multispectral logarithmic image sensor I developed has better main performance indicators than the state-of-the-art NIR fluorescence imaging instruments. The image sensors achieve up to 140 dB dynamic range. The sensitivity under surgical illumination achieves 6108 V/(mW/cm2), which is up to 25 times higher. The signal-to-noise ratio is up to 56 dB, which is 11 dB greater. These enable high sensitivity fluorescence imaging under surgical illumination. The pixelated interference filters enable temperature-independent co-registration accuracy between multimodal images. Pre-clinical trials with small animal model demonstrate that the sensor can achieve up to 95% sensitivity and 94% specificity with tumor-targeted NIR molecular probes. The holographic AR goggle provides the physician with a non-disruptive 3-dimensional display in the clinical setup. This is the first display system that co-registers a virtual image with human eyes and allows video rate image transmission. The imaging system is tested in the veterinary science operating room on canine patients with naturally occurring cancers. In addition, a time domain pulse-width-modulation address-event-representation multispectral image sensor and a handheld multispectral camera prototype are developed. Conclusion: The major problems of current state-of-the-art NIR fluorescence imaging systems are successfully solved. Due to enhanced performance and user experience, the bio-inspired sensors and augmented reality display system will give medical care providers much needed technology to enable more accurate value-based healthcare

Liquid Crystal on Silicon Devices: Modeling and Advanced Spatial Light Modulation Applications

Liquid Crystal on Silicon (LCoS) has become one of the most widespread technologies for spatial light modulation in optics and photonics applications. These reflective microdisplays are composed of a high-performance silicon complementary metal oxide semiconductor (CMOS) backplane, which controls the light-modulating properties of the liquid crystal layer. State-of-the-art LCoS microdisplays may exhibit a very small pixel pitch (below 4 ?m), a very large number of pixels (resolutions larger than 4K), and high fill factors (larger than 90%). They modulate illumination sources covering the UV, visible, and far IR. LCoS are used not only as displays but also as polarization, amplitude, and phase-only spatial light modulators, where they achieve full phase modulation. Due to their excellent modulating properties and high degree of flexibility, they are found in all sorts of spatial light modulation applications, such as in LCOS-based display systems for augmented and virtual reality, true holographic displays, digital holography, diffractive optical elements, superresolution optical systems, beam-steering devices, holographic optical traps, and quantum optical computing. In order to fulfil the requirements in this extensive range of applications, specific models and characterization techniques are proposed. These devices may exhibit a number of degradation effects such as interpixel cross-talk and fringing field, and time flicker, which may also depend on the analog or digital backplane of the corresponding LCoS device. The use of appropriate characterization and compensation techniques is then necessary

Advancements in Optical See-through Near-Eye Display

With the development of optical design and manufacturing, the optical see-through near-eye display becomes a promising research topic in recent decades, which can be applied in medical devices, education, aviation, entertainment et al. Typical products include Head-mounted Displays (HMDs) and Augmented Reality (AR) glasses. The optical display system of AR devices mainly consists of a miniature projecting module and an optical imaging module. In this chapter, the display systems used by AR glasses on the market, including various mini-display screens and optical imaging elements, have been systematically summarized. Therein, the differences in optical combinators are the key part to distinguish various AR display systems. Thus, it is essential to figure out the advantages and disadvantages of each optical imaging technology applied in this area. Besides, the characteristics of the projectors are crucial to the quality of the images