무안경식 3 차원 디스플레이와 투사형 디스플레이를 이용한 깊이 융합 디스플레이의 관찰 특성 향상

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 8. 이병호.In this dissertation, various methods for enhancing the viewing characteristics of the depth-fused display are proposed with combination of projection-type displays or integral imaging display technologies. Depth-fused display (DFD) is one kind of the volumetric three-dimensional (3D) displays composed of multiple slices of depth images. With a proper weighting to the luminance of the images on the visual axis of the observer, it provides continuous change of the accommodation within the volume confined by the display layers. Because of its volumetric property depth-fused 3D images can provide very natural volumetric images, but the base images should be located on the exact positions on the viewing axis, which gives complete superimpose of the images. If this condition is not satisfied, the images are observed as two separated images instead of continuous volume. This viewing characteristic extremely restricts the viewing condition of the DFD resulting in the limited applications of DFDs. While increasing the number of layers can result in widening of the viewing angle and depth range by voxelizing the reconstructed 3D images, the required system complexity also increases along with the number of image layers. For solving this problem with a relatively simple configuration of the system, hybrid techniques are proposed for DFDs. The hybrid technique is the combination of DFD with other display technologies such as projection-type displays or autostereoscopic displays. The projection-type display can be combined with polarization-encoded depth method for projection of 3D information. Because the depth information is conveyed by polarization states, there is no degradation in spatial resolution or video frame in the reconstructed 3D images. The polarized depth images are partially selected at the stacked polarization selective screens according to the given depth states. As the screen does not require any active component for the reconstruction of images, projection part and reconstruction part can be totally separated. Also, the projection property enables the scalability of the reconstructed images like a conventional projection display, which can give immersive 3D experience by providing large 3D images. The separation of base images due to the off-axis observation can be compensated by shifting the base images along the viewers visual axis. It can be achieved by adopting multi-view techniques. While conventional multi-view displays provide different view images for different viewers positions, it can be used for showing shifted base images for DFD. As a result, multiple users can observe the depth-fused 3D images at the same time. Another hybrid method is the combination of floating method with DFD. Convex lens can optically translate the depth position of the object. Based on this principle, the optical gap between two base images can be extended beyond the physical dimension of the images. Employing the lens with a short focal length, the gap between the base images can be greatly reduced. For a practical implementation of the system, integral imaging method can be used because it is composed of array of lenses. The floated image can be located in front of the lens as well as behind the lens. Both cases result in the expansion of depth range beyond the physical gap of base images, but real-mode floating enables interactive application of the DFD. In addition to the expansion of depth range, the viewing angle of the hybrid system can be increased by employing tracking method. Viewer tracking method also enables dynamic parallax for the DFD with real-time update of base images along with the viewing direction of the tracked viewers. Each chapter of this dissertation explains the theoretical background of the proposed hybrid method and demonstrates the feasibility of the idea with experimental systems.Abstract i Contents iv List of Figures vi List of Tables xii Chapter 1 Introduction 1 1.1 Overview of three-dimensional displays 1 1.2 Motivation 7 1.3 Scope and organization 9 Chapter 2 Multi-layered depth-fused display with projection-type display 10 2.1 Introduction 10 2.2 Polarization-encoded depth information for depth-fused display 12 2.3 Visualization with passive scattering film 16 2.4 Summary 30 Chapter 3 Compact depth-fused display with enhanced depth and viewing angle 31 3.1 Introduction 31 3.2 Enhancement of viewing characteristics 34 3.2.1 Viewing angle enhancement using multi-view method 34 3.2.2 Depth enhancement using integral imaging 37 3.2.3 Depth and viewing angle enhancement 39 3.3 Implementation of experimental system with enhanced viewing parameters 44 3.4 Summary 51 Chapter 4 Real-mode depth-fused display with viewer tracking 52 4.1 Introduction 52 4.2 Viewer tracking method 55 4.2.1 Viewer-tracked depth-fused display 55 4.2.2 Viewer-tracked integral imaging for a depth-fused display 58 4.3 Implementation of viewer-tracked integral imaging 63 4.4 Summary 71 Chapter 5 Conclusion 72 Bibliography 74 초록 83Docto

비등방성 광학 소자를 이용한 광 시야각 근안 디스플레이

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 전기·컴퓨터공학부, 2019. 2. 이병호.Near-eye display is considered as a promising display technique to realize augmented reality by virtue of its high sense of immersion and user-friendly interface. Among the important performances of near-eye display, a field of view is the most crucial factor for providing a seamless and immersive experience for augmented reality. In this dissertation, a transmissive eyepiece is devised instead of a conventional reflective eyepiece and it is discussed how to widen the field of view without loss of additional system performance. In order to realize the transmissive eyepiece, the eyepiece should operate lens to virtual information and glass to real-world scene. Polarization multiplexing technique is used to implement the multi-functional optical element, and anisotropic optical elements are used as material for multi-functional optical element. To demonstrate the proposed idea, an index-matched anisotropic crystal lens has been presented that reacts differently depending on polarization. With the combination of isotropic material and anisotropic crystal, the index-matched anisotropic crystal lens can be the transmissive eyepiece and achieve the large field of view. Despite the large field of view by the index-matched anisotropic crystal lens, many problems including form factor still remain to be solved. In order to overcome the limitations of conventional optics, a metasurface is adopted to the augmented reality application. With a stunning optical performance of the metasurface, a see-through metasurface lens is proposed and designed for implementing wide field of view near-eye display. The proposed novel eyepieces are expected to be an initiative study not only improving the specification of the existing near-eye display but opening the way for a next generation near-eye display.근안 디스플레이는 높은 몰입감과 사용자 친화적인 인터페이스로 인해 증강 현실을 구현하는 가장 효과적인 기술로 최근 활발한 연구가 계속되고 있다. 이러한 근안 디스플레이의 중요한 성능 중 시야각은 매끄럽고 몰입감 있는 경험을 사용자에게 전해줌으로써 가장 중요한 광학적 평가지표 중에 하나이다. 본 논문에서는 기존의 반사형 아이피스 (eyepiece) 를 대신하는 투과형 아이피스를 제안한다. 이러한 투과형 아이피스를 구현하기 위해서는 외부 정보에 대해서는 투명한 유리와 같이 투과시키며, 동시에 가상 정보는 렌즈로 작동하여 먼 거리에 띄울 수 있는 광학소자를 개발하여야 한다. 이러한 투과형 아이피스를 구현하기 위해서 편광에 따라 다르게 반응하는 굴절률 정합 이방성 결정 렌즈 (index-matched anisotropic crystal lens) 를 제안하였다. 이방성 결정 구조 (anisotropic crystal)로 이루어진 렌즈와 이를 둘러싼 등방성 물질 (isotropic crytal) 로 이루어진 굴절률 정합 이방성 결정 렌즈는 편광에 따라 다르게 작동한다. 이러한 투과형 아이피스는 기존의 근안 디스플레이에 비해 넓은 시야각을 제공할 수 있지만 이방성 결정 구조의 낮은 굴절률 차이로 인해 시스템의 크기가 커지는 단점을 가지고 있다. 본 논문에서는 이러한 단점을 개선하기 위해 메타 표면을 증강 현실 디스플레이 분야에 적용하였다. 메타 표면의 기존 광학 소자를 능가하는 놀라운 광학 성능을 이용하여 넓은 시야각을 가지는 근안 디스플레이를 구현하기 위해 투명 메타 렌즈를 제안하였다. 편광에 따라 다르게 반응하는 투명 메타렌즈는 넓은 시야각과 경량화 시스템 구현이 가능하며 이를 입증하기 위해 투명 메타렌즈의 설계 방법 뿐 아니라 실제 구현을 통한 가능성을 입증하였다. 이러한 새로운 아이피스에 대한 개념은 기존의 근안 디스플레이의 사양 개선에 유용하게 사용될 뿐 아니라 차세대 근안 디스플레이를 위한 선도적인 역할을 할 것으로 기대된다.Abstract Contents List of Tables List of Figures Near-eye displays with wide field of view using anisotropic optical elements Chapter 1 Introduction 1.1 Near-eye displays for augmented reality 1.2 Optical performances of near-eye display 1.3 State-of-the-arts of near-eye display 1.4 Motivation and contribution of this dissertation Chapter 2 Transmissive eyepiece for wide field of view near-eye display 2.1 Transmissive eyepiece for near-eye display Chapter 3 Near-eye display using index-matched anisotropic crystal lens 3.1 Introduction 3.2 Index-matched anisotropic crystal lens 3.2.1 Principle of the index-matched anisotropic crystal lens 3.2.2 Aberration analysis of index-matched anisotropic crystal lens 3.2.3 Implementation 3.3 Near-eye displays using index-matched anisotropic crystal lens 3.3.1 Near-eye display using index-matched anisotropic crystal lens 3.3.2 Flat panel type near-eye display using IMACL 3.3.3 Polarization property of transparent screen 3.4 Conclusion Chapter 4 Near-eye display using metasurface lens 4.1 Introduction 4.2 See-through metasurface lens 4.2.1 Metasurface lens 4.3 Full-color near-eye display using metasurface lens 4.3.1 Full-color near-eye display using metasurface lens 4.3.2 Holographic near-eye display using metasurface lens for aberration compensation 4.4 Conclusion Chapter 5 Conclusion Bibliography AppendixDocto

High-dynamic-range Foveated Near-eye Display System

Wearable near-eye display has found widespread applications in education, gaming, entertainment, engineering, military training, and healthcare, just to name a few. However, the visual experience provided by current near-eye displays still falls short to what we can perceive in the real world. Three major challenges remain to be overcome: 1) limited dynamic range in display brightness and contrast, 2) inadequate angular resolution, and 3) vergence-accommodation conflict (VAC) issue. This dissertation is devoted to addressing these three critical issues from both display panel development and optical system design viewpoints. A high-dynamic-range (HDR) display requires both high peak brightness and excellent dark state. In the second and third chapters, two mainstream display technologies, namely liquid crystal display (LCD) and organic light emitting diode (OLED), are investigated to extend their dynamic range. On one hand, LCD can easily boost its peak brightness to over 1000 nits, but it is challenging to lower the dark state to \u3c 0.01 nits. To achieve HDR, we propose to use a mini-LED local dimming backlight. Based on our simulations and subjective experiments, we establish practical guidelines to correlate the device contrast ratio, viewing distance, and required local dimming zone number. On the other hand, self-emissive OLED display exhibits a true dark state, but boosting its peak brightness would unavoidably cause compromised lifetime. We propose a systematic approach to enhance OLED\u27s optical efficiency while keeping indistinguishable angular color shift. These findings will shed new light to guide future HDR display designs. In Chapter four, in order to improve angular resolution, we demonstrate a multi-resolution foveated display system with two display panels and an optical combiner. The first display panel provides wide field of view for peripheral vision, while the second panel offers ultra-high resolution for the central fovea. By an optical minifying system, both 4x and 5x enhanced resolutions are demonstrated. In addition, a Pancharatnam-Berry phase deflector is applied to actively shift the high-resolution region, in order to enable eye-tracking function. The proposed design effectively reduces the pixelation and screen-door effect in near-eye displays. The VAC issue in stereoscopic displays is believed to be the main cause of visual discomfort and fatigue when wearing VR headsets. In Chapter five, we propose a novel polarization-multiplexing approach to achieve multiplane display. A polarization-sensitive Pancharatnam-Berry phase lens and a spatial polarization modulator are employed to simultaneously create two independent focal planes. This method enables generation of two image planes without the need of temporal multiplexing. Therefore, it can effectively reduce the frame rate by one-half. In Chapter six, we briefly summarize our major accomplishments

Efficient image-based rendering

Recent advancements in real-time ray tracing and deep learning have significantly enhanced the realism of computer-generated images. However, conventional 3D computer graphics (CG) can still be time-consuming and resource-intensive, particularly when creating photo-realistic simulations of complex or animated scenes. Image-based rendering (IBR) has emerged as an alternative approach that utilizes pre-captured images from the real world to generate realistic images in real-time, eliminating the need for extensive modeling. Although IBR has its advantages, it faces challenges in providing the same level of control over scene attributes as traditional CG pipelines and accurately reproducing complex scenes and objects with different materials, such as transparent objects. This thesis endeavors to address these issues by harnessing the power of deep learning and incorporating the fundamental principles of graphics and physical-based rendering. It offers an efficient solution that enables interactive manipulation of real-world dynamic scenes captured from sparse views, lighting positions, and times, as well as a physically-based approach that facilitates accurate reproduction of the view dependency effect resulting from the interaction between transparent objects and their surrounding environment. Additionally, this thesis develops a visibility metric that can identify artifacts in the reconstructed IBR images without observing the reference image, thereby contributing to the design of an effective IBR acquisition pipeline. Lastly, a perception-driven rendering technique is developed to provide high-fidelity visual content in virtual reality displays while retaining computational efficiency.Jüngste Fortschritte im Bereich Echtzeit-Raytracing und Deep Learning haben den Realismus computergenerierter Bilder erheblich verbessert. Konventionelle 3DComputergrafik (CG) kann jedoch nach wie vor zeit- und ressourcenintensiv sein, insbesondere bei der Erstellung fotorealistischer Simulationen von komplexen oder animierten Szenen. Das bildbasierte Rendering (IBR) hat sich als alternativer Ansatz herauskristallisiert, bei dem vorab aufgenommene Bilder aus der realen Welt verwendet werden, um realistische Bilder in Echtzeit zu erzeugen, so dass keine umfangreiche Modellierung erforderlich ist. Obwohl IBR seine Vorteile hat, ist es eine Herausforderung, das gleiche Maß an Kontrolle über Szenenattribute zu bieten wie traditionelle CG-Pipelines und komplexe Szenen und Objekte mit unterschiedlichen Materialien, wie z.B. transparente Objekte, akkurat wiederzugeben. In dieser Arbeit wird versucht, diese Probleme zu lösen, indem die Möglichkeiten des Deep Learning genutzt und die grundlegenden Prinzipien der Grafik und des physikalisch basierten Renderings einbezogen werden. Sie bietet eine effiziente Lösung, die eine interaktive Manipulation von dynamischen Szenen aus der realen Welt ermöglicht, die aus spärlichen Ansichten, Beleuchtungspositionen und Zeiten erfasst wurden, sowie einen physikalisch basierten Ansatz, der eine genaue Reproduktion des Effekts der Sichtabhängigkeit ermöglicht, der sich aus der Interaktion zwischen transparenten Objekten und ihrer Umgebung ergibt. Darüber hinaus wird in dieser Arbeit eine Sichtbarkeitsmetrik entwickelt, mit der Artefakte in den rekonstruierten IBR-Bildern identifiziert werden können, ohne das Referenzbild zu betrachten, und die somit zur Entwicklung einer effektiven IBR-Erfassungspipeline beiträgt. Schließlich wird ein wahrnehmungsgesteuertes Rendering-Verfahren entwickelt, um visuelle Inhalte in Virtual-Reality-Displays mit hoherWiedergabetreue zu liefern und gleichzeitig die Rechenleistung zu erhalten

Computational See-Through Near-Eye Displays

See-through near-eye displays with the form factor and field of view of eyeglasses are a natural choice for augmented reality systems: the non-encumbering size enables casual and extended use and large field of view enables general-purpose spatially registered applications. However, designing displays with these attributes is currently an open problem. Support for enhanced realism through mutual occlusion and the focal depth cues is also not found in eyeglasses-like displays. This dissertation provides a new strategy for eyeglasses-like displays that follows the principles of computational displays, devices that rely on software as a fundamental part of image formation. Such devices allow more hardware simplicity and flexibility, showing greater promise of meeting form factor and field of view goals while enhancing realism. This computational approach is realized in two novel and complementary see-through near-eye display designs. The first subtractive approach filters omnidirectional light through a set of optimized patterns displayed on a stack of spatial light modulators, reproducing a light field corresponding to in-focus imagery. The design is thin and scales to wide fields of view; see-through is achieved with transparent components placed directly in front of the eye. Preliminary support for focal cues and environment occlusion is also demonstrated. The second additive approach uses structured point light illumination to form an image with a minimal set of rays. Each of an array of defocused point light sources is modulated by a region of a spatial light modulator, essentially encoding an image in the focal blur. See-through is also achieved with transparent components and thin form factors and wide fields of view (>= 100 degrees) are demonstrated. The designs are examined in theoretical terms, in simulation, and through prototype hardware with public demonstrations. This analysis shows that the proposed computational near-eye display designs offer a significantly different set of trade-offs than conventional optical designs. Several challenges remain to make the designs practical, most notably addressing diffraction limits.Doctor of Philosoph

Characteristics of flight simulator visual systems

The physical parameters of the flight simulator visual system that characterize the system and determine its fidelity are identified and defined. The characteristics of visual simulation systems are discussed in terms of the basic categories of spatial, energy, and temporal properties corresponding to the three fundamental quantities of length, mass, and time. Each of these parameters are further addressed in relation to its effect, its appropriate units or descriptors, methods of measurement, and its use or importance to image quality

포토폴리머 상의 홀로그래픽 기록 기술을 이용한 무안경식 삼차원 이미징 방법

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2014. 8. 이병호.이 박사학위 논문에서는 새로운 홀로그래픽 기록방법들을 이용하여 기존의 무안경식 삼차원 디스플레이의 제한점들을 개선하는 방법에 대해 논한다. 두 종류의 새로운 홀로그래픽 기록방법을 무안경식 삼차원 디스플레이를 구현하기 위해 제안한다. 첫 번째 방법은 호겔 중첩을 이용하여 홀로그래픽 스테레오그램의 해상도를 증가시키는 방법이며, 다른 하나의 방법은 렌즈어레이 홀로그래픽 광학소자를 이용하여 투명한 특성을 가지는 이차원 및 삼차원 영상을 구현하는 방법이다. 이 박사학위 논문에서는 홀로그래픽 기록매질로 광중합체 필름을 사용한다. 단일파장 기록방법과 세 파장 다중화 기록방법을 이용해 제작된 광중합체 필름 상에 기록되는 체적홀로그램에 대한 노출반응 특성을 실험을 통해 분석한다. 광중합체 필름에 기록된 세 파장 다중화된 홀로그램의 투명한 특성과 회절효율을 재생실험을 통해 평가하며, 광중합체 필름의 수축특성에 대해 이론적으로 분석하고 실험적으로 검증한다. 홀로그래픽 프린팅 방법을 이용해 기록된 홀로그래픽 스테레오그램의 해상도를 증가시키기 위한 호겔중첩 방법을 제안한다. 홀로그래픽 프린팅 방법을 이용해 홀로그래픽 스테레오그램을 기록할 때 해상도 증가를 위해 호겔의 크기를 무한정 줄일 수 없음을 컴퓨터 시뮬레이션을 통해 검증한다. 호겔 크기를 줄여 해상도를 증가시키는 방법 대신에 호겔을 중첩 기록시키는 방법을 이용해 홀로그래픽 스테레오그램의 해상도를 증가시킬 수 있는 방법을 제안한다. 홀로그래픽 스테레오그램을 기록할 수 있는 실험환경을 구축하였으며 이를 이용하여 기존의 방법과 제안된 호겔 중첩 방법을 적용한 서로 다른 두 개의 홀로그래픽 스테레오그램을 제작한다. 제작된 두 개의 홀로그래픽 스테레오그램의 재생 영상을 비교함으로써 제안된 호겔중첩 방법을 적용한 홀로그래픽 스테레오그램이 기존의 방법보다 높은 해상도를 가짐을 검증한다. 집적영상을 통해 투명한 특성을 가지는 이차원 및 삼차원 가상영상을 재생할 수 있는 렌즈어레이 홀로그래픽 광학소자를 제안한다. 홀로그래픽 광학소자를 기록하기 위한 실험환경에 파장다중화 기록 방법과 공간다중화 기록방법을 적용하여 총천연색 홀로그램 기록과 대면적 기록을 가능하게 하는 방법에 대해 논한다. 제안된 렌즈어레이 홀로그래픽 광학소자의 특성을 재생실험을 통해 평가한다. 제안된 구조에서 재생된 집적영상을 통해 렌즈어레이 홀로그래픽 광학소자가 증강현실 기술에 적용하기 적합한 광학적으로 투명한 특성을 만족시키면서 총천연색의 삼차원 가상영상을 재생 가능함을 검증한다. 제안된 렌즈어레이 홀로그래픽 광학소자를 통해 재생되는 영상의 관찰특성들을 렌즈어레이 홀로그래픽 광학소자의 광학변수들을 이용하여 평가하며, 이를 이용해 제안된 방법이 이차원/삼차원 투명스크린으로 사용될 수 있는 가능성을 제시한다. 이차원과 삼차원 투명스크린 역할을 할 수 있는 두 개의 서로 다른 사양의 렌즈어레이 홀로그래픽 광학소자들을 제작한다. 이 두 렌즈어레이 홀로그래픽 광학소자들은 재생실험의 결과를 통해 이차원/삼차원 투명 스크린으로 응용할 수 있음을 보이며, 이 때 재생되는 영상들의 관찰 특성들을 평가한다. 또한 외부의 이미지 프로젝터 등의 광변조장치를 통해 제공되는 영상정보를 사용하여 렌즈어레이 홀로그래픽 광학소자 상에서 동적 삼차원 가상영상을 재생하는 방법을 제시한다. 동적 요소영상을 컴퓨터 그래픽스를 이용하여 제작하였으며, 제안된 렌즈어레이 홀로그래픽 광학소자가 동적 삼차원 가상영상을 재생할 수 있음을 실험적으로 검증한다.The novel holographic recording techniques suggested in this dissertation are experimentally investigated to improve the limitations of conventional autostereoscopic three-dimensional (3D) displays. Two types of holographic recording techniques for implementing novel autostereoscopic 3D imaging methods are presented in this dissertation work: i) hogel overlapping method for enhancing lateral resolution of a holographic stereogram, and ii) lens-array holographic optical element (HOE) for providing a see-through property in an integral imaging method. Photopolymer film is used as a holographic material in this work. Dosage responses of a single wavelength and a three-wavelength multiplexed hologram recorded in the photopolymer film are presented. See-through property and diffraction efficiencies of the three-wavelength multiplexed hologram recorded in the photopolymer film are evaluated by the display experiments. Additionally, shrinkage of the photopolymer film is theoretically analyzed and measured by the experiments. The hogel overlapping method for a holographic printing technique is proposed to enhance the lateral resolution of holographic stereograms. A numerical analysis by computer simulation shows that there is a limitation on decreasing the hogel size while recording holographic stereograms. Instead of reducing the size of hogel, the lateral resolution of holographic stereograms can be improved by printing overlapped hogels. An experimental setup for holographic printing is built, and two holographic stereograms using the conventional and proposed overlapping methods are recorded, respectively. The resultant images from the experimentally generated holographic stereograms make a comparison between the conventional and proposed methods. The experimental results confirm that the proposed hogel overlapping method improves the lateral resolution of holographic stereograms compared to the conventional holographic printing method. The lens-array HOE is suggested for a see-through 3D imaging based on the integral imaging. The full-color lens-array HOE provides a see-through property with three-dimensional virtual images. An HOE recording setup is built, and the full-color lens-array HOEs are recorded by using holographic recording techniques of a spatial and wavelength multiplexing. The experimental results confirm that the suggested method can provide the full-color 3D virtual images with the see-through property. The viewing characteristics of the presented autostereoscopic 3D display are evaluated by the optical parameters of the lens-array HOE. Two lens-array HOEs with different optical parameters are fabricated to have both functions of the two-dimensional (2D) and 3D transparent screens. Display experiments for the 2D and 3D imaging on the proposed transparent screens are carried out, and the viewing characteristics in both cases are discussed. The autostereoscopic 3D display using the lens-array HOE can provide dynamic 3D virtual images because it has an external spatial light modulator. The dynamic elemental images are generated by computer graphics, and the feasibility of displaying the dynamic 3D virtual images on the lens-array HOE is experimentally verified.Chapter 1 Introduction 1 1.1 Background and current issues of autostereoscopic three-dimensional display 1 1.2 Motivation of this dissertation work 8 1.3 Objective and scope of this dissertation 11 Chapter 2 Photopolymer film for holographic material 14 2.1 Introduction 14 2.2 Principles of refractive index modulation in photopolymer film 18 2.3 Dosage response of photopolymer film for the hologram recording in a reflection geometry 23 2.4 Shrinkage characteristic of photopolymer film 37 2.5 Results 45 Chapter 3 High resolution autostereoscopic 3D display using holographic printing 47 3.1 Introduction 47 3.2 Overview of holographic printing method 49 3.3 Limitation on enhancing lateral resolution of holographic printing 54 3.4 Hogel overlapping method for enhancing lateral resolution of holographic printer 60 3.5 Experiments 64 3.6 Results 71 Chapter 4 See-through autostereoscopic 3D display using lens-array holographic optical elements 73 4.1 Introduction 73 4.2 Full-color lens-array holographic optical elements for displaying integral images 77 4.2.1 Principles of full-color lens-array holographic optical elements 77 4.2.2 Hologram recording setup for fabricating proposed lens-array holographic optical elements 80 4.2.3 Three-dimensional imaging on full-color lens-array holographic optical elements 85 4.3 Viewing characteristic analysis on lens-array holographic optical elements 90 4.3.1 Viewing characteristic of lens-array holographic optical elements 91 4.3.2 2D and 3D imaging on lens-array holographic optical elements with different viewing parameters 94 4.3.3 Experiments 98 4.4 Dynamic autostereoscopic 3D images displayed on the lens-array HOE 109 4.5 Results 113 Chapter 5 Conclusion and recommendation for future work 115 5.1 Conclusion 115 5.2 Recommendation for future work 120 Bibliography 124 Appendix 138 초 록 139Docto

Methods for Light Field Display Profiling and Scalable Super-Multiview Video Coding

Light field 3D displays reproduce the light field of real or synthetic scenes, as observed by multiple viewers, without the necessity of wearing 3D glasses. Reproducing light fields is a technically challenging task in terms of optical setup, content creation, distributed rendering, among others; however, the impressive visual quality of hologramlike scenes, in full color, with real-time frame rates, and over a very wide field of view justifies the complexity involved. Seeing objects popping far out from the screen plane without glasses impresses even those viewers who have experienced other 3D displays before.Content for these displays can either be synthetic or real. The creation of synthetic (rendered) content is relatively well understood and used in practice. Depending on the technique used, rendering has its own complexities, quite similar to the complexity of rendering techniques for 2D displays. While rendering can be used in many use-cases, the holy grail of all 3D display technologies is to become the future 3DTVs, ending up in each living room and showing realistic 3D content without glasses. Capturing, transmitting, and rendering live scenes as light fields is extremely challenging, and it is necessary if we are about to experience light field 3D television showing real people and natural scenes, or realistic 3D video conferencing with real eye-contact.In order to provide the required realism, light field displays aim to provide a wide field of view (up to 180°), while reproducing up to ~80 MPixels nowadays. Building gigapixel light field displays is realistic in the next few years. Likewise, capturing live light fields involves using many synchronized cameras that cover the same display wide field of view and provide the same high pixel count. Therefore, light field capture and content creation has to be well optimized with respect to the targeted display technologies. Two major challenges in this process are addressed in this dissertation.The first challenge is how to characterize the display in terms of its capabilities to create light fields, that is how to profile the display in question. In clearer terms this boils down to finding the equivalent spatial resolution, which is similar to the screen resolution of 2D displays, and angular resolution, which describes the smallest angle, the color of which the display can control individually. Light field is formalized as 4D approximation of the plenoptic function in terms of geometrical optics through spatiallylocalized and angularly-directed light rays in the so-called ray space. Plenoptic Sampling Theory provides the required conditions to sample and reconstruct light fields. Subsequently, light field displays can be characterized in the Fourier domain by the effective display bandwidth they support. In the thesis, a methodology for displayspecific light field analysis is proposed. It regards the display as a signal processing channel and analyses it as such in spectral domain. As a result, one is able to derive the display throughput (i.e. the display bandwidth) and, subsequently, the optimal camera configuration to efficiently capture and filter light fields before displaying them.While the geometrical topology of optical light sources in projection-based light field displays can be used to theoretically derive display bandwidth, and its spatial and angular resolution, in many cases this topology is not available to the user. Furthermore, there are many implementation details which cause the display to deviate from its theoretical model. In such cases, profiling light field displays in terms of spatial and angular resolution has to be done by measurements. Measurement methods that involve the display showing specific test patterns, which are then captured by a single static or moving camera, are proposed in the thesis. Determining the effective spatial and angular resolution of a light field display is then based on an automated analysis of the captured images, as they are reproduced by the display, in the frequency domain. The analysis reveals the empirical limits of the display in terms of pass-band both in the spatial and angular dimension. Furthermore, the spatial resolution measurements are validated by subjective tests confirming that the results are in line with the smallest features human observers can perceive on the same display. The resolution values obtained can be used to design the optimal capture setup for the display in question.The second challenge is related with the massive number of views and pixels captured that have to be transmitted to the display. It clearly requires effective and efficient compression techniques to fit in the bandwidth available, as an uncompressed representation of such a super-multiview video could easily consume ~20 gigabits per second with today’s displays. Due to the high number of light rays to be captured, transmitted and rendered, distributed systems are necessary for both capturing and rendering the light field. During the first attempts to implement real-time light field capturing, transmission and rendering using a brute force approach, limitations became apparent. Still, due to the best possible image quality achievable with dense multi-camera light field capturing and light ray interpolation, this approach was chosen as the basis of further work, despite the massive amount of bandwidth needed. Decompression of all camera images in all rendering nodes, however, is prohibitively time consuming and is not scalable. After analyzing the light field interpolation process and the data-access patterns typical in a distributed light field rendering system, an approach to reduce the amount of data required in the rendering nodes has been proposed. This approach, on the other hand, requires rectangular parts (typically vertical bars in case of a Horizontal Parallax Only light field display) of the captured images to be available in the rendering nodes, which might be exploited to reduce the time spent with decompression of video streams. However, partial decoding is not readily supported by common image / video codecs. In the thesis, approaches aimed at achieving partial decoding are proposed for H.264, HEVC, JPEG and JPEG2000 and the results are compared.The results of the thesis on display profiling facilitate the design of optimal camera setups for capturing scenes to be reproduced on 3D light field displays. The developed super-multiview content encoding also facilitates light field rendering in real-time. This makes live light field transmission and real-time teleconferencing possible in a scalable way, using any number of cameras, and at the spatial and angular resolution the display actually needs for achieving a compelling visual experience