Color space adaptation for video coding

Processament d'imatges abans de ser codificades pel codificador HEVC amb la finalitat d'augmentar la qualitat i la fidelitat.[ANGLÈS] Project on the objective and subjective improvements by pre-processing images to be encoded into a video.[CASTELLÀ] Proyecto sobre la repercusión en la mejora de calidad objetiva y subjetiva del pre-procesado de imágenes a codificar con vídeo.[CATALÀ] Projecte sobre la repercussió en la millora de la qualitat objectiva i subjectiva del pre-processament d'imatges a codificar amb vídeo

Content-Adaptive Non-Stationary Projector Resolution Enhancement

For any projection system, one goal will surely be to maximize the quality of projected imagery at a minimized hardware cost, which is considered a challenging engineering problem. Experience in applying different image filters and enhancements to projected video suggests quite clearly that the quality of a projected enhanced video is very much a function of the content of the video itself. That is, to first order, whether the video contains content which is moving as opposed to still plays an important role in the video quality, since the human visual system tolerates much more blur in moving imagery but at the same time is significantly sensitive to the flickering and aliasing caused by moving sharp textures. Furthermore, the spatial and statistical characteristics of text and non-text images are quite distinct. We would, therefore, assert that the text-like, moving and background pixels of a given video stream should be enhanced differently using class-dependent video enhancement filters to achieve maximum visual quality. In this thesis, we present a novel text-dependent content enhancement scheme, a novel motion-dependent content enhancement scheme and a novel content-adaptive resolution enhancement scheme based on a text-like / non-text-like classification and a pixel-wise moving / non-moving classification, with the actual enhancement obtained via class--dependent Wiener deconvolution filtering. Given an input image, the text and motion detection methods are used to generate binary masks to indicate the location of the text and moving regions in the video stream. Then enhanced images are obtained by applying a plurality of class-dependent enhancement filters, with text-like regions sharpened more than the background and moving regions sharpened less than the background. Later, one or more resulting enhanced images are combined into a composite output image based on the corresponding mask of different features. Finally, a higher resolution projected video stream is conducted by controlling one or more projectors to project the plurality of output frame streams in a rapid overlapping way. Experimental results on the test images and videos show that the proposed schemes all offer improved visual quality over projection without enhancement as well as compared to a recent state-of-the-art enhancement method. Particularly, the proposed content-adaptive resolution enhancement scheme increases the PSNR value by at least 18.2% and decreases MSE value by at least 25%

편광 다중화를 이용하여 향상된 기능을 제공하는 도파관 기반의 근안 디스플레이

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2021. 2. 이병호.This dissertation presents the studies on the optical design method that enhances the display performance of see-through waveguide-based near-eye displays (WNEDs) using the polarization multiplexing technique. The studies focus on the strategies to improve the crucial display performances without compromising a small form factor, the most attractive merit of the WNEDs. To achieve this goal, thin and lightweight polarization-dependent optical elements are devised and employed in the WNED structure. The polarization-dependent devices can allow multiple optical functions or optical paths depending on the polarization state of the input beam, which can break through the limitation of the waveguide system with the polarization multiplexing. To realize the function-selective eyepiece for AR applications, the proposed devices should operate as an optically transparent window for the real scene while performing specific optical functions for the virtual image. The proposed devices are manufactured in a combination structure in which polarization-dependent optical elements are stacked. The total thickness of the stacked structure is about 1 mm, and it can be attached to the waveguide surface without conspicuously increasing the form factor of the optical system. Using the proposed polarization-dependent devices, the author proposes three types of novel WNED systems with enhanced performance. First, the author suggests a compact WNED with dual focal planes. Conventional WNEDs have an inherent limitation that the focal plane of the virtual image is at an infinite distance because they extract a stream of collimated light at the out-coupler. By using the polarization-dependent eyepiece lens, an additional focal plane can be generated with the polarization multiplexing in addition to infinity depth. The proposed configuration can provide comfortable AR environments by alleviating visual fatigue caused by vergence-accommodation conflict. Second, the novel WNED configuration with extended field-of-view (FOV) is presented. In the WNEDs, the maximum allowable FOV is determined by the material properties of the diffraction optics and the substrate. By using the polarization-dependent steering combiner, the FOV can be extended up to two times, which can provide more immersive AR experiences. In addition, this dissertation demonstrates that the distortion for the real scene caused by the stacked structure cannot severely disturb the image quality, considering the acuity of human vision. Lastly, the author presents a retinal projection-based WNED with switchable viewpoints by simultaneously adopting the polarization-dependent lens and grating. The proposed system can convert the viewpoint according to the position of the eye pupil without mechanical movement. The polarization-dependent viewpoint switching can resolve the inherent problem of a narrow eyebox in retinal projection displays without employing the bulky optics for mechanical movement. In conclusion, the dissertation presents the practical optical design and detailed analysis for enhanced WNED based on the polarization multiplexing technique through various simulations and experiments. The proposed approaches are expected to be utilized as an innovative solution for compact wearable displays.본 박사학위 논문에서는 편광 다중화 기법을 이용하여 도파관 기반의 증강현실 근안 디스플레이의 성능을 향상시키는 광학 설계 및 이에 대한 분석에 대해 논의한다. 본 연구는 도파관 기반 근안 디스플레이의 가장 큰 장점인 소형 폼 팩터를 유지하면서 디스플레이 성능을 개선하는 것에 중점을 둔다. 이를 위해 기존 광학 소자에 비해 매우 가볍고 얇은 편광 의존형 결합기 광학 소자가 새롭게 제안되며, 이는 입사광의 편광 상태에 따라 독립적인 광 경로 제어를 가능케 하여 편광 다중화를 통해 향상된 성능을 제공 할 수 있다. 또한 실제 영상의 빛은 그대로 투과 시킴으로써 증강현실을 위한 소자로 활용 가능하다. 본 연구에서 제안하는 편광 의존형 결합기 광학 소자는 기하학적 위상(geometric phase, GP)에 기반하여 동작한다. GP 기반의 광학소자가 서로 직교하는 원형 편광 입사광에 대해 상보적인 기능을 수행하는 것을 이용하여, 두 개 이상의 GP 소자와 편광 제어를 위한 광학 필름들을 중첩 시킴으로써 증강현실 결합기 광학 소자를 구현할 수 있다. 이들 광학소자는 매우 얇기 때문에, 본 연구에서 제작된 편광 의존형 결합기 광학 소자의 총 두께는 1 mm 수준으로 폼 팩터 제약으로부터 자유롭다. 또한 평평한 필름 형태이므로, 평판형 도파관에 부착하기 쉽다는 이점을 지닌다. 고안된 편광 의존형 결합기 광학 소자를 사용하여 세 가지 유형의 새로운 도파관 기반의 근안 디스플레이 구조를 제안한다. 첫 번째는 입사광의 편광 상태에 따라 투명 광학 창 또는 오목 렌즈로 작동하는 편광 의존형 결합기 렌즈를 적용하여 가상 영상에 대해 이중 초점면을 제공하는 시스템이다. 제안된 구조는 기존의 도파관 기반 근안 디스플레이가 무한대 위치에 단일 초점면을 제공함으로써 발생하는 시각적 피로 및 흐릿한 증강현실 영상의 문제를 완화할 수 있다. 두 번째로는 입사광의 편광 상태에 따라 광 경로를 좌측 또는 우측으로 제어할 수 있는 편광 격자를 활용하여 가상 영상의 시야각을 기존보다 최대 2배까지 확장할 수 있는 시스템을 제안한다. 이는 단일 도파관 기반 근안 디스플레이에서 영상 결합기 (imaging combiner)로 활용되는 회절 소자의 설계 변수에 의해 제한되는 시야각 한계점을 돌파할 수 있는 구조로 컴팩트한 폼 팩터로 더욱 몰입감 있는 대화면 증강현실 영상을 제공할 수 있다. 마지막으로 위에서 제안된 두 가지 편광 의존형 광학 소자를 모두 사용하여 시점 전환이 가능한 도파관 기반의 망막 투사형 디스플레이 구조를 제안한다. 편광 다중화를 통해 다중 초점들을 선택적으로 활성화함으로써, 확장된 시청영역을 제공하는 동시에 동공 크기 변화 또는 움직임에 의한 이중 영상 문제를 완화할 수 있다. 또한 기계적 움직임 없이 시점 간의 고속 전환이 가능하다는 장점을 지니고 있다. 본 박사학위 논문에서 제시한 편광 다중화를 활용한 새로운 결합기 광학 소자 및 광학 구조들은 도파관 기반 근안 디스플레이의 향상된 성능을 제공하는 해결책 및 새로운 가능성으로 제시할 수 있을 것이라 기대된다.Abstract i Contents iii List of Tables vi List of Figures vii Chapter. 1 Introduction 1 1.1 Augmented reality near-eye display 1 1.2 Key performance parameters of near-eye displays 4 1.3 Basic scheme of waveguide-based near-eye displays 22 1.4 Motivation and purpose of this dissertation 33 1.5 Scope and organization 37 Chapter 2 Dual-focal waveguide-based near-eye display using polarization-dependent combiner lens 39 2.1 Introduction 39 2.2 Optical design for polarization-dependent combiner lens 42 2.2.1 Design and principle of polarization-dependent combiner lens 42 2.2.2 Prototype implementation 48 2.3 Waveguide-based augmented reality near-eye display with dual-focal plane using polarization-dependent combiner lens 51 2.3.1 Implementation of the prototype and experimental results 51 2.3.2 Performance analysis and discussion 57 2.4 Conclusion 69 Chapter 3 Extended-field-of-view waveguide-based near-eye display via polarization-dependent steering combiner 70 3.1 Introduction 70 3.2 Optical design for polarization-dependent steering combiner 73 3.2.1 Principle of polarization grating 73 3.2.2 Principle of polarization-dependent steering combiner 76 3.2.3 Analysis and verification experiment for real-scene distortion 77 3.3 Waveguide-based augmented reality near-eye display with extended-field-of-view 81 3.3.1 Field-of-view for volume grating based waveguide technique 81 3.3.2 Implementation of the prototype and experimental results 84 3.3.3 Performances analysis and discussion 87 3.4 Conclusion 92 Chapter 4 Viewpoint switchable retinal-projection-based near-eye display with waveguide configuration 93 4.1 Introduction 93 4.2 Polarization-dependent switchable eyebox 97 4.2.1 Optical devices for polarization-dependent switching of viewpoints 97 4.2.2 System configuration for proposed method 100 4.2.3 Design of waveguide and imaging combiner 105 4.3 Compact retinal projection-based near-eye display with switchable viewpoints via waveguide configuration 114 4.3.1 Implementation of the prototype and experimental results 114 4.3.2 Performance analysis and discussion 118 4.4 Conclusion 122 Chapter 5. Conclusion 123 Bibliography 127 Appendix 135Docto

Liquid Crystal on Silicon Devices: Modeling and Advanced Spatial Light Modulation Applications

Liquid Crystal on Silicon (LCoS) has become one of the most widespread technologies for spatial light modulation in optics and photonics applications. These reflective microdisplays are composed of a high-performance silicon complementary metal oxide semiconductor (CMOS) backplane, which controls the light-modulating properties of the liquid crystal layer. State-of-the-art LCoS microdisplays may exhibit a very small pixel pitch (below 4 ?m), a very large number of pixels (resolutions larger than 4K), and high fill factors (larger than 90%). They modulate illumination sources covering the UV, visible, and far IR. LCoS are used not only as displays but also as polarization, amplitude, and phase-only spatial light modulators, where they achieve full phase modulation. Due to their excellent modulating properties and high degree of flexibility, they are found in all sorts of spatial light modulation applications, such as in LCOS-based display systems for augmented and virtual reality, true holographic displays, digital holography, diffractive optical elements, superresolution optical systems, beam-steering devices, holographic optical traps, and quantum optical computing. In order to fulfil the requirements in this extensive range of applications, specific models and characterization techniques are proposed. These devices may exhibit a number of degradation effects such as interpixel cross-talk and fringing field, and time flicker, which may also depend on the analog or digital backplane of the corresponding LCoS device. The use of appropriate characterization and compensation techniques is then necessary

Brain-Inspired Computing

This open access book constitutes revised selected papers from the 4th International Workshop on Brain-Inspired Computing, BrainComp 2019, held in Cetraro, Italy, in July 2019. The 11 papers presented in this volume were carefully reviewed and selected for inclusion in this book. They deal with research on brain atlasing, multi-scale models and simulation, HPC and data infra-structures for neuroscience as well as artificial and natural neural architectures

Parallel algorithms and architectures for low power video decoding

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 197-204).Parallelism coupled with voltage scaling is an effective approach to achieve high processing performance with low power consumption. This thesis presents parallel architectures and algorithms designed to deliver the power and performance required for current and next generation video coding. Coding efficiency, area cost and scalability are also addressed. First, a low power video decoder is presented for the current state-of-the-art video coding standard H.264/AVC. Parallel architectures are used along with voltage scaling to deliver high definition (HD) decoding at low power levels. Additional architectural optimizations such as reducing memory accesses and multiple frequency/voltage domains are also described. An H.264/AVC Baseline decoder test chip was fabricated in 65-nm CMOS. It can operate at 0.7 V for HD (720p, 30 fps) video decoding and with a measured power of 1.8 mW. The highly scalable decoder can tradeoff power and performance across >100x range. Second, this thesis demonstrates how serial algorithms, such as Context-based Adaptive Binary Arithmetic Coding (CABAC), can be redesigned for parallel architectures to enable high throughput with low coding efficiency cost. A parallel algorithm called the Massively Parallel CABAC (MP-CABAC) is presented that uses syntax element partitions and interleaved entropy slices to achieve better throughput-coding efficiency and throughput-area tradeoffs than H.264/AVC. The parallel algorithm also improves scalability by providing a third dimension to tradeoff coding efficiency for power and performance. Finally, joint algorithm-architecture optimizations are used to increase performance and reduce area with almost no coding penalty. The MP-CABAC is mapped to a highly parallel architecture with 80 parallel engines, which together delivers >10x higher throughput than existing H.264/AVC CABAC implementations. A MP-CABAC test chip was fabricated in 65-nm CMOS to demonstrate the power-performance-coding efficiency tradeoff.by Vivienne. Sze.Ph.D

Roadmap on spatiotemporal light fields

Spatiotemporal sculpturing of light pulse with ultimately sophisticated structures represents the holy grail of the human everlasting pursue of ultrafast information transmission and processing as well as ultra-intense energy concentration and extraction. It also holds the key to unlock new extraordinary fundamental physical effects. Traditionally, spatiotemporal light pulses are always treated as spatiotemporally separable wave packet as solution of the Maxwell's equations. In the past decade, however, more generalized forms of spatiotemporally nonseparable solution started to emerge with growing importance for their striking physical effects. This roadmap intends to highlight the recent advances in the creation and control of increasingly complex spatiotemporally sculptured pulses, from spatiotemporally separable to complex nonseparable states, with diverse geometric and topological structures, presenting a bird's eye viewpoint on the zoology of spatiotemporal light fields and the outlook of future trends and open challenges.Comment: This is the version of the article before peer review or editing, as submitted by an author to Journal of Optics. IOP Publishing Ltd is not responsible for any errors or omissions in this version of the manuscript or any version derived from i