투명한 매질에서의 광 경로 분석을 이용한 집약적 3차원 디스플레이

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2017. 2. 이병호.본 박사학위 논문에서는 광학적으로 투명한 매질에서의 광 경로 분석을 바탕으로 집약적인 3차원 디스플레이 시스템을 구현하는 접근 방법에 대하여 논의한다. 3차원 영상 장치를 구성하는 요소와 시청자 사이의 물리적인 거리를 줄이는 것은 집약적인 3차원 디스플레이 시스템을 구현하는 직관적인 방법이다. 또한, 기존 시스템의 크기를 유지하면서 더 많은 양의 3차원 영상 정보를 표현하는 것 또한 집약적 3차원 디스플레이 시스템을 의미한다. 높은 대역폭과 작은 구조를 가진 집약적 3차원 디스플레이 시스템을 구현하기 위하여 다음의 두 가지 광학 현상을 이용한다. 등방성 물질에서의 전반사 특성과 이방성 물질에서의 복굴절 특성이다. 가시광 영역에서 빛을 투과시키는 두 매질의 고유 광학 특성을 기존의 3차원 디스플레이 시스템에 적용하기 위하여 광 경로 추적을 통하여 분석한다. 광 도파로의 전반사 특성은 집약적 다중 투사 3차원 디스플레이 시스템을 구현하기 위하여 사용한다. 투사 광학계의 영상 정보는 광 도파로로 입사, 내부에서 전반사를 통하여 진행하고, 이에 수평 투사 거리는 광 도파로의 두께로 제한된다. 다수의 전반사 이후 영상 정보는 광 도파로의 출사 면을 통해 빠져나가고, 렌즈에 의하여 최적 시청 지점에서 시점을 형성한다. 광 도파로 내부에서의 광 경로를 등가 모델을 통하여 조사하고, 이를 통해 다수의 투사 광학계로부터 생성된 다수의 시점 영상이 왜곡되는 것을 분석하고 보정한다. 10개의 시점을 제공하는 집약적 다중 투사 3차원 디스플레이 시스템을 통해 제안된 방법을 검증한다. 향상된 대역폭 특성을 가진 다중 투사 3차원 디스플레이와 다중 초점 헤드 마운트 디스플레이 구현을 위한 이방성 판을 이용한 편광 다중화 방법을 제안한다. 빛의 편광 상태, 이방성 판의 광축 방향에 따라 광 경로가 달라진다. 측면 방향으로의 광 경로 전환은 다중 투사 3차원 디스플레이 기술과 결합하여 시점을 측면 방향으로 두 배로 증가시킨다. 깊이 방향으로의 광 경로 전환은 헤드 마운트 디스플레이에서 다중 초점 기능을 구현한다. 광 경로 추적 시뮬레이션을 통해 이방성 판의 모양, 광축, 파장 등의 다양한 파라미터 변화에 따른 광 경로 전환을 분석한다. 각각의 기능에 맞도록 설계된 이방성 판과 편광 회전자를 실시간으로 결합하여, 다중 투사 3차원 디스플레이와 다중 초점 헤드 마운트 디스플레이의 대역폭이 2배 증가한다. 각 시스템에 대한 시작품을 제작하고, 제안된 방법을 실험적으로 검증한다. 본 논문에서는 광 도파로와 복굴절 물질을 이용하여 그 광 경로를 분석, 대형의 다중 투사 3차원 디스플레이 시스템과 개인 사용자의 헤드 마운트 디스플레이 시스템의 크기를 감소시키고, 표현 가능한 정보량을 증가시키는 방법을 제안한다. 광 도파로와 이방성 판은 기존의 3차원 디스플레이 시스템과 쉽게 결합이 가능하며, 제안된 방법은 향후 소형뿐만 아니라 중대형 3차원 디스플레이 시스템의 집약화에 기여할 수 있을 것으로 기대된다.This dissertation investigates approaches for realizing compact three-dimensional (3D) display systems based on optical path analysis in optically transparent medium. Reducing the physical distance between 3D display apparatuses and an observer is an intuitive method to realize compact 3D display systems. In addition, it is considered compact 3D display systems when they present more 3D data than conventional systems while preserving the size of the systems. For implementing compact 3D display systems with high bandwidth and minimized structure, two optical phenomena are investigated: one is the total internal reflection (TIR) in isotropic materials and the other is the double refraction in birefringent crystals. Both materials are optically transparent in visible range and ray tracing simulations for analyzing the optical path in the materials are performed to apply the unique optical phenomenon into conventional 3D display systems. An optical light-guide with the TIR is adopted to realize a compact multi-projection 3D display system. A projection image originated from the projection engine is incident on the optical light-guide and experiences multiple folds by the TIR. The horizontal projection distance of the system is effectively reduced as the thickness of the optical light-guide. After multiple folds, the projection image is emerged from the exit surface of the optical light-guide and collimated to form a viewing zone at the optimum viewing position. The optical path governed by the TIR is analyzed by adopting an equivalent model of the optical light-guide. Through the equivalent model, image distortion for multiple view images in the optical light-guide is evaluated and compensated. For verifying the feasibility of the proposed system, a ten-view multi-projection 3D display system with minimized projection distance is implemented. To improve the bandwidth of multi-projection 3D display systems and head-mounted display (HMD) systems, a polarization multiplexing technique with the birefringent plate is proposed. With the polarization state of the image and the direction of optic axis of the birefringent plate, the optical path of rays varies in the birefringent material. The optical path switching in the lateral direction is applied in the multi-projection system to duplicate the viewing zone in the lateral direction. Likewise, a multi-focal function in the HMD is realized by adopting the optical path switching in the longitudinal direction. For illuminating the detailed optical path switching and the image characteristic such as an astigmatism and a color dispersion in the birefringent material, ray tracing simulations with the change of optical structure, the optic axis, and wavelengths are performed. By combining the birefringent material and a polarization rotation device, the bandwidth of both the multi-projection 3D display and the HMD is doubled in real-time. Prototypes of both systems are implemented and the feasibility of the proposed systems is verified through experiments. In this dissertation, the optical phenomena of the TIR and the double refraction realize the compact 3D display systems: the multi-projection 3D display for public and the multi-focal HMD display for individual. The optical components of the optical light-guide and the birefringent plate can be easily combined with the conventional 3D display system and it is expected that the proposed method can contribute to the realization of future 3D display systems with compact size and high bandwidth.Chapter 1 Introduction 10 1.1 Overview of modern 3D display providing high quality 3D images 10 1.2 Motivation of this dissertation 15 1.3 Scope and organization 18 Chapter 2 Compact multi-projection 3D displays with optical path analysis of total internal reflection 20 2.1 Introduction 20 2.2 Principle of compact multi-projection 3D display system using optical light-guide 23 2.2.1 Multi-projection 3D display system 23 2.2.2 Optical light-guide for multi-projection 3D display system 26 2.2.3 Analysis on image characteristics of projection images in optical light-guide 34 2.2.4 Pre-distortion method for view image compensation 44 2.3 Implementation of prototype of multi-projection 3D display system with reduced projection distance 47 2.4 Summary and discussion 52 Chapter 3 Compact multi-projection 3D displays with optical path analysis of double refraction 53 3.1 Introduction 53 3.2 Principle of viewing zone duplication in multi-projection 3D display system 57 3.2.1 Polarization-dependent optical path switching in birefringent crystal 57 3.2.2 Analysis on image formation through birefringent plane-parallel plate 60 3.2.3 Full-color generation of dual projection 64 3.3 Implementation of prototype of viewing zone duplication of multi-projection 3D display system 68 3.3.1 Experimental setup for viewing zone duplication of multi-projection 3D display system 68 3.3.2 Luminance distribution measurement of viewing zone duplication of multi-projection 3D display system 74 3.4 Summary and discussion 79 Chapter 4 Compact multi-focal 3D HMDs with optical path analysis of double refraction 81 4.1 Introduction 81 4.2 Principle of multi-focal 3D HMD system 86 4.2.1 Multi-focal 3D HMD system using Savart plate 86 4.2.2 Astigmatism compensation by modified Savart plate 89 4.2.3 Analysis on lateral chromatic aberration of extraordinary plane 96 4.2.4 Additive type compressive light field display 101 4.3 Implementation of prototype of multi-focal 3D HMD system 104 4.4 Summary and discussion 112 Chapter 5 Conclusion 114 Bibliography 117 Appendix 129 초 록 130Docto

Tracking people within groups with rgb-d data

Abstract-This paper proposes a very fast and robust multi-people tracking algorithm suitable for mobile platforms equipped with a RGB-D sensor. Our approach features a novel depth-based sub-clustering method explicitly designed for detecting people within groups or near the background and a three-term joint likelihood for limiting drifts and ID switches. Moreover, an online learned appearance classifier is proposed, that robustly specializes on a track while using the other detections as negative examples. Tests have been performed with data acquired from a mobile robot in indoor environments and on a publicly available dataset acquired with three RGB-D sensors and results have been evaluated with the CLEAR MOT metrics. Our method reaches near state of the art performance and very high frame rates in our distributed ROS-based CPU implementation

High-speed Volumetric Functional Imaging with Light Field Microscopy

The continuous advancement in microscopy has been unveiling the hidden world of tissues, cells, and molecules. In the quest for deeper spatiotemporal insights into biological processes, light field microscopy (LFM) has emerged as a powerful and intriguing tool. Unlike traditional imaging systems that capture focused images, LFM records multiplexed signals with single snapshot that encodes information within a three-dimensional (3D) volume. By leveraging computational reconstruction algorithms, this approach enables the observation of transient volumetric dynamics with remarkable efficiency and speed. This thesis presents a series of efforts to apply LFM in functional imaging, enabling researchers to monitor real-time changes in live organisms, including ion fluxes, electrical signaling, and cells interactions. The exceptional temporal resolution makes LFM a unique tool to visualize and analyze rapid processes that are difficult to capture with conventional 3D microscopy. We demonstrated calcium imaging of motor neurons in freely moving C. elegans and tracked flowing blood cells in-vivo within a beating zebrafish heart. The excessive and unpredictable motion observed in these processes requires capturing hundreds of 3D volumes per second, a demanding but necessary task that provides insights into the underlying mechanisms of neural and cardiac functions. We further moved forward to voltage imaging, a frontier in neuroscience, which directly measures the neural action potential as well as sub-thresholding activities. Our LFM provides kilohertz volumetric imaging on leech ganglion and mouse hippocampus. It measures 7.3 gigavoxels per second in a 3D field of view of 550×550×300 μm^3, which makes it capable of recording the accurate timing and waveform of neural spikes across entire volume. These demonstrations are achieved through several innovative redesigns of LFM, detailed in Chapter 3 to 5. The first approach, VCD-LFM, addresses the inherent trade-off between spatial resolution and depth information in light field imaging by introducing a learning-based reconstruction algorithm. By incorporating data priors and constraints, this method aims to mitigate the issues of low spatial resolution and artifacts in conventional LFM without compromising imaging speed. The second approach, Squeezed Light Field Microscopy (SLIM), leverages data redundancy in light fields and revises the optical hardware to achieve kilohertz volume rate. Designed to meet the high-speed demands of voltage imaging, SLIM offers a powerful and robust imaging tool for sparse volumetric processes. Lastly, the third approach, Light Field Tomography (LIFT), adapts LFM for one-dimensional (1D) measurements through optical Radon transformation. This method enables the use of low-dimensional detectors, such as line sensor, to capture high-dimensional light fields, resulting in enhanced sensitivity, reduced cost and even greater temporal resolution

Electron-optical cameras with ultrahigh time resolution in one and two spatial dimensions

Imperial Users onl

Modern lithographic techniques applied to stereographic imaging

The main aim of the research has been to produce and evaluate a high-quality diffusion screen to display projected film and television images. The screens have also been found to effectively de-pixelate LCD arrays viewed at a magnification of approximately 4x. The production process relies on the formation of localized refractive index gradients in a photopolymer. The photopolymer, specially formulated and supplied by Du Pont, is exposed to actinic light through a precision contact mask to initiate polymerization within the exposed areas. As polymerization proceeds, a monomer concentration gradient exists between the exposed and unexposed regions allowing the monomer molecules to diffuse. Since the longer polymer chains do not diffuse as readily, the molecular concentration of the material, which is related to its refractive index, is then no longer uniform. The generation of this refractive index profile can, to some extent, be controlled by careful exposure of the photopolymer through the correct mask so that the resulting diffusion screen can be tailored to suit specific viewing requirements. [Continues.

Optimized Distance Measurement with 3D-CMOS Image Sensor and Real Time Processing of the 3D Data for Applications in Automotive and Safety Engineering

This thesis describes and characterizes an advanced range camera for the distance range from 2 m to 25 m and novel real-time 3D image processing algorithms for object detection, tracking and classification on the basis of the three-dimensional features of the camera output data. The technology is based on a 64x8 pixel array CMOS image sensor which is capable of capturing three-dimensional images. This is accomplished by executing indirect time of flight measurement of NIR laser pulses emitted by the camera and reflected by the objects in the field of view of the camera. An analytic description of the measurement signals and a derivation of the distance measuring algorithms are conducted in this thesis as well as a comparative examination of the distance measuring algorithms by calculation, simulation and experiments; in doing so, the MDSI3 algorithm showed the best results over the whole measurement range and is thus chosen as standard method of the distance measuring system. A camera prototype was developed with a measurement accuracy in the centimeter range at an image repetition rate up to 100 Hz; a detailed evaluation of the components and of the over-all system is presented. Main aspects are the characterization of the time critical measurement signals, of the system noise, and of the distance measuring capabilities. Furthermore this thesis introduces novel real-time image processing of the output data stream of the camera aiming at the detection of objects being located in the observed area and the derivation of reliable position, speed and acceleration estimates. The used segmentation algorithm utilizes all three spatial dimensions of the position information as well as the intensity values and thus yields significant improvement compared to segmentation in conventional 2D images. Position, velocity, and acceleration values of the segmented objects are estimated by means of Kalman filtering in 3D space. The filter is dynamically adapted to the measurement properties of the according object to take care of changes of the data properties. The good performance of the image processing algorithms is presented by means of example scenes.Optimierte Entfernungsmessung mit 3D-CMOS Bildsensor und Echtzeit-Verarbeitung der 3D-Daten für Anwendungen in der Automobil- und Sicherheitstechnik Diese Arbeit beschreibt und charakterisiert eine neu entwickelte Entfernungskamera für Reichweiten von 2 m bis 25 m und spezielle 3D-Echtzeit-Bildverarbeitungsalgorithmen zum Detektieren, Tracken und Klassifizieren von Objekten auf der Grundlage der dreidimensionalen Kameradaten. Die Technologie basiert auf einem 64x8 Pixel CMOS Bildsensor, welcher im Stande ist, dreidimensionale Szenen zu erfassen. Dies wird mittels indirekter Laufzeitmessung von NIR Laserpulsen, die von der Kamera ausgesandt und an Objekten im Blickfeld der Kamera reflektiert werden, realisiert. Eine analytische Beschreibung der Messsignale und eine darauf aufbauende Herleitung der verschiedenartigen Entfernungsmessungsalgorithmen wird in dieser Arbeit ebenso durchgefhrt, wie die vergleichende Betrachtung der Entfernungsmessungsalgorithmen durch Rechnung, Simulation und Experimente; dabei zeigt der MDSI3-Algorithmus die besten Ergebnisse über den gesamten Messbereich, und wird deshalb zum Standardalgorithmus des Entfernungsmesssystems. Ein Kameraprototyp mit Messgenauigkeiten im cm-Bereich bei einer Bildwiederholrate bis zu 100 Hz wurde entwickelt; eine detaillierte Evaluierung der Komponenten und des Systems ist hier beschrieben. Hauptaspekte sind dabei die Charakterisierung der zeitkritischen Messsignale, des Systemrauschens und der Entfernungsmesseigenschaften. Desweiteren wird in dieser Arbeit die neu entwickelte Echtzeit-Bildverarbeitung des Kameradatenstroms vorgestellt, die auf die Detektion von Objekten im Beobachtungsbereich und die verlässliche Ermittlung von Positions-,Geschwindigkeits- und Beschleunigungsschtzwerten abzielt. Der dabei verwendete Segmentierungsalgorithmus nutzt alle drei Dimensionen der Positionsmesswerte kombiniert mit den Intensitätswerten der Messsignale, und liefert so eine signifikante Verbesserung im Vergleich zur Segmentierung in konventionellen 2D Bildern. Position, Geschwindigkeit und Beschleunigung werden mit Hilfe eines Kalman-Filters im 3-dimensionalen Raum geschätzt. Das Filter passt sich dynamisch den Messbedingungen des jeweils gemessenen Objekts an, und berücksichtigt so Veränderungen der Dateneigenschaften. Die Leistungsfähigkeit der Bildverarbeitungsalgorithmen wird anhand von Beispielszenen demonstriert

Robust perception of humans for mobile robots RGB-depth algorithms for people tracking, re-identification and action recognition

Human perception is one of the most important skills for a mobile robot sharing its workspace with humans. This is not only true for navigation, because people have to be avoided differently than other obstacles, but also because mobile robots must be able to truly interact with humans. In a near future, we can imagine that robots will be more and more present in every house and will perform services useful to the well-being of humans. For this purpose, robust people tracking algorithms must be exploited and person re-identification techniques play an important role for allowing robots to recognize a person after a full occlusion or after long periods of time. Moreover, they must be able to recognize what humans are doing, in order to react accordingly, helping them if needed or also learning from them. This thesis tackles these problems by proposing approaches which combine algorithms based on both RGB and depth information which can be obtained with recently introduced consumer RGB-D sensors. Our key contribution to people detection and tracking research is a depth-clustering method which allows to apply a robust image-based people detector only to a small subset of possible detection windows, thus decreasing the number of false detections while reaching high computational efficiency. We also advance person re-identification research by proposing two techniques exploiting depth-based skeletal tracking algorithms: one is targeted to short-term re-identification and creates a compact, yet discrimative signature of people based on computing features at skeleton keypoints, which are highly repeatable and semantically meaningful; the other extract long-term features, such as 3D shape, to compare people by matching the corresponding 3D point cloud acquired with a RGB-D sensor. In order to account for the fact that people are articulated and not rigid objects, it exploits 3D skeleton information for warping people point clouds to a standard pose, thus making them directly comparable by means of least square fitting. Finally, we describe an extension of flow-based action recognition methods to the RGB-D domain which computes motion over time of persons' 3D points by exploiting joint color and depth information and recognizes human actions by classifying gridded descriptors of 3D flow. A further contribution of this thesis is the creation of a number of new RGB-D datasets which allow to compare different algorithms on data acquired by consumer RGB-D sensors. All these datasets have been publically released in order to foster research in these fields

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Proceedings of the 2010 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

On the annual Joint Workshop of the Fraunhofer IOSB and the Karlsruhe Institute of Technology (KIT), Vision and Fusion Laboratory, the students of both institutions present their latest research findings on image processing, visual inspection, pattern recognition, tracking, SLAM, information fusion, non-myopic planning, world modeling, security in surveillance, interoperability, and human-computer interaction. This book is a collection of 16 reviewed technical reports of the 2010 Joint Workshop