An automated calibration method for non-see-through head mounted displays

Accurate calibration of a head mounted display (HMD) is essential both for research on the visual system and for realistic interaction with virtual objects. Yet, existing calibration methods are time consuming and depend on human judgements, making them error prone, and are often limited to optical see-through HMDs. Building on our existing approach to HMD calibration Gilson et al. (2008), we show here how it is possible to calibrate a non-see-through HMD. A camera is placed inside a HMD displaying an image of a regular grid, which is captured by the camera. The HMD is then removed and the camera, which remains fixed in position, is used to capture images of a tracked calibration object in multiple positions. The centroids of the markers on the calibration object are recovered and their locations re-expressed in relation to the HMD grid. This allows established camera calibration techniques to be used to recover estimates of the HMD display's intrinsic parameters (width, height, focal length) and extrinsic parameters (optic centre and orientation of the principal ray). We calibrated a HMD in this manner and report the magnitude of the errors between real image features and reprojected features. Our calibration method produces low reprojection errors without the need for error-prone human judgements

FocalSpace: Multimodal Activity Tracking, Synthetic Blur and Adapative Presentation for Video Conferencing

We introduce FocalSpace, a video conferencing system that dynamically recognizes relevant activities and objects through depth sensing and hybrid tracking of multimodal cues, such as voice, gesture, and proximity to surfaces. FocalSpace uses this information to enhance users' focus by diminishing the background through synthetic blur effects. We present scenarios that support the suppression of visual distraction, provide contextual augmentation, and enable privacy in dynamic mobile environments. Our user evaluation indicates increased memory accuracy and user preference for FocalSpace techniques compared to traditional video conferencing

Elements of design for indoor visualisation

Indoor visualisation has received little attention. Research related to indoor environments have primarily focussed on the data structuring, localisation and navigation components (Zlatanova et al., 2013). Visualisation is an integral component in addressing the diverse array of indoor environments. In simple words, 'What is the most efficient way to visualise the surrounding indoor environment so that the user can concisely understand their surroundings as well as facilitating the process of navigation?' This dissertation proposes a holistic approach that consists of two components. The significance of this approach is that it provides a robust and adaptable method in providing a standard to which indoor visualisation can be referenced against. The first component is a theoretical framework focussing on indoor visualisation and it comprises of principles from several disciplines such as geovisualisation, human-perception theory, spatial cognition, dynamic and 3D environments as well as accommodating emotional processes resulting from human-computer interaction. The second component is based on the theoretical framework and adopts a practical approach towards indoor visualisation. It consists of a set of design properties that can be used for the design of effective indoor visualisations. The framework is referred to as the "Elements of Design" framework. Both these components aim to provide a set of principles and guidelines that can be used as best practices for the design of indoor visualisations. In order to practically demonstrate the holistic indoor visualisation approach, multiple indoor visualisation renderings were developed. The visualisation renderings were represented in a three-dimensional virtual environment from a first-person perspective. Each rendering used the design framework differently. Also, each rendering was graded using a parallel chart that compares how the different visual elements were used per the rendering. The main findings were that the techniques/ renderings that used the visual elements effectively (enhanced human-perception) resulted in better acquisition and construction of knowledge about the surrounding indoor environment

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Unstructured light fields

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 35-38).We present a system for interactively acquiring and rendering light fields using a hand-held commodity camera. The main challenge we address is assisting a user in achieving good coverage of the 4D domain despite the challenges of hand-held acquisition. We define coverage by bounding reprojection error between viewpoints, which accounts for all 4 dimensions of the light field. We use this criterion together with a recent Simultaneous Localization and Mapping technique to compute a coverage map on the space of viewpoints. We provide users with real-time feedback and direct them toward under-sampled parts of the light field. Our system is lightweight and has allowed us to capture hundreds of light fields. We further present a new rendering algorithm that is tailored to the unstructured yet dense data we capture. Our method can achieve piecewise-bicubic reconstruction using a triangulation of the captured viewpoints and subdivision rules applied to reconstruction weights.by Myers Abraham Davis (Abe Davis).S.M

Methods for reducing visual discomfort in stereoscopic 3D: A review

This work was supported by the EPSRC Grant EP/M01469X/1, “Geometric Evaluation of Stereoscopic Video”

비등방성 광학 소자를 이용한 광 시야각 근안 디스플레이

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 전기·컴퓨터공학부, 2019. 2. 이병호.Near-eye display is considered as a promising display technique to realize augmented reality by virtue of its high sense of immersion and user-friendly interface. Among the important performances of near-eye display, a field of view is the most crucial factor for providing a seamless and immersive experience for augmented reality. In this dissertation, a transmissive eyepiece is devised instead of a conventional reflective eyepiece and it is discussed how to widen the field of view without loss of additional system performance. In order to realize the transmissive eyepiece, the eyepiece should operate lens to virtual information and glass to real-world scene. Polarization multiplexing technique is used to implement the multi-functional optical element, and anisotropic optical elements are used as material for multi-functional optical element. To demonstrate the proposed idea, an index-matched anisotropic crystal lens has been presented that reacts differently depending on polarization. With the combination of isotropic material and anisotropic crystal, the index-matched anisotropic crystal lens can be the transmissive eyepiece and achieve the large field of view. Despite the large field of view by the index-matched anisotropic crystal lens, many problems including form factor still remain to be solved. In order to overcome the limitations of conventional optics, a metasurface is adopted to the augmented reality application. With a stunning optical performance of the metasurface, a see-through metasurface lens is proposed and designed for implementing wide field of view near-eye display. The proposed novel eyepieces are expected to be an initiative study not only improving the specification of the existing near-eye display but opening the way for a next generation near-eye display.근안 디스플레이는 높은 몰입감과 사용자 친화적인 인터페이스로 인해 증강 현실을 구현하는 가장 효과적인 기술로 최근 활발한 연구가 계속되고 있다. 이러한 근안 디스플레이의 중요한 성능 중 시야각은 매끄럽고 몰입감 있는 경험을 사용자에게 전해줌으로써 가장 중요한 광학적 평가지표 중에 하나이다. 본 논문에서는 기존의 반사형 아이피스 (eyepiece) 를 대신하는 투과형 아이피스를 제안한다. 이러한 투과형 아이피스를 구현하기 위해서는 외부 정보에 대해서는 투명한 유리와 같이 투과시키며, 동시에 가상 정보는 렌즈로 작동하여 먼 거리에 띄울 수 있는 광학소자를 개발하여야 한다. 이러한 투과형 아이피스를 구현하기 위해서 편광에 따라 다르게 반응하는 굴절률 정합 이방성 결정 렌즈 (index-matched anisotropic crystal lens) 를 제안하였다. 이방성 결정 구조 (anisotropic crystal)로 이루어진 렌즈와 이를 둘러싼 등방성 물질 (isotropic crytal) 로 이루어진 굴절률 정합 이방성 결정 렌즈는 편광에 따라 다르게 작동한다. 이러한 투과형 아이피스는 기존의 근안 디스플레이에 비해 넓은 시야각을 제공할 수 있지만 이방성 결정 구조의 낮은 굴절률 차이로 인해 시스템의 크기가 커지는 단점을 가지고 있다. 본 논문에서는 이러한 단점을 개선하기 위해 메타 표면을 증강 현실 디스플레이 분야에 적용하였다. 메타 표면의 기존 광학 소자를 능가하는 놀라운 광학 성능을 이용하여 넓은 시야각을 가지는 근안 디스플레이를 구현하기 위해 투명 메타 렌즈를 제안하였다. 편광에 따라 다르게 반응하는 투명 메타렌즈는 넓은 시야각과 경량화 시스템 구현이 가능하며 이를 입증하기 위해 투명 메타렌즈의 설계 방법 뿐 아니라 실제 구현을 통한 가능성을 입증하였다. 이러한 새로운 아이피스에 대한 개념은 기존의 근안 디스플레이의 사양 개선에 유용하게 사용될 뿐 아니라 차세대 근안 디스플레이를 위한 선도적인 역할을 할 것으로 기대된다.Abstract Contents List of Tables List of Figures Near-eye displays with wide field of view using anisotropic optical elements Chapter 1 Introduction 1.1 Near-eye displays for augmented reality 1.2 Optical performances of near-eye display 1.3 State-of-the-arts of near-eye display 1.4 Motivation and contribution of this dissertation Chapter 2 Transmissive eyepiece for wide field of view near-eye display 2.1 Transmissive eyepiece for near-eye display Chapter 3 Near-eye display using index-matched anisotropic crystal lens 3.1 Introduction 3.2 Index-matched anisotropic crystal lens 3.2.1 Principle of the index-matched anisotropic crystal lens 3.2.2 Aberration analysis of index-matched anisotropic crystal lens 3.2.3 Implementation 3.3 Near-eye displays using index-matched anisotropic crystal lens 3.3.1 Near-eye display using index-matched anisotropic crystal lens 3.3.2 Flat panel type near-eye display using IMACL 3.3.3 Polarization property of transparent screen 3.4 Conclusion Chapter 4 Near-eye display using metasurface lens 4.1 Introduction 4.2 See-through metasurface lens 4.2.1 Metasurface lens 4.3 Full-color near-eye display using metasurface lens 4.3.1 Full-color near-eye display using metasurface lens 4.3.2 Holographic near-eye display using metasurface lens for aberration compensation 4.4 Conclusion Chapter 5 Conclusion Bibliography AppendixDocto

Integrative visual augmentation content and its optimization based on human visual processing

In many daily visual tasks, our brain is remarkably good at prioritizing visual information. Nonetheless, it is undoubtedly not always capable of performing optimally, and all the more so in the ever-evolving demanding world. Supplementary visual guidance could enrich our lives from many perspectives on the individual and population scales. Through rapid technological advancements such as VR and AR systems, diverse visual cues demonstrate a powerful potential to deliberately guide attention and improve users’ performance in daily tasks. Currently, existing solutions are confronting the challenge of overloading and overruling the natural strategy of the user with excessive visual information once digital content is superimposed on the real-world environment. The subtle nature of augmentation content, which considers human visual processing factors, is an essential milestone towards developing adaptive, supportive, and not overwhelming AR systems. The focus of the present thesis was, thus, to investigate how the manipulation of spatial and temporal properties of visual cues affects human performance. Based on the findings of three studies published in peer-reviewed journals, I consider various everyday challenging settings and propose perceptually optimal augmentation solutions. I furthermore discuss possible extensions of the present work and recommendations for future research in this exciting field

Efficient Distance Accuracy Estimation Of Real-World Environments In Virtual Reality Head-Mounted Displays

Virtual reality (VR) is a very promising technology with many compelling industrial applications. As many advancements have been made recently to deploy and use VR technology in virtual environments, they are still less mature to be used to render real environments. The current VR systems settings, which are developed for virtual environments rendering, fail to adequately address the challenges of capturing and displaying real-world virtual reality that these systems entail. Before these systems can be used in real life settings, their performance needs to be investigated, more specifically, depth perception and how distances to objects in the rendered scenes are estimated. The perceived depth is influenced by Head Mounted Displays (HMD) that inevitability decrease the virtual content’s depth perception. Distances are consistently underestimated in virtual environments (VEs) compared to the real world. The reason behind this underestimation is still not understood. This thesis investigates another version of this kind of system, that to the best of authors knowledge has not been explored by any previous research. Previous research used a computer-generated scene. This work is examining distance estimation in real environments rendered to Head-Mounted Displays, where distance estimations is among the most challenging issues that are still investigated and not fully understood.This thesis introduces a dual-camera video feed system through a virtual reality head mounted display with two models: a video-based and a static photo-based model, in which, the purpose is to explore whether the misjudgment of distances in HMDs could be due to a lack of realism, or not, with the use of a real-world scene rendering system. Distance judgments performance in the real world and these two evaluated VE models were compared using protocols already proven to accurately measure real-world distance estimations. An improved model based on enhancing the field of view (FOV) of the displayed scenes to improve distance judgements when displaying real-world VR content to HMDs was developed; allowing to mitigate the limited FOV, which is among the first potential causes of distance underestimation, specially, the mismatch of FOV between the camera and the HMD field of views. The proposed model is using a set of two cameras to generate the video instead of hundreds of input cameras or tens of cameras mounted on a circular rig as previous works from the literature. First Results from the first implementation of this system found that when the model was rendered as static photo-based, the underestimation was less as compared with the live video feed rendering. The video-based (real + HMD) model and the static photo-based (real + photo + HMD) model averaged 80.2% of the actual distance, and 81.4% respectively compared to the Real-World estimations that averaged 92.4%. The improved developed approach (Real + HMD + FOV) was compared to these two models and showed an improvement of 11%, increasing the estimation accuracy from 80% to 91% and reducing the estimation error from 1.29% to 0.56%. This thesis results present strong evidence of the need for novel distance estimation improvements methods for real world VR content systems and provides effective initial work towards this goal