3DTouch: A wearable 3D input device with an optical sensor and a 9-DOF inertial measurement unit

We present 3DTouch, a novel 3D wearable input device worn on the fingertip for 3D manipulation tasks. 3DTouch is designed to fill the missing gap of a 3D input device that is self-contained, mobile, and universally working across various 3D platforms. This paper presents a low-cost solution to designing and implementing such a device. Our approach relies on relative positioning technique using an optical laser sensor and a 9-DOF inertial measurement unit. 3DTouch is self-contained, and designed to universally work on various 3D platforms. The device employs touch input for the benefits of passive haptic feedback, and movement stability. On the other hand, with touch interaction, 3DTouch is conceptually less fatiguing to use over many hours than 3D spatial input devices. We propose a set of 3D interaction techniques including selection, translation, and rotation using 3DTouch. An evaluation also demonstrates the device's tracking accuracy of 1.10 mm and 2.33 degrees for subtle touch interaction in 3D space. Modular solutions like 3DTouch opens up a whole new design space for interaction techniques to further develop on.Comment: 8 pages, 7 figure

Construction and Evaluation of an Ultra Low Latency Frameless Renderer for VR.

© 2016 IEEE.Latency-the delay between a users action and the response to this action-is known to be detrimental to virtual reality. Latency is typically considered to be a discrete value characterising a delay, constant in time and space-but this characterisation is incomplete. Latency changes across the display during scan-out, and how it does so is dependent on the rendering approach used. In this study, we present an ultra-low latency real-time ray-casting renderer for virtual reality, implemented on an FPGA. Our renderer has a latency of 1 ms from tracker to pixel. Its frameless nature means that the region of the display with the lowest latency immediately follows the scan-beam. This is in contrast to frame-based systems such as those using typical GPUs, for which the latency increases as scan-out proceeds. Using a series of high and low speed videos of our system in use, we confirm its latency of 1 ms. We examine how the renderer performs when driving a traditional sequential scan-out display on a readily available HMO, the Oculus Rift OK2. We contrast this with an equivalent apparatus built using a GPU. Using captured human head motion and a set of image quality measures, we assess the ability of these systems to faithfully recreate the stimuli of an ideal virtual reality system-one with a zero latency tracker, renderer and display running at 1 kHz. Finally, we examine the results of these quality measures, and how each rendering approach is affected by velocity of movement and display persistence. We find that our system, with a lower average latency, can more faithfully draw what the ideal virtual reality system would. Further, we find that with low display persistence, the sensitivity to velocity of both systems is lowered, but that it is much lower for ours

홀로그래픽 프린터를 이용한 증강현실 디스플레이의 맞춤형 홀로그래픽 광학 소자 제작

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 이병호.This dissertation presents the studies on the design and fabrication method of a holographic optical element (HOE) for augmented reality (AR) near-eye display (NED) by using a holographic printing technique. The studies enable us to manufacture HOEs based on the digitalized design process and allow more freedom to design HOEs, beyond the conventional HOE manufacturing process. The manufactured HOE can play the role of the image combiner of the AR NED and can be designed precisely according to each users distinctive characteristics. The prototype of the HOE printer is presented and the structure is analyzed. The HOE printer can record a hogel with 1900 × 1900 pixels in 1 mm2 and can give complex wavefront information via using an amplitude SLM and sideband filtering technique. The author adopts an index-matching frame with a passive optical isolator, which consists of quarter waveplates and linear polarizers, to eliminate the internal reflection noise. With the HOE printer, a lens HOE with field of view (FOV) 50° is manufactured, and a holographic AR NED is implemented with the lens HOE. The experimental result shows the lens HOE and the HOE printer work properly as our purpose. Using the prototype HOE printer, the author proposes two types of novel AR NEDs. First, the author suggests a customized HOE for an eye-box extended holographic AR NED. The limitation of the conventional holographic AR NED is that the eye-box becomes very narrow when large FOV is implemented due to the limited spatial bandwidth product. By using the proposed HOE printer, the eye-box can be extended along with both horizontal and vertical directions without any mechanical scanning devices. Also, the position of the extended eye-box can be designed to fit with the movement of the eye pupil. This prevents the vignetting effect due to the eye-box mismatch. Second, the author presents a freeform mirror array (FMA) HOE and implement a retinal projection AR NED with the HOE. By using the FMA HOE, the holographic mirrors no longer block the sight of the observer. Also, the freeform phase function allows the FMA HOE to float the display to the desired location without any additional optics, such as a lens. In this way, a wide depth of field and extended eye-box retinal projection AR NED with a compact form factor is implemented. It is expected that this dissertation can help to develop a customized AR NED based on the customers needs. Furthermore, it is believed that this work can show new possibilities for research on the design and fabrication of HOEs.본 박사학위 논문에서는 근안 증강현실 디스플레이의 홀로그래픽 영상 결합 소자를 홀로그래픽 프린팅 기술을 이용하여 설계 및 제작하는 방법에 대하여 논한다. 이를 통하여 기존의 아날로그 방법에 의존한 홀로그래픽 광학 소자 제작 기법을 디지털화 할 수 있다. 또한 홀로그래픽 광학 소자의 설계 자유도가 증가하여 사용자 특징에 따른 근안 증강현실 디스플레이의 맞춤형 영상 결합 소자를 제작할 수 있다. 이 박사학위 논문에서는 홀로그래픽 광학 소자 프린터의 프로토타입을 제작 및 소개한다. 해당 프로토타입은 1 mm2의 면적 안에 1900 × 1900 복소 광파 정보를 표현 할 수 있다. 광파의 복소 변조를 위하여 진폭 변조 공간광변조를 이용한 sideband filtering 기법이 사용된다. 또한 굴절률이 보상된 프레임에 1/4 파장판 및 선형 편광자를 이용한 수동 광분리소자를 적용하여 홀로그래픽 광학 소자를 기록 할 때 발생하는 내부 반사 노이즈를 효과적으로 제거할 수 있다. 이와 같은 홀로그래픽 프린터의 프로토타입이 의도한 대로 제작되었음을 검증하기 위하여 홀로그래픽 광학 소자 렌즈를 제작 및, 해당 홀로그래픽 광학 소자 렌즈가 근안 홀로그래픽 증강현실 디스플레이의 영상 결합 소자로 사용될 수 있음을 보인다. 제작된 홀로그래픽 광학 소자 프린터를 이용하여 두 가지의 새로운 근안 증강현실 디스플레이를 제안한다. 첫 번째는 시청영역이 증가한 근안 홀로그래픽 증강현실 디스플레이로, 공간대역폭에 의하여 제한된 시청 영역을 수직 및 수평 방향으로 동시에 확장할 수 있다. 또한 확장된 시청 영역은 사용자의 안구 길이 및 회전 각도에 맞춰 설계되어 시청영역 불일치로 인한 비네팅 등의 이미지 왜곡을 최소화한다. 마지막으로 망막투사 형태의 근안 증강현실 디스플레이에 사용될 수 있는 프리폼 거울 어레이 홀로그래픽 광학 소자를 제안한다. 이를 이용하여, 기존 거울 어레이 기반의 망막투사 디스플레이의 문제점 중 하나인 거울이 시야를 가리는 문제를 해결한다. 또한 홀로그래픽 거울 배열에 위상 변조 패턴을 기록하여 추가적인 렌즈 등의 광학계 없이 원하는 깊이에 디스플레이 평면을 띄울 수 있게 된다. 이를 이용하여 작은 폼팩터의 넓은 깊이 표현 범위를 지니는 망막투사형 근안 증강현실 디스플레이를 구현한다. 본 박사학위 논문의 결과는 사용자의 필요에 기반한 맞춤형 근안 증강현실 디스플레이의 개발에 도움이 될 것으로 기대된다. 나아가, 본 연구는 홀로그래픽 광학 소자의 설계와 제작에 관한 연구의 새로운 가능성을 보여줄 것으로 기대된다.1 Introduction 1 1.1 Image combiners of augmented reality near-eye display 1 1.2 Motivation and purpose of this dissertation 8 1.3 Scope and organization 10 2 Holographic optical element printer 12 2.1 Introduction 12 2.2 Overview of the prototype of holographic optical element printer 16 2.3 Analysis of the signal path 21 2.4 Considerations in designing an HOE 27 2.5 Removal of the internal reflection noise using passive optical isolator 32 2.6 Manufacturing customized lens holographic optical element 37 2.7 Discussion 41 2.7.1 HOE printer to modulate both signal and reference beams 41 2.7.2 The term "hogel" used in this dissertation 41 2.8 Summary 44 3 Holographically customized optical combiner for eye-box extended near-eye display 45 3.1 Introduction 45 3.2 Proposed method and its implementation 51 3.3 Implemented prototype 57 3.4 Experiments and results 61 3.5 Discussion 63 3.5.1 Vignetting effect from mismatched pupil position along axial direction 63 3.5.2 Diffraction efficiency simulation according to incident angle 65 3.6 Summary 67 4 Holographically printed freeform mirror array for augmented reality near-eye display 68 4.1 Introduction 68 4.2 Retinal projection NED based on small aperture array 70 4.3 Proposed method 72 4.4 Design method of FMA HOE 75 4.4.1 Depth of field analysis 75 4.4.2 The size of the mirror 77 4.4.3 The distance between the mirrors 79 4.5 Experiments and results 82 4.6 Discussion 86 4.6.1 Eye-box of the system via the angular selectivity of the HOE 86 4.7 Summary 89 5 Conclusion 90 Appendix 104 Abstract (In Korean) 105Docto

Framework to Enhance Teaching and Learning in System Analysis and Unified Modelling Language

Cowling, MA ORCiD: 0000-0003-1444-1563; Munoz Carpio, JC ORCiD: 0000-0003-0251-5510Systems Analysis modelling is considered foundational for Information and Communication Technology (ICT) students, with introductory and advanced units included in nearly all ICT and computer science degrees. Yet despite this, novice systems analysts (learners) find modelling and systems thinking quite difficult to learn and master. This makes the process of teaching the fundamentals frustrating and time intensive. This paper will discuss the foundational problems that learners face when learning Systems Analysis modelling. Through a systematic literature review, a framework will be proposed based on the key problems that novice learners experience. In this proposed framework, a sequence of activities has been developed to facilitate understanding of the requirements, solutions and incremental modelling. An example is provided illustrating how the framework could be used to incorporate visualization and gaming elements into a Systems Analysis classroom; therefore, improving motivation and learning. Through this work, a greater understanding of the approach to teaching modelling within the computer science classroom will be provided, as well as a framework to guide future teaching activities

On Inter-referential Awareness in Collaborative Augmented Reality

For successful collaboration to occur, a workspace must support inter-referential awareness - or the ability for one participant to refer to a set of artifacts in the environment, and for that reference to be correctly interpreted by others. While referring to objects in our everyday environment is a straight-forward task, the non-tangible nature of digital artifacts presents us with new interaction challenges. Augmented reality (AR) is inextricably linked to the physical world, and it is natural to believe that the re-integration of physical artifacts into the workspace makes referencing tasks easier; however, we find that these environments combine the referencing challenges from several computing disciplines, which compound across scenarios. This dissertation presents our studies of this form of awareness in collaborative AR environments. It stems from our research in developing mixed reality environments for molecular modeling, where we explored spatial and multi-modal referencing techniques. To encapsulate the myriad of factors found in collaborative AR, we present a generic, theoretical framework and apply it to analyze this domain. Because referencing is a very human-centric activity, we present the results of an exploratory study which examines the behaviors of participants and how they generate references to physical and virtual content in co-located and remote scenarios; we found that participants refer to content using physical and virtual techniques, and that shared video is highly effective in disambiguating references in remote environments. By implementing user feedback from this study, a follow-up study explores how the environment can passively support referencing, where we discovered the role that virtual referencing plays during collaboration. A third study was conducted in order to better understand the effectiveness of giving and interpreting references using a virtual pointer; the results suggest the need for participants to be parallel with the arrow vector (strengthening the argument for shared viewpoints), as well as the importance of shadows in non-stereoscopic environments. Our contributions include a framework for analyzing the domain of inter-referential awareness, the development of novel referencing techniques, the presentation and analysis of our findings from multiple user studies, and a set of guidelines to help designers support this form of awareness