Synopsis of an engineering solution for a painful problem Phantom Limb Pain

This paper is synopsis of a recently proposed solution for treating patients who suffer from Phantom Limb Pain (PLP). The underpinning approach of this research and development project is based on an extension of “mirror box” therapy which has had some promising results in pain reduction. An outline of an immersive individually tailored environment giving the patient a virtually realised limb presence, as a means to pain reduction is provided. The virtual 3D holographic environment is meant to produce immersive, engaging and creative environments and tasks to encourage and maintain patients’ interest, an important aspect in two of the more challenging populations under consideration (over-60s and war veterans). The system is hoped to reduce PLP by more than 3 points on an 11 point Visual Analog Scale (VAS), when a score less than 3 could be attributed to distraction alone

Towards High-Frequency Tracking and Fast Edge-Aware Optimization

This dissertation advances the state of the art for AR/VR tracking systems by increasing the tracking frequency by orders of magnitude and proposes an efficient algorithm for the problem of edge-aware optimization. AR/VR is a natural way of interacting with computers, where the physical and digital worlds coexist. We are on the cusp of a radical change in how humans perform and interact with computing. Humans are sensitive to small misalignments between the real and the virtual world, and tracking at kilo-Hertz frequencies becomes essential. Current vision-based systems fall short, as their tracking frequency is implicitly limited by the frame-rate of the camera. This thesis presents a prototype system which can track at orders of magnitude higher than the state-of-the-art methods using multiple commodity cameras. The proposed system exploits characteristics of the camera traditionally considered as flaws, namely rolling shutter and radial distortion. The experimental evaluation shows the effectiveness of the method for various degrees of motion. Furthermore, edge-aware optimization is an indispensable tool in the computer vision arsenal for accurate filtering of depth-data and image-based rendering, which is increasingly being used for content creation and geometry processing for AR/VR. As applications increasingly demand higher resolution and speed, there exists a need to develop methods that scale accordingly. This dissertation proposes such an edge-aware optimization framework which is efficient, accurate, and algorithmically scales well, all of which are much desirable traits not found jointly in the state of the art. The experiments show the effectiveness of the framework in a multitude of computer vision tasks such as computational photography and stereo.Comment: PhD thesi

홀로그래픽 프린터를 이용한 증강현실 디스플레이의 맞춤형 홀로그래픽 광학 소자 제작

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 이병호.This dissertation presents the studies on the design and fabrication method of a holographic optical element (HOE) for augmented reality (AR) near-eye display (NED) by using a holographic printing technique. The studies enable us to manufacture HOEs based on the digitalized design process and allow more freedom to design HOEs, beyond the conventional HOE manufacturing process. The manufactured HOE can play the role of the image combiner of the AR NED and can be designed precisely according to each users distinctive characteristics. The prototype of the HOE printer is presented and the structure is analyzed. The HOE printer can record a hogel with 1900 × 1900 pixels in 1 mm2 and can give complex wavefront information via using an amplitude SLM and sideband filtering technique. The author adopts an index-matching frame with a passive optical isolator, which consists of quarter waveplates and linear polarizers, to eliminate the internal reflection noise. With the HOE printer, a lens HOE with field of view (FOV) 50° is manufactured, and a holographic AR NED is implemented with the lens HOE. The experimental result shows the lens HOE and the HOE printer work properly as our purpose. Using the prototype HOE printer, the author proposes two types of novel AR NEDs. First, the author suggests a customized HOE for an eye-box extended holographic AR NED. The limitation of the conventional holographic AR NED is that the eye-box becomes very narrow when large FOV is implemented due to the limited spatial bandwidth product. By using the proposed HOE printer, the eye-box can be extended along with both horizontal and vertical directions without any mechanical scanning devices. Also, the position of the extended eye-box can be designed to fit with the movement of the eye pupil. This prevents the vignetting effect due to the eye-box mismatch. Second, the author presents a freeform mirror array (FMA) HOE and implement a retinal projection AR NED with the HOE. By using the FMA HOE, the holographic mirrors no longer block the sight of the observer. Also, the freeform phase function allows the FMA HOE to float the display to the desired location without any additional optics, such as a lens. In this way, a wide depth of field and extended eye-box retinal projection AR NED with a compact form factor is implemented. It is expected that this dissertation can help to develop a customized AR NED based on the customers needs. Furthermore, it is believed that this work can show new possibilities for research on the design and fabrication of HOEs.본 박사학위 논문에서는 근안 증강현실 디스플레이의 홀로그래픽 영상 결합 소자를 홀로그래픽 프린팅 기술을 이용하여 설계 및 제작하는 방법에 대하여 논한다. 이를 통하여 기존의 아날로그 방법에 의존한 홀로그래픽 광학 소자 제작 기법을 디지털화 할 수 있다. 또한 홀로그래픽 광학 소자의 설계 자유도가 증가하여 사용자 특징에 따른 근안 증강현실 디스플레이의 맞춤형 영상 결합 소자를 제작할 수 있다. 이 박사학위 논문에서는 홀로그래픽 광학 소자 프린터의 프로토타입을 제작 및 소개한다. 해당 프로토타입은 1 mm2의 면적 안에 1900 × 1900 복소 광파 정보를 표현 할 수 있다. 광파의 복소 변조를 위하여 진폭 변조 공간광변조를 이용한 sideband filtering 기법이 사용된다. 또한 굴절률이 보상된 프레임에 1/4 파장판 및 선형 편광자를 이용한 수동 광분리소자를 적용하여 홀로그래픽 광학 소자를 기록 할 때 발생하는 내부 반사 노이즈를 효과적으로 제거할 수 있다. 이와 같은 홀로그래픽 프린터의 프로토타입이 의도한 대로 제작되었음을 검증하기 위하여 홀로그래픽 광학 소자 렌즈를 제작 및, 해당 홀로그래픽 광학 소자 렌즈가 근안 홀로그래픽 증강현실 디스플레이의 영상 결합 소자로 사용될 수 있음을 보인다. 제작된 홀로그래픽 광학 소자 프린터를 이용하여 두 가지의 새로운 근안 증강현실 디스플레이를 제안한다. 첫 번째는 시청영역이 증가한 근안 홀로그래픽 증강현실 디스플레이로, 공간대역폭에 의하여 제한된 시청 영역을 수직 및 수평 방향으로 동시에 확장할 수 있다. 또한 확장된 시청 영역은 사용자의 안구 길이 및 회전 각도에 맞춰 설계되어 시청영역 불일치로 인한 비네팅 등의 이미지 왜곡을 최소화한다. 마지막으로 망막투사 형태의 근안 증강현실 디스플레이에 사용될 수 있는 프리폼 거울 어레이 홀로그래픽 광학 소자를 제안한다. 이를 이용하여, 기존 거울 어레이 기반의 망막투사 디스플레이의 문제점 중 하나인 거울이 시야를 가리는 문제를 해결한다. 또한 홀로그래픽 거울 배열에 위상 변조 패턴을 기록하여 추가적인 렌즈 등의 광학계 없이 원하는 깊이에 디스플레이 평면을 띄울 수 있게 된다. 이를 이용하여 작은 폼팩터의 넓은 깊이 표현 범위를 지니는 망막투사형 근안 증강현실 디스플레이를 구현한다. 본 박사학위 논문의 결과는 사용자의 필요에 기반한 맞춤형 근안 증강현실 디스플레이의 개발에 도움이 될 것으로 기대된다. 나아가, 본 연구는 홀로그래픽 광학 소자의 설계와 제작에 관한 연구의 새로운 가능성을 보여줄 것으로 기대된다.1 Introduction 1 1.1 Image combiners of augmented reality near-eye display 1 1.2 Motivation and purpose of this dissertation 8 1.3 Scope and organization 10 2 Holographic optical element printer 12 2.1 Introduction 12 2.2 Overview of the prototype of holographic optical element printer 16 2.3 Analysis of the signal path 21 2.4 Considerations in designing an HOE 27 2.5 Removal of the internal reflection noise using passive optical isolator 32 2.6 Manufacturing customized lens holographic optical element 37 2.7 Discussion 41 2.7.1 HOE printer to modulate both signal and reference beams 41 2.7.2 The term "hogel" used in this dissertation 41 2.8 Summary 44 3 Holographically customized optical combiner for eye-box extended near-eye display 45 3.1 Introduction 45 3.2 Proposed method and its implementation 51 3.3 Implemented prototype 57 3.4 Experiments and results 61 3.5 Discussion 63 3.5.1 Vignetting effect from mismatched pupil position along axial direction 63 3.5.2 Diffraction efficiency simulation according to incident angle 65 3.6 Summary 67 4 Holographically printed freeform mirror array for augmented reality near-eye display 68 4.1 Introduction 68 4.2 Retinal projection NED based on small aperture array 70 4.3 Proposed method 72 4.4 Design method of FMA HOE 75 4.4.1 Depth of field analysis 75 4.4.2 The size of the mirror 77 4.4.3 The distance between the mirrors 79 4.5 Experiments and results 82 4.6 Discussion 86 4.6.1 Eye-box of the system via the angular selectivity of the HOE 86 4.7 Summary 89 5 Conclusion 90 Appendix 104 Abstract (In Korean) 105Docto

Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis

Background!#!Hand amputation can have a truly debilitating impact on the life of the affected person. A multifunctional myoelectric prosthesis controlled using pattern classification can be used to restore some of the lost motor abilities. However, learning to control an advanced prosthesis can be a challenging task, but virtual and augmented reality (AR) provide means to create an engaging and motivating training.!##!Methods!#!In this study, we present a novel training framework that integrates virtual elements within a real scene (AR) while allowing the view from the first-person perspective. The framework was evaluated in 13 able-bodied subjects and a limb-deficient person divided into intervention (IG) and control (CG) groups. The IG received training by performing simulated clothespin task and both groups conducted a pre- and posttest with a real prosthesis. When training with the AR, the subjects received visual feedback on the generated grasping force. The main outcome measure was the number of pins that were successfully transferred within 20 min (task duration), while the number of dropped and broken pins were also registered. The participants were asked to score the difficulty of the real task (posttest), fun-factor and motivation, as well as the utility of the feedback.!##!Results!#!The performance (median/interquartile range) consistently increased during the training sessions (4/3 to 22/4). While the results were similar for the two groups in the pretest, the performance improved in the posttest only in IG. In addition, the subjects in IG transferred significantly more pins (28/10.5 versus 14.5/11), and dropped (1/2.5 versus 3.5/2) and broke (5/3.8 versus 14.5/9) significantly fewer pins in the posttest compared to CG. The participants in IG assigned (mean ± std) significantly lower scores to the difficulty compared to CG (5.2 ± 1.9 versus 7.1 ± 0.9), and they highly rated the fun factor (8.7 ± 1.3) and usefulness of feedback (8.5 ± 1.7).!##!Conclusion!#!The results demonstrated that the proposed AR system allows for the transfer of skills from the simulated to the real task while providing a positive user experience. The present study demonstrates the effectiveness and flexibility of the proposed AR framework. Importantly, the developed system is open source and available for download and further development

Massive MIMO is a Reality -- What is Next? Five Promising Research Directions for Antenna Arrays

Massive MIMO (multiple-input multiple-output) is no longer a "wild" or "promising" concept for future cellular networks - in 2018 it became a reality. Base stations (BSs) with 64 fully digital transceiver chains were commercially deployed in several countries, the key ingredients of Massive MIMO have made it into the 5G standard, the signal processing methods required to achieve unprecedented spectral efficiency have been developed, and the limitation due to pilot contamination has been resolved. Even the development of fully digital Massive MIMO arrays for mmWave frequencies - once viewed prohibitively complicated and costly - is well underway. In a few years, Massive MIMO with fully digital transceivers will be a mainstream feature at both sub-6 GHz and mmWave frequencies. In this paper, we explain how the first chapter of the Massive MIMO research saga has come to an end, while the story has just begun. The coming wide-scale deployment of BSs with massive antenna arrays opens the door to a brand new world where spatial processing capabilities are omnipresent. In addition to mobile broadband services, the antennas can be used for other communication applications, such as low-power machine-type or ultra-reliable communications, as well as non-communication applications such as radar, sensing and positioning. We outline five new Massive MIMO related research directions: Extremely large aperture arrays, Holographic Massive MIMO, Six-dimensional positioning, Large-scale MIMO radar, and Intelligent Massive MIMO.Comment: 20 pages, 9 figures, submitted to Digital Signal Processin

On Inter-referential Awareness in Collaborative Augmented Reality

For successful collaboration to occur, a workspace must support inter-referential awareness - or the ability for one participant to refer to a set of artifacts in the environment, and for that reference to be correctly interpreted by others. While referring to objects in our everyday environment is a straight-forward task, the non-tangible nature of digital artifacts presents us with new interaction challenges. Augmented reality (AR) is inextricably linked to the physical world, and it is natural to believe that the re-integration of physical artifacts into the workspace makes referencing tasks easier; however, we find that these environments combine the referencing challenges from several computing disciplines, which compound across scenarios. This dissertation presents our studies of this form of awareness in collaborative AR environments. It stems from our research in developing mixed reality environments for molecular modeling, where we explored spatial and multi-modal referencing techniques. To encapsulate the myriad of factors found in collaborative AR, we present a generic, theoretical framework and apply it to analyze this domain. Because referencing is a very human-centric activity, we present the results of an exploratory study which examines the behaviors of participants and how they generate references to physical and virtual content in co-located and remote scenarios; we found that participants refer to content using physical and virtual techniques, and that shared video is highly effective in disambiguating references in remote environments. By implementing user feedback from this study, a follow-up study explores how the environment can passively support referencing, where we discovered the role that virtual referencing plays during collaboration. A third study was conducted in order to better understand the effectiveness of giving and interpreting references using a virtual pointer; the results suggest the need for participants to be parallel with the arrow vector (strengthening the argument for shared viewpoints), as well as the importance of shadows in non-stereoscopic environments. Our contributions include a framework for analyzing the domain of inter-referential awareness, the development of novel referencing techniques, the presentation and analysis of our findings from multiple user studies, and a set of guidelines to help designers support this form of awareness

TRANSPARENT TUNABLE OPTICAL ELEMENTS WITH STRUCTURALLY-MODIFIED ELECTROACTIVE POLYMER

An optical element may include a primary electrode, a secondary electrode overlapping at least a portion of the primary electrode, and a structurally-modified and transparent electroactive polymer disposed between and abutting the primary electrode and the secondary electrode. An optical device may include a tunable lens and an optical element disposed over at least one surface of the tunable lens. Various other articles, devices, systems, and methods are also disclosed

SYSTEM INCLUDING CONOSCOPE LENS FOR MEASURING POLARIZATION CHARACTERISTICS OF WIDE FIELD-OF-VIEW LENSES

A lens measurement system including 1) a light source, 2) a lens mounting support configured to hold a lens so that light emitted by the light source is incident on the lens, 3) a conoscope lens positioned to receive light refracted by the lens, and 4) a polarization camera positioned to receive light emitted from the conoscope lens