Augmented reality X-ray vision on optical see-through head mounted displays

Abstract. In this thesis, we present the development and evaluation of an augmented reality X-ray system on optical see-through head-mounted displays. Augmented reality X-ray vision allows users to see through solid surfaces such as walls and facades, by augmenting the real view with virtual images representing the hidden objects. Our system is developed based on the optical see-through mixed reality headset Microsoft Hololens. We have developed an X-ray cutout algorithm that uses the geometric data of the environment and enables seeing through surfaces. We have developed four different visualizations as well based on the algorithm. The first visualization renders simply the X-ray cutout without displaying any information about the occluding surface. The other three visualizations display features extracted from the occluder surface to help the user to get better depth perception of the virtual objects. We have used Sobel edge detection to extract the information. The three visualizations differ in the way to render the extracted features. A subjective experiment is conducted to test and evaluate the visualizations and to compare them with each other. The experiment consists of two parts; depth estimation task and a questionnaire. Both the experiment and its results are presented in the thesis

Waveguide-Type Head-Mounted Display System for AR Application

Currently, a lot of institutes and industries are working on the development of the virtual reality and augmented reality techniques, and these techniques have been recognized as the determination for the direction of the three-dimensional display development in the near future. In this chapter, we mainly discussed the design and application of several wearable head-mounted display (HMD) systems with the waveguide structure using the in- and out-couplers which are fabricated by the diffractive optical elements or holographic volume gratings. Although the structure is simple, the waveguide-type HMDs are very efficient, especially in the practical applications, especially in the augmented reality applications, which make the device light-weighted. In addition, we reviewed the existing major head-mounted display and augmented reality systems

Tailored displays to compensate for visual aberrations

We introduce tailored displays that enhance visual acuity by decomposing virtual objects and placing the resulting anisotropic pieces into the subject's focal range. The goal is to free the viewer from needing wearable optical corrections when looking at displays. Our tailoring process uses aberration and scattering maps to account for refractive errors and cataracts. It splits an object's light field into multiple instances that are each in-focus for a given eye sub-aperture. Their integration onto the retina leads to a quality improvement of perceived images when observing the display with naked eyes. The use of multiple depths to render each point of focus on the retina creates multi-focus, multi-depth displays. User evaluations and validation with modified camera optics are performed. We propose tailored displays for daily tasks where using eyeglasses are unfeasible or inconvenient (e.g., on head-mounted displays, e-readers, as well as for games); when a multi-focus function is required but undoable (e.g., driving for farsighted individuals, checking a portable device while doing physical activities); or for correcting the visual distortions produced by high-order aberrations that eyeglasses are not able to.Conselho Nacional de Pesquisas (Brazil) (CNPq-Brazil fellowship 142563/2008-0)Conselho Nacional de Pesquisas (Brazil) (CNPq-Brazil fellowship 308936/2010-8)Conselho Nacional de Pesquisas (Brazil) (CNPq-Brazil fellowship 480485/2010- 0)National Science Foundation (U.S.) (NSF CNS 0913875)Alfred P. Sloan Foundation (fellowship)United States. Defense Advanced Research Projects Agency (DARPA Young Faculty Award)Massachusetts Institute of Technology. Media Laboratory (Consortium Members

Development of Immersive and Interactive Virtual Reality Environment for Two-Player Table Tennis

Although the history of Virtual Reality (VR) is only about half a century old, all kinds of technologies in the VR field are developing rapidly. VR is a computer generated simulation that replaces or augments the real world by various media. In a VR environment, participants have a perception of “presence”, which can be described by the sense of immersion and intuitive interaction. One of the major VR applications is in the field of sports, in which a life-like sports environment is simulated, and the body actions of players can be tracked and represented by using VR tracking and visualisation technology. In the entertainment field, exergaming that merges video game with physical exercise activities by employing tracking or even 3D display technology can be considered as a small scale VR. For the research presented in this thesis, a novel realistic real-time table tennis game combining immersive, interactive and competitive features is developed. The implemented system integrates the InterSense tracking system, SwissRanger 3D camera and a three-wall rear projection stereoscopic screen. The Intersense tracking system is based on ultrasonic and inertia sensing techniques which provide fast and accurate 6-DOF (i.e. six degrees of freedom) tracking information of four trackers. Two trackers are placed on the two players’ heads to provide the players’ viewing positions. The other two trackers are held by players as the racquets. The SwissRanger 3D camera is mounted on top of the screen to capture the player’

Optical techniques applied to measurements in art

Optical diagnostic techniques are particularly attractive for the non-destructive detection of incipient damage and the evaluation of the state of surface decay. Non-contact, high precision measurements of the shape and deformation of an artifact can be performed using laser methods based on holographic and speckle interferometry. [Continues.

Development of a multi-wavelength lensless digital holography system for 3D deformations and shape measurements of tympanic membranes

Current methodologies for characterization of tympanic membranes (TMs) have some limitations. They: are qualitative rather than quantitative, consist of single point mobility measurements, or only include one-dimensional deformation measurements. Furthermore, none of the current clinical tools for diagnosis of hearing losses have the capability to measure the shape of TM, which is very useful for anatomical or pathological investigations. The multi-wavelength lensless digital holography system (MLDHS) reported in this work consists of laser delivery (LD), optical head (OH), and computing platform (CP) subsystems, with capabilities of real-time, non-contact, full-field of view measurements. One version of the LD houses two tunable near-infrared external-cavity diode lasers with central wavelengths of 780.24nm and 779.74nm respectively, an acousto-optic modulator, and a laser-to-fiber mechanism. The output of the LD is delivered to an ultra-fast MEMS-based fiber optic switch and the light beam is directed to the OH, which is arranged to perform imaging and measurements by phase-shifting holography. The second LD version subsystem contains one tunable near-infrared diode laser in the range from 770nm to 789nm, an anamorphic prism pair, an acousto-optic modulator, a half-wave plate, and a fiber coupler assembly. The output of the LD is delivered to the OH directly. The OH is designed by 3D optical ray tracing simulations in which components are rotated at specific angles to overcome reflection issues. A high-resolution digital camera with pixel size of 6.7μm by 6.7μm in the OH is used for image recording at high-rates while the CP acquires and processes images in either time-averaged or double-exposure modes. The choice of working version depends on the requirements of the measurement and the sample under test. MLDHS can obtain shape and one-dimensional deformations along one optical axis (z-axis). In order to recover 3D deformations, assumptions based on elasticity theory are prerequisites for the calculations: (a) the TM is analyzed as a thin shell; (b) shape before and after deformation is considered nearly the same since acoustic pressure typically introduces nanometer scale deformations; and (c) normal vectors remain perpendicular to the deformed mid-plane of the TM. Another part of this Thesis is the design and prototyping of the MLDHS, which translates this holographic platform into a simple and compact holographic instrument for measurements of the visible tympanic-membrane motions in live patients. Therefore, the OH subsystem needs to be light and portable, as it can be mounted on a robotic arm be near the ear canal, while the LD subsystem needs to be stable and safely protected. Preliminary results of acoustically induced 3D deformations and shape measurements by a single instrument that demonstrate the capabilities of the devices developed in this Thesis are presented

Holographic reality: enhancing the artificial reality experience throuhg interactive 3D holography

Holography was made know by several science-fiction productions, however this technology dates back to the year 1940. Despite the considerable age of this discovery, this technology remains inaccessible to the average consumer. The main goal of this manuscript is to advance the state of the art in interactive holography, providing an accessible and low-cost solution. The final product intends to nudge the HCI com munity to explore potential applications, in particular to be aquatic centric and environmentally friendly. Two main user studies are performed, in order to determine the impact of the proposed solution by a sample audience. Provided user studies include a first prototype as a Tangible User Interface - TUI for Holographic Reality - HR Second study included the Holographic Mounted Display - HMD for proposed HR interface, further analyzing the interactive holographic experience without hand-held devices. Both of these studies were further compared with an Augmented Reality setting. Obtained results demonstrate a significantly higher score for the HMD approach. This suggests it is the better solution, most likely due to the added simplicity and immersiveness features it has. However the TUI study did score higher in several key parameters, and should be considered for future studies. Comparing with an AR experience, the HMD study scores slightly lower, but manages to surpass AR in several parameters. Several approaches were outlined and evaluated, depicting different methods for the creation of Interactive Holographic Reality experiences. In spite of the low maturity of holographic technology, it can be concluded it is comparable and can keep up to other more developed and mature artificial reality settings, further supporting the need for the existence of the Holographic Reality conceptA tecnologia holográfica tornou-se conhecida através da ficção científica, contudo esta tecnologia remonta até ao ano 1940. Apesar da considerável idade desta descoberta, esta tecnologia continua a não ser acessíveil para o consumidor. O objetivo deste manuscrito é avançar o estado de arte da Holografia Interactiva, e fornecer uma solução de baixo custo. O objetivo do produto final é persuadir a comunidade HCI para a exploração de aplicações desta tecnologia, em particular em contextos aquáticos e pró-ambientais. Dois estudos principais foram efetuados, de modo a determinar qual o impacto da solução pro posta numa amostra. Os estudos fornecidos incluem um protótipo inicial baseado numa Interface Tangível e Realidade Holográfica e um dispositivo tangível. O segundo estudo inclui uma interface baseada num dispositivo head-mounted e em Realidade Holográfica, de modo a analisar e avaliar a experiência interativa e holográfica. Ambos os estudos são comparados com uma experiência semelhante, em Realidade Aumentada. Os resultados obtidos demonstram que o estudo HMD recebeu uma avaliação significante mel hor, em comparação com a abordagem TUI. Isto sugere que uma abordagem "head-mounted" tende a ser melhor solução, muito provavelmente devido às vantagens que possui em relação à simplicidade e imersividade que oferece. Contudo, o estudo TUI recebeu pontuações mais altas em alguns parâmetros chave, e deve ser considerados para a implementação de futuros estudos. Comparando com uma experiência de realidade aumentada, o estudo HMD recebeu uma avaliação ligeiramente menor, mas por uma margem mínima, e ultrapassando a AR em alguns parâmetros. Várias abordagens foram deliniadas e avaliadas, com diferentes métodos para a criação de experiências de Realidade Holográfica. Apesar da pouca maturidade da tecnologia holográfica, podemos concluir que a mesma é comparável e consegue acompanhar outros tipos de realidade artificial, que são muito mais desenvolvidos, o que suporta a necessidade da existência do conceito de Realidade Holográfica