Content creation for seamless augmented experiences with projection mapping

This dissertation explores systems and methods for creating projection mapping content that seamlessly merges virtual and physical. Most virtual reality and augmented reality technologies rely on screens for display and interaction, where a mobile device or head mounted display mediates the user's experience. In contrast, projection mapping uses off-the-shelf video projectors to augment the appearance of physical objects, and with projection mapping there is no screen to mediate the experience. The physical world simply becomes the display. Projection mapping can provide users with a seamless augmented experience, where virtual and physical become indistinguishable in an apparently unmediated way. Projection mapping is an old concept dating to Disney's 1969 Haunted Mansion. The core technical foundations were laid back in 1999 with UNC's Office of the Future and Shader Lamps projects. Since then, projectors have gotten brighter, higher resolution, and drastically decreased in price. Yet projection mapping has not crossed the chasm into mainstream use. The largest remaining challenge for projection mapping is that content creation is very difficult and time consuming. Content for projection mapping is still created via a tedious manual process by warping a 2D video file onto a 3D physical object using existing tools (e.g. Adobe Photoshop) which are not made for defining animated interactive effects on 3D object surfaces. With existing tools, content must be created for each specific display object, and cannot be re-used across experiences. For each object the artist wants to animate, the artist must manually create a custom texture for that specific object, and warp the texture to the physical object. This limits projection mapped experiences to controlled environments and static scenes. If the artist wants to project onto a different object from the original, they must start from scratch creating custom content for that object. This manual content creation process is time consuming, expensive and doesn't scale. This thesis explores new methods for creating projection mapping content. Our goal is to make projection mapping easier, cheaper and more scalable. We explore methods for adaptive projection mapping, which enables artists to create content once, and that content adapts based on the color and geometry of the display surface. Content can be created once, and re-used on any surface. This thesis is composed of three proof-of-concept prototypes, exploring new methods for content creation for projection mapping. IllumiRoom expands video game content beyond the television screen and into the physical world using a standard video projector to surround a television with projected light. IllumiRoom works in any living room, the projected content dynamically adapts based on the color and geometry of the room. RoomAlive expands on this idea, using multiple projectors to cover an entire living room in input/output pixels and dynamically adapts gaming experiences to fill an entire room. Finally, Projectibles focuses on the physical aspect of projection mapping. Projectibles optimizes the display surface color to increase the contrast and resolution of the overall experience, enabling artists to design the physical object along with the virtual content. The proof-of-concept prototypes presented in this thesis are aimed at the not-to-distant future. The projects in this thesis are not theoretical concepts, but fully working prototype systems that demonstrate the practicality of projection mapping to create immersive experiences. It is the sincere hope of the author that these experiences quickly move of the lab and into the real world

Projector Compensation for Unconventional Projection Surface

Projecting onto irregular textured surfaces found on buildings, automobiles and theatre stages calls for the development of radiometric and geometric compensation algorithms that require no user intervention and compensate for the patterning and colourization of the background surface. This process needs a projector-camera setup where the feedback from the camera is used to learn the background's geometric and radiometric properties. In this thesis, radiometric compensation, which is used to correct for the background texture distortion, is discussed in detail. Existing compensation frameworks assume no inter--pixel coupling and develop an independent compensation model for each projector pixel. This assumption is valid on background with uniform texture variation but fails at sharp contrast differences leading to visible edge artifacts in the compensated image. To overcome the edge artifacts, a novel radiometric compensation approach is presented that directly learns the compensation model, rather than inverting a learned forward model. That is, the proposed method uses spatially uniform camera images to learn the projector images that successfully hide the background. The proposed approach can be used with any existing radiometric compensation algorithm to improve its performance. Comparisons with classical and state-of-the-art methods show the superiority of the proposed method in terms of the perceived image quality and computational complexity. The modified target image from the radiometric compensation algorithm can exceed the limited dynamic range of the projector resulting in saturation artifacts in the compensated image. Since the achievable range of luminance on the background surface with the given projector is limited, the projector compensation should also consider the contents of the target image along with the background properties while calculating the projector image. A novel spatially optimized luminance modification approach is proposed using human visual system properties to reduce the saturation artifacts. Here, the tolerance of the human visual system is exploited to make perceptually less sensitive modifications to the target image that in turn reduces the luminance demands from the projector. The proposed spatial modification approach can be combined with any radiometric compensation models to improve its performance. The simulated results of the proposed luminance modification are evaluated to show the improvement in perceptual performance. The inverse approach combined with the spatial luminance modification concludes the proposed projector compensation, which enables the optimum compensated projection on an arbitrary background surface

Abstracts of 51st EASD Annual Meeting

Background and aims: Presence and frequency of beta cell (BC) dysfunction(BCD) and insulin resistance (IR) in patients with newly diagnosedtype 2 diabetes mellitus (NDT2D) are imperfectly known, becauseprevious studies used small cohorts and/or only surrogate indexes of BCfunction and IR.We sought to assess BC function and IR with state-of-artmethods in the VNDS.Materials and methods: In 712 GADA-negative, drug naïve, consecutiveItalian NDT2D patients we assessed: 1. standard parameters; 2. insulinsensitivity (IS) by the euglycaemic insulin clamp); 3. BC functionby state-of-art modeling of prolonged (5 hours) OGTT-derived glucose/C-peptide curves. Thresholds for BCD and IR were the 25th percentilesof BC function and IS assessed with the same methods of the VNDS inItalian subjects with normal glucose regulation of the GENFIEV (n=340)and GISIR (n=386) studies, respectively.Results: In the VNDS, 89.8% [95% C.I.: 87.6 - 92.0%] and87.8% [85.4 - 90.2] patients had BCD and IR, respectively. Patientswith only one defect were 19.7% [16.8 - 22.6]. IsolatedBCD and isolated IR were present in 10.9% [8.6 - 13.2] and8.9% [6.8 - 11.0] patients, respectively. Coexistence of BCDand IR was observed in 78.9% [75.9 - 81.9] of the patients.1.4% [0.5 - 2.3] of the patients had no detectable alterations inBC function and IS. Patients (19.7%) with only one metabolicdefect had lower BMI, fasting glucose, HbA1c, triglycerides andBC function, and higher HDL-cholesterol and IS than patientswith both BCD and IR (p<0.01 or less after Bonferroni’scorrection).Conclusion: In conclusion, in NDT2DM patients: 1. at least 75.9% haveboth BCD and IR; 2. At least 87.6% and 85.4% have BCD and IR,respectively; 3. At least 16.8% have only one defect and a significantlydifferent (milder) metabolic phenotype compared to patients with bothdefects. These findings may be relevant to therapeutic strategies centeredon the metabolic phenotype of the patient.Clinical Trial Registration Number: NCT00879801; NCT01526720Supported by: University of Veron