38 research outputs found
Recommended from our members
A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus
Global or indirect illumination effects such as interreflections and subsurface scattering severely degrade the performance of structured light-based 3D scanning. In this paper, we analyze the errors in structured light, caused by both long-range (interreflections) and short-range (subsurface scattering) indirect illumination. The errors depend on the frequency of the projected patterns, and the nature of indirect illumination. In particular, we show that long-range effects cause decoding errors for low-frequency patterns, whereas short-range effects affect high-frequency patterns. Based on this analysis, we present a practical 3D scanning system which works in the presence of a broad range of indirect illumination. First, we design binary structured light patterns that are resilient to individual indirect illumination effects using simple logical operations and tools from combinatorial mathematics. Scenes exhibiting multiple phenomena are handled by combining results from a small ensemble of such patterns. This combination also allows detecting any residual errors that are corrected by acquiring a few additional images. Our methods can be readily incorporated into existing scanning systems without significant overhead in terms of capture time or hardware. We show results for several scenes with complex shape and material properties
Micro Phase Shifting
We consider the problem of shape recovery for real world scenes, where a variety of global illumination (inter-reflections, subsurface scattering, etc.) and illumination defocus effects are present. These effects introduce systematic and often significant errors in the recovered shape. We introduce a structured light technique called Micro Phase Shifting, which overcomes these problems. The key idea is to project sinusoidal patterns with frequencies limited to a narrow, high-frequency band. These patterns produce a set of images over which global illumination and defocus effects remain constant for each point in the scene. This enables high quality reconstructions of scenes which have traditionally been considered hard, using only a small number of images. We also derive theoretical lower bounds on the number of input images needed for phase shifting and show that Micro PS achieves the bound
Projector-Based Augmentation
Projector-based augmentation approaches hold the potential of combining the advantages of well-establishes spatial virtual reality and spatial augmented reality. Immersive, semi-immersive and augmented visualizations can be realized in everyday environments â without the need for special projection screens and dedicated display configurations. Limitations of mobile devices, such as low resolution and small field of view, focus constrains, and ergonomic issues can be overcome in many cases by the utilization of projection technology. Thus, applications that do not require mobility can benefit from efficient spatial augmentations. Examples range from edutainment in museums (such as storytelling projections onto natural stone walls in historical buildings) to architectural visualizations (such as augmentations of complex illumination simulations or modified surface materials in real building structures). This chapter describes projector-camera methods and multi-projector techniques that aim at correcting geometric aberrations, compensating local and global radiometric effects, and improving focus properties of images projected onto everyday surfaces
Towards Intelligent Telerobotics: Visualization and Control of Remote Robot
Human-machine cooperative or co-robotics has been recognized as the next generation of robotics. In contrast to current systems that use limited-reasoning strategies or address problems in narrow contexts, new co-robot systems will be characterized by their flexibility, resourcefulness, varied modeling or reasoning approaches, and use of real-world data in real time, demonstrating a level of intelligence and adaptability seen in humans and animals. The research I focused is in the two sub-field of co-robotics: teleoperation and telepresence. We firstly explore the ways of teleoperation using mixed reality techniques. I proposed a new type of display: hybrid-reality display (HRD) system, which utilizes commodity projection device to project captured video frame onto 3D replica of the actual target surface. It provides a direct alignment between the frame of reference for the human subject and that of the displayed image. The advantage of this approach lies in the fact that no wearing device needed for the users, providing minimal intrusiveness and accommodating users eyes during focusing. The field-of-view is also significantly increased. From a user-centered design standpoint, the HRD is motivated by teleoperation accidents, incidents, and user research in military reconnaissance etc. Teleoperation in these environments is compromised by the Keyhole Effect, which results from the limited field of view of reference. The technique contribution of the proposed HRD system is the multi-system calibration which mainly involves motion sensor, projector, cameras and robotic arm. Due to the purpose of the system, the accuracy of calibration should also be restricted within millimeter level. The followed up research of HRD is focused on high accuracy 3D reconstruction of the replica via commodity devices for better alignment of video frame. Conventional 3D scanner lacks either depth resolution or be very expensive. We proposed a structured light scanning based 3D sensing system with accuracy within 1 millimeter while robust to global illumination and surface reflection. Extensive user study prove the performance of our proposed algorithm. In order to compensate the unsynchronization between the local station and remote station due to latency introduced during data sensing and communication, 1-step-ahead predictive control algorithm is presented. The latency between human control and robot movement can be formulated as a linear equation group with a smooth coefficient ranging from 0 to 1. This predictive control algorithm can be further formulated by optimizing a cost function.
We then explore the aspect of telepresence. Many hardware designs have been developed to allow a camera to be placed optically directly behind the screen. The purpose of such setups is to enable two-way video teleconferencing that maintains eye-contact. However, the image from the see-through camera usually exhibits a number of imaging artifacts such as low signal to noise ratio, incorrect color balance, and lost of details. Thus we develop a novel image enhancement framework that utilizes an auxiliary color+depth camera that is mounted on the side of the screen. By fusing the information from both cameras, we are able to significantly improve the quality of the see-through image. Experimental results have demonstrated that our fusion method compares favorably against traditional image enhancement/warping methods that uses only a single image
Recommended from our members
Rendering Deformable Surface Reflectance Fields
Animation of photorealistic computer graphics models is an important goal for many applications. Image-based modeling has emerged as a promising approach to capture and visualize real-world objects. Animating image-based models, however, is still a largely unsolved problem. In this paper, we extend a popular image-based representation called surface reflectance field to animate and render deformable real-world objects under arbitrary illumination. Deforming the surface reflectance field is achieved by modifying the underlying impostor geometry. We augment the impostor by a local parameterization that allows the correct evaluation of acquired reflectance images, preserving the original light model on the deformed surface. We present a deferred shading scheme to handle the increased amount of data involved in shading the deformable surface reflectance field. We show animations of various objects that were acquired with 3D photography.Engineering and Applied Science
Reconstruction active par projection de lumieÌre non structureÌe
Cette theÌse porte sur la reconstruction active de modeÌles 3D aÌ lâaide dâune cameÌra et dâun projecteur. Les meÌthodes de reconstruction standards utilisent des motifs de lumieÌre codeÌe qui ont leurs forces et leurs faiblesses. Nous introduisons de nouveaux motifs baseÌs sur la lumieÌre non structureÌe afin de pallier aux manques des meÌthodes existantes. Les travaux preÌsenteÌs sâarticulent autour de trois axes : la robustesse, la preÌcision et finalement la comparaison des patrons de lumieÌre non structureÌe aux autres meÌthodes.
Les patrons de lumieÌre non structureÌe se diffeÌrencient en premier lieu par leur robustesse aux interreÌflexions et aux discontinuiteÌs de profondeur. Ils sont conçus de sorte aÌ homogeÌneÌiser la quantiteÌ dâillumination indirecte causeÌe par la projection sur des surfaces difficiles. En contrepartie, la mise en correspondance des images projeteÌes et captureÌes est plus complexe quâavec les meÌthodes dites structureÌes. Une meÌthode dâappariement probabiliste et efficace est proposeÌe afin de reÌsoudre ce probleÌme.
Un autre aspect important des reconstructions baseÌes sur la lumieÌre non structureÌe est la capaciteÌ de retrouver des correspondances sous-pixels, câest-aÌ-dire aÌ un niveau de preÌcision plus fin que le pixel. Nous preÌsentons une meÌthode de geÌneÌration de code de treÌs grande longueur aÌ partir des motifs de lumieÌre non structureÌe. Ces codes ont lâavantage double de permettre lâextraction de correspondances plus preÌcises tout en requeÌrant lâutilisation de moins dâimages. Cette contribution place notre meÌthode parmi les meilleures au niveau de la preÌcision tout en garantissant une treÌs bonne robustesse.
Finalement, la dernieÌre partie de cette theÌse sâinteÌresse aÌ la comparaison des meÌthodes existantes, en particulier sur la relation entre la quantiteÌ dâimages projeteÌes et la qualiteÌ de la reconstruction. Bien que certaines meÌthodes neÌcessitent un nombre constant dâimages, dâautres, comme la noÌtre, peuvent se contenter dâen utiliser moins aux deÌpens dâune qualiteÌ moindre. Nous proposons une meÌthode simple pour eÌtablir une correspondance optimale pouvant servir de reÌfeÌrence aÌ des fins de comparaison. Enfin, nous preÌsentons des meÌthodes hybrides qui donnent de treÌs bons reÌsultats avec peu dâimages.This thesis deals with active 3D reconstruction from camera-projector systems. Standard reconstruction methods use coded light patterns that come with their strengths and weaknesses. We introduce unstructured light patterns that feature several improvements compared to the current state of the art. The research presented revolves around three main axes : robustness, precision and comparison of existing unstructured light patterns to existing methods.
Unstructured light patterns stand out first and foremost by their robustness to interreflections and depth discontinuities. They are specifically designed to homogenize the indirect lighting generated by their projection on hard to scan surfaces. The downside of these patterns is that matching projected and captured images is not straightforward anymore. A probabilistic correspondence method is formulated to solve this problem efficiently.
Another important aspect of reconstruction obtained with unstructured light pat- terns is their ability to recover subpixel correspondences, that is with a precision finer than the pixel level. We present a method to produce long codes using unstructured light. These codes enable us to extract more precise correspondences while requiring less patterns. This contribution makes our method one of the most accurate - yet robust to standard challenges - method of active reconstruction in the domain.
Finally, the last part of this thesis adresses the comparison of existing reconstruction methods on several aspects, but mainly on the impact of using less and less patterns on the quality of the reconstruction. While some methods need a fixed number of images, some, like ours, can accommodate fewer patterns in exchange for some quality loss. We devise a simple method to capture an optimal correspondence map that can be used as a groundtruth for comparison purposes. Last, we present several hybrid methods that perform quite well even with few images
Projector-Based Augmentation
Projector-based augmentation approaches hold the potential of combining the advantages of well-establishes spatial virtual reality and spatial augmented reality. Immersive, semi-immersive and augmented visualizations can be realized in everyday environments â without the need for special projection screens and dedicated display configurations. Limitations of mobile devices, such as low resolution and small field of view, focus constrains, and ergonomic issues can be overcome in many cases by the utilization of projection technology. Thus, applications that do not require mobility can benefit from efficient spatial augmentations. Examples range from edutainment in museums (such as storytelling projections onto natural stone walls in historical buildings) to architectural visualizations (such as augmentations of complex illumination simulations or modified surface materials in real building structures). This chapter describes projector-camera methods and multi-projector techniques that aim at correcting geometric aberrations, compensating local and global radiometric effects, and improving focus properties of images projected onto everyday surfaces
Le cinĂ©ma omnistĂ©rĂ©o ou l'art d'avoir des yeux tout le tour de la tĂȘte
Cette thÚse s'intéresse à des aspects du tournage, de la projection et de la perception du cinéma stéréo panoramique, appelé aussi cinéma omnistéréo. Elle s'inscrit en grande partie dans le domaine de la vision par ordinateur, mais elle touche aussi aux domaines de l'infographie et de la perception visuelle humaine.
Le cinéma omnistéréo projette sur des écrans immersifs des vidéos qui fournissent de l'information sur la profondeur de la scÚne tout autour des spectateurs. Ce type de cinéma comporte des défis liés notamment au tournage de vidéos omnistéréo de scÚnes dynamiques, à la projection polarisée sur écrans trÚs réfléchissants rendant difficile l'estimation de leur forme par reconstruction active, aux distorsions introduites par l'omnistéréo pouvant fausser la perception des profondeurs de la scÚne.
Notre thĂšse a tentĂ© de relever ces dĂ©fis en apportant trois contributions majeures. PremiĂšrement, nous avons dĂ©veloppĂ© la toute premiĂšre mĂ©thode de crĂ©ation de vidĂ©os omnistĂ©rĂ©o par assemblage d'images pour des mouvements stochastiques et localisĂ©s. Nous avons mis au point une expĂ©rience psychophysique qui montre l'efficacitĂ© de la mĂ©thode pour des scĂšnes sans structure isolĂ©e, comme des courants d'eau. Nous proposons aussi une mĂ©thode de tournage qui ajoute Ă ces vidĂ©os des mouvements moins contraints, comme ceux d'acteurs. DeuxiĂšmement, nous avons introduit de nouveaux motifs lumineux qui permettent Ă une camĂ©ra et un projecteur de retrouver la forme d'objets susceptibles de produire des interrĂ©flexions. Ces motifs sont assez gĂ©nĂ©raux pour reconstruire non seulement les Ă©crans omnistĂ©rĂ©o, mais aussi des objets trĂšs complexes qui comportent des discontinuitĂ©s de profondeur du point de vue de la camĂ©ra. TroisiĂšmement, nous avons montrĂ© que les distorsions omnistĂ©rĂ©o sont nĂ©gligeables pour un spectateur placĂ© au centre d'un Ă©cran cylindrique, puisqu'elles se situent Ă la pĂ©riphĂ©rie du champ visuel oĂč l'acuitĂ© devient moins prĂ©cise.This thesis deals with aspects of shooting, projection and perception of stereo panoramic cinema, also called omnistereo cinema. It falls largely in the field of computer vision, but it also in the areas of computer graphics and human visual perception.
Omnistereo cinema uses immersive screens to project videos that provide depth information of a scene all around the spectators. Many challenges remain in omnistereo cinema, in particular shooting omnistereo videos for dynamic scenes, polarized projection on highly reflective screens making difficult the process to recover their shape by active reconstruction, and perception of depth distortions introduced by omnistereo images.
Our thesis addressed these challenges by making three major contributions. First, we developed the first mosaicing method of omnistereo videos for stochastic and localized motions. We developed a psychophysical experiment that shows the effectiveness of the method for scenes without isolated structure, such as water flows. We also propose a shooting method that adds to these videos foreground motions that are not as constrained, like a moving actor. Second, we introduced new light patterns that allow a camera and a projector to recover the shape of objects likely to produce interreflections. These patterns are general enough to not only recover the shape of omnistereo screens, but also very complex objects that have depth discontinuities from the viewpoint of the camera. Third, we showed that omnistereo distortions are negligible for a viewer located at the center of a cylindrical screen, as they are in the periphery of the visual field where the human visual system becomes less accurate