Photo repair and 3d structure from flatbed scanners

We introduce a technique that allows 3D information to be captured from a conventional flatbed scanner. The technique requires no hardware modification and allows untrained users to easily capture 3D datasets. Once captured, these datasets can be used for interactive relighting and enhancement of surface detail on physical objects. We have also found that the method can be used to scan and repair damaged photographs. Since the only 3D structure on these photographs will typically be surface tears and creases, our method provides an accurate procedure for automatically detecting these flaws without any user intervention. Once detected, automatic techniques, such as infilling and texture synthesis, can be leveraged to seamlessly repair such damaged areas. We first present a method that is able to repair damaged photographs with minimal user interaction and then show how we can achieve similar results using a fully automatic process

Image gradient based 3D roughness estimation and rendering for haptic palpation from a single skin image

Background/purpose: Skin palpation and property analysis (roughness, dryness, stiffness, temperature) are crucial for skin examination and diagnose. That is why is needed a noncontact-based method that allows to carry it out avoiding secondary infections or damage. A haptic device with haptic feedback was designed some years ago, but accurate results for 3D skin surface reconstruction and roughness estimation are still in research and improvement. In this study is proposed a gradient-based skin surface 3D roughness estimation algorithm that will enable haptic palpation and roughness examination. Methods: 3D roughness is estimated from 2D single image. First step is pre-processing the image, to improve the quality and reduce the noise by using contrast stretching and bilateral filtering. After, the gradient field is computed and used to obtain the 3D surface reconstruction using a surface-from-gradient algorithm, which will allow 3D roughness computation for a later dynamic haptic rendering. Results: Texture and curvature of the 3D reconstructed surface are checked in the first experiment, comparing roughness and geometry errors between a reconstructed surface using the proposed algorithm and two other algorithms, as well with a ground truth surface. The second experiment tests the method using in-vivo real skin disease images to compute roughness estimation and decomposition and also tasting haptic rendering in a haptic device. The experimental results verify the validity of our method. Conclusion: Roughness is a crucial property for dermatologists to examine skin disease (e.g., cases of psoriasis, atopic eczema or aging), that is why the proposed method of roughness estimation for haptic rendering will be extremely useful for dermatologists, improving the skin diagnose. In addition, the proposed method does not require complex medical systems to be implemented, since single image reconstruction is used.Outgoin

Enforcing Integrability for Surface Reconstruction Algorithms Using Belief Propagation in Graphical Models

The reviewers are encouraged to view the supplementary material (video files) Accurate calculation of the three dimensional shape of an object is one of the classic research areas of computer vision. Many of the existing methods are based on surface normal estimation, and subsequent integration of surface gradients. In general, these methods do not produce valid surface due to violation of surface integrability. We introduce a new method for shape reconstruction by integration of valid surface gradient maps. The essence of the new approach is in the strict enforcement of the surface integrability via belief propagation across graphical model. The graphical model is selected in such a way to extract information from underlying, possibly noisy, surface gradient estimators, utilize the surface integrability constraint, and produce the maximum a-posteriori estimate of a valid surface. We demonstrate the algorithm for two classic shape reconstruction techniques; shape-from-shading and photometric stereo. On a set of real and synthetic examples the new approach is shown to be fast and accurate, in the sense that shape can be rendered even in the presence of high levels of noise and sharp occlusion boundaries.

Computer-assisted animation creation techniques for hair animation and shade, highlight, and shadow

制度:新 ; 報告番号:甲3062号 ; 学位の種類:博士(工学) ; 授与年月日:2010/2/25 ; 早大学位記番号:新532

Modified belief propagation for reconstruction of office environments

Belief Propagation (BP) is an algorithm that has found broad application in many areas of computer science. The range of these areas includes Error Correcting Codes, Kalman filters, particle filters, and -- most relevantly -- stereo computer vision. Many of the currently best algorithms for stereo vision benchmarks, e.g. the Middlebury dataset, use Belief Propagation. This dissertation describes improvements to the core algorithm to improve its applicability and usefulness for computer vision applications. A Belief Propagation solution to a computer vision problem is commonly based on specification of a Markov Random Field that it optimizes. Both Markov Random Fields and Belief Propagation have at their core some definition of nodes and neighborhoods' for each node. Each node has a subset of the other nodes defined to be its neighborhood. In common usages for stereo computer vision, the neighborhoods are defined as a pixel's immediate four spatial neighbors. For any given node, this neighborhood definition may or may not be correct for the specific scene. In a setting with video cameras, I expand the neighborhood definition to include corresponding nodes in temporal neighborhoods in addition to spatial neighborhoods. This amplifies the problem of erroneous neighborhood assignments. Part of this dissertation addresses the erroneous neighborhood assignment problem. Often, no single algorithm is always the best. The Markov Random Field formulation appears amiable to integration of other algorithms: I explore that potential here by integrating priors from independent algorithms. This dissertation makes core improvements to BP such that it is more robust to erroneous neighborhood assignments, is more robust in regions with inputs that are near-uniform, and can be biased in a sensitive manner towards higher level priors. These core improvements are demonstrated by the presented results: application to office environments, real-world datasets, and benchmark datasets