Skip to main content
Article thumbnail
Location of Repository

3-D Reconstruction of Multi-object Scenes from Multiple Images

By Matthew Grum


Photorealistic 3-D models are used in a wide variety of applications from entertainment and games, through to simulation, training . Algorithms to automatically create such models from ordinary photographs can vastly reduce the workload and expense associated with acquiring such models. The vast majority of research into reconstructing 3-D models from images has concentrated\ud on the case of single objects.\ud \ud This thesis presents a method to model complex multi-object scenes in a series of steps starting with a set of images which surround a scene and ?nally producing a complete photorealistic representation of the objects. The probabilistic space carving algorithm is used to provide an initial estimate of shape as it makes no assumptions about the shape of the scene aside from the bounding cuboid. This representation is smoothed by fitting a Radial Basis Function implicit surface, which smoothes noise and interpolates any missing data. Errors which persist are addressed by a matching surface points between images and estimating the perspective transformation between them which is used to calculate the correct position for the point, which is consistent with the input images. The model may be corrected by constraining the surface to pass through these points. The smoothing properties of RBFs can cause problems by interpolating across objects which are close together, causing them to be joined in the representation. A method is presented to correct this by enforcing consistency between edges in 2-D and 3-D.\ud \ud Experiments are conducted using real image sequences of complex multi-object scenes. Both qualitative and quantitative evaluations are performed demonstrating the effectiveness of the methods presented. In addition to modelling all of the objects present, colour surfaces are produced from which even the text is legible. A detailed study is undertaken into the factors which influence the effectiveness of techniques to recover partially or fully fused objects and conclusions are drawn which hint at the ultimate limit of accuracy in the case of multiple objects

Publisher: Computer Science (York)
Year: 2009
OAI identifier:

Suggested articles


  1. (1988). A combined corner and edge detector. doi
  2. (2000). A compact algorithm for rectification of stereo pairs. doi
  3. A comparison of methods for generating Poisson disk distributions. status: accepted. doi
  4. (1996). A factorization method for shape and motion from line correspondences. doi
  5. (1982). A generalization of algebraic surface drawing. doi
  6. (1997). A geometric snake model for segmentation of medical imagery. doi
  7. (1994). A paraperspective factorization method for shape and motion recovery. doi
  8. (2001). A probabilistic framework for space carving. doi
  9. (2009). A semi-supervised approach to space carving. Pattern Recognition, doi
  10. (2003). A sparse texture representation using affine-invariant regions. doi
  11. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. doi
  12. (2000). A theory of shape by space carving. doi
  13. (2004). An automated method for large-scale, ground-based city model acquisition. doi
  14. (1991). An efficient method of triangulating equi-valued surfaces by using tetrahedral cells.
  15. (1999). Bayesian structure from motion. doi
  16. (2003). Ben Appleton, and Changming Sun. Embedded voxel colouring. doi
  17. (2006). Carved visual hulls for image-based modeling. doi
  18. (2002). Carving 3d models from uncalibrated views. doi
  19. (2005). Color appearance models. doi
  20. (1993). Comparing images using the Hausdorff distance. doi
  21. (1998). Computation of the quadrifocal tensor. doi
  22. (1974). Computer matching of areas in stereo images.
  23. (2003). Contour generator points for threshold selection and a novel photo-consistency measure for space carving.
  24. Deformable mesh model for complex multi-object 3d motion estimation from multi-viewpoint video. doi
  25. (1996). Deformable models in medical image analysis: a survey. Medical image analysis, doi
  26. (1992). Detecting and tracking multiple moving objects using temporal integration. doi
  27. (2002). Detecting multiple texture planes using local spectral distortion. doi
  28. (2001). Fast approximate energy minimization via graph cuts. doi
  29. (1999). Fast training of support vector machines using sequential minimal optimization. doi
  30. (1988). Fronts propagating with curvature dependent speed: Algorithms based on Hamilton Jacobi formulations. doi
  31. (2008). Fusion of feature-and area-based information for urban buildings modeling from aerial imagery. doi
  32. (2000). Generalized voxel coloring. doi
  33. (2000). Geometric camera calibration using circular control points. doi
  34. (1997). Gradient vector flow: A new external force for snakes. doi
  35. (1995). How far 3D shapes can be understood from 2D silhouettes. doi
  36. (2008). Integration of Multiview Stereo and Silhouettes Via Convex Functionals on Convex Domains. doi
  37. (1998). Interactive 3d modeling from multiple images using scene regularities. doi
  38. (2001). Interpolating implicit surfaces from scattered surface data using compactly supported radial basis functions. doi
  39. (1990). Introduction to algorithms, doi
  40. (2004). Jo˜ ao G.M.Gonc ¸alves, and V´ ıtor Sequeira. Acquisition, modelling and rendering of very large urban environments. doi
  41. (1997). Lines and points in three views and the trifocal tensor.
  42. (2002). Mean-shift segmentation with wavelet-based bandwidth selection. doi
  43. (1962). Methods and means for recognising complex patterns.
  44. (1998). Metric 3d surface reconstruction from uncalibrated image sequences. doi
  45. (2002). Modelling with implicit surfaces that interpolate. doi
  46. Multi-View Reconstruction using Narrow-Band Graph-Cuts and Surface Normal doi
  47. (2005). Multi-view stereo via volumetric graph-cuts. doi
  48. (1996). Multiple order laplacian splines - including splines with tension.
  49. Multiple View Geometry in Computer Vision. doi
  50. New constraints on data-closeness and needle map consistency for shape-from-shading. doi
  51. (1994). Non-parametric local transforms for computing visual correspondence. doi
  52. (2001). Object recognition using shape-from-shading. doi
  53. (1999). one-Alpes. A hybrid approach for computing visual hulls of complex objects. doi
  54. Photorealistic scene reconstruction by voxel coloring. doi
  55. (1995). Piecewise polynomial, positive definite and compactly supported radial functions of minimal degree. doi
  56. (2004). Pose estimation for multi-camera systems. doi
  57. (1993). Priors stabilizers and basis functions: From regularization to radial, tensor and additive splines. doi
  58. (1987). Probabilistic solution of ill-posed problems in computational vision. doi
  59. (2000). Real-time tracking of multiple moving object contours in a moving camera image sequence. doi
  60. (2002). Reconstructing surfaces by volumetric regularization using radial basis functions. doi
  61. (2001). Reconstruction and representation of 3D objects with radial basis functions. doi
  62. (2008). Reduced depth and visual hulls of complex 3D scenes. doi
  63. (1999). Reliable feature matching across widely separated views. doi
  64. (2006). Robust structure and motion from outlines of smooth curved surfaces. doi
  65. (2001). Shadow carving. doi
  66. (1992). Shape and motion from image streams: a factorization method: Full report on the orthographic case. doi
  67. (1990). Shape and motion without depth. doi
  68. (2004). Shape and view independent reflectance map from multiple views. InTom´ doi
  69. (1989). Shape from contour using symmetries. doi
  70. (2004). Shape reconstruction from 3D and 2D data using PDE-based deformable surfaces. doi
  71. (2003). Smooth surface reconstruction from noisy range data. doi
  72. (2007). Spatial texture models for video compression. doi
  73. (1990). Spline Models for Observational Data. doi
  74. (1998). Stereo matching with nonlinear diffusion. doi
  75. (2003). Stochastic refinement of the visual hull to satisfy photometric and silhouette consistency constraints. doi
  76. (1989). Stochastic stereo matching over scale. doi
  77. (1991). Structure and motion in two dimensions from multiple images: A least squares approach. doi
  78. (1999). Structure from motion without correspondence. doi
  79. (2006). Survey on block matching motion estimation algorithms and architectures with new results. doi
  80. (1998). The crust and the beta-skeleton: Combinatorial curve reconstruction. Graphical Models and Image Processing, doi
  81. (1975). The estimation of the gradient of a density function, with applications in pattern recognition. doi
  82. (1996). The fundamental matrix: Theory, algorithms, and stability analysis. doi
  83. (1993). The JISCT Stereo Evaluation’. In Image Understanding Workshop:
  84. (1999). The mpeg4 multimedia coding standard: Algorithms, architectures and applications. doi
  85. (2001). Three-dimensional modeling from two-dimensional video. doi
  86. (1998). Variational principles, surface evolution, PDEs, level set methods, and the stereo problem. doi
  87. (1990). Vector median filters. doi
  88. (2001). Viewpoint invariant texture matching and wide baseline stereo. doi
  89. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. doi
  90. (2000). Volumetric warping for voxel coloring on an infinite domain. doi
  91. (1984). What does the occluding contour tell us about solid shape. doi

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.