Robots for Exploration, Digital Preservation and Visualization of Archeological Sites

Monitoring and conservation of archaeological sites are important activities necessary to prevent damage or to perform restoration on cultural heritage. Standard techniques, like mapping and digitizing, are typically used to document the status of such sites. While these task are normally accomplished manually by humans, this is not possible when dealing with hard-to-access areas. For example, due to the possibility of structural collapses, underground tunnels like catacombs are considered highly unstable environments. Moreover, they are full of radioactive gas radon that limits the presence of people only for few minutes. The progress recently made in the artificial intelligence and robotics field opened new possibilities for mobile robots to be used in locations where humans are not allowed to enter. The ROVINA project aims at developing autonomous mobile robots to make faster, cheaper and safer the monitoring of archaeological sites. ROVINA will be evaluated on the catacombs of Priscilla (in Rome) and S. Gennaro (in Naples)

Coloring the Past: Neural Historical Buildings Reconstruction from Archival Photography

Historical buildings are a treasure and milestone of human cultural heritage. Reconstructing the 3D models of these building hold significant value. The rapid development of neural rendering methods makes it possible to recover the 3D shape only based on archival photographs. However, this task presents considerable challenges due to the limitations of such datasets. Historical photographs are often limited in number and the scenes in these photos might have altered over time. The radiometric quality of these images is also often sub-optimal. To address these challenges, we introduce an approach to reconstruct the geometry of historical buildings, employing volumetric rendering techniques. We leverage dense point clouds as a geometric prior and introduce a color appearance embedding loss to recover the color of the building given limited available color images. We aim for our work to spark increased interest and focus on preserving historical buildings. Thus, we also introduce a new historical dataset of the Hungarian National Theater, providing a new benchmark for the reconstruction method

Development and evaluation of a digital tool for virtual reconstruction of historic Islamic geometric patterns

For the purpose of cultural heritage preservation, the task of recording and reconstructing visually complicated architectural geometrical patterns is facing many practical challenges. Existing traditional technologies rely heavily on the subjective nature of our perceptual power in understanding its complexity and depicting its color differences. This study explores one possible solution, through utilizing digital techniques for reconstructing detailed historical Islamic geometric patterns. Its main hypothesis is that digital techniques offer many advantages over the human eye in terms of recognizing subtle differences in light and color. The objective of the study is to design, test and evaluate an automatic visual tool for identifying deteriorated or incomplete archaeological Islamic geometrical patterns captured in digital images, and then restoring them digitally, for the purpose of producing accurate 2D reconstructed metric models. An experimental approach is used to develop, test and evaluate the specialized software. The goal of the experiment is to analyze the output reconstructed patterns for the purpose of evaluating the digital tool in respect to reliability and structural accuracy, from the point of view of the researcher in the context of historic preservation. The research encapsulates two approaches within its methodology; Qualitative approach is evident in the process of program design, algorithm selection, and evaluation. Quantitative approach is manifested through using mathematical knowledge of pattern generation to interpret available data and to simulate the rest based on it. The reconstruction process involves induction, deduction and analogy. The proposed method was proven to be successful in capturing the accurate structural geometry of the deteriorated straight-lines patterns generated based on the octagon-square basic grid. This research also concluded that it is possible to apply the same conceptual method to reconstruct all two-dimensional Islamic geometric patterns. Moreover, the same methodology can be applied to reconstruct many other pattern systems. The conceptual framework proposed by this study can serve as a platform for developing professional softwares related to historic documentation. Future research should be directed more towards developing artificial intelligence and pattern recognition techniques that have the ability to suplement human power in accomplishing difficult tasks

Virtual 3D Reconstruction of Archaeological Pottery Using Coarse Registration

The 3D reconstruction of objects has not only improved visualisation of digitised objects, it has helped researchers to actively carry out archaeological pottery. Reconstructing pottery is significant in archaeology but is challenging task among practitioners. For one, excavated potteries are hardly complete to provide exhaustive and useful information, hence archaeologists attempt to reconstruct them with available tools and methods. It is also challenging to apply existing reconstruction approaches in archaeological documentation. This limitation makes it difficult to carry out studies within a reasonable time. Hence, interest has shifted to developing new ways of reconstructing archaeological artefacts with new techniques and algorithms. Therefore, this study focuses on providing interventions that will ease the challenges encountered in reconstructing archaeological pottery. It applies a data acquisition approach that uses a 3D laser scanner to acquire point cloud data that clearly show the geometric and radiometric properties of the object’s surface. The acquired data is processed to remove noise and outliers before undergoing a coarse-to-fine registration strategy which involves detecting and extracting keypoints from the point clouds and estimating descriptions with them. Additionally, correspondences are estimated between point pairs, leading to a pairwise and global registration of the acquired point clouds. The peculiarity of the approach of this thesis is in its flexibility due to the peculiar nature of the data acquired. This improves the efficiency, robustness and accuracy of the approach. The approach and findings show that the use of real 3D dataset can attain good results when used with right tools. High resolution lenses and accurate calibration help to give accurate results. While the registration accuracy attained in the study lies between 0.08 and 0.14 mean squared error for the data used, further studies will validate this result. The results obtained are nonetheless useful for further studies in 3D pottery reassembly

A Survey of Geometric Analysis in Cultural Heritage

We present a review of recent techniques for performing geometric analysis in cultural heritage (CH) applications. The survey is aimed at researchers in the areas of computer graphics, computer vision and CH computing, as well as to scholars and practitioners in the CH field. The problems considered include shape perception enhancement, restoration and preservation support, monitoring over time, object interpretation and collection analysis. All of these problems typically rely on an understanding of the structure of the shapes in question at both a local and global level. In this survey, we discuss the different problem forms and review the main solution methods, aided by classification criteria based on the geometric scale at which the analysis is performed and the cardinality of the relationships among object parts exploited during the analysis. We finalize the report by discussing open problems and future perspectives

Digital reintegration of distributed mural paintings at different architectural phases: the case of St. Quirze de Pedret

Sant Quirze de Pedret is a Romanesque church located in Cercs (Catalonia, Spain) at the foothills of the Pyrenees. Its walls harbored one of the most important examples of mural paintings in Catalan Romanesque Art. However, in two different campaigns (in 1921 and 1937) the paintings were removed using the strappo technique and transferred to museums for safekeeping. This detachment protected the paintings from being sold in the art market, but at the price of breaking the integrity of the monument. Nowadays, the paintings are exhibited in the Museu Nacional d'Art de Catalunya - MNAC (Barcelona, Catalonia) and the Museu Diocesà i Comarcal de Solsona - MDCS (Solsona, Catalonia). Some fragments of the paintings are still on the walls of the church. In this work, we present the methodology to digitally reconstruct the church building at its different phases and group the dispersed paintings in a single virtual church, commissioned by the MDCS. We have combined 3D reconstruction (LIDAR and photogrammetric using portable artificial illumination) and modeling techniques (including texture transfer between different shapes) to recover the integrity of the monument in a single 3D virtual model. Furthermore, we have reconstructed the church building at different significant historical moments and placed actual paintings on its virtual walls, based on archaeological knowledge. This set of 3D models allows experts and visitors to better understand the monument as a whole, the relations between the different paintings, and its evolution over time.The project has been promoted by the Museu Diocesà i Comarcal de Solsona (special thanks are due to Carles Freixes and Lídia Fàbregas), co-financed by "La Caixa" Foundation and the Department of Culture of the Generalitat de Catalunya. Likewise, the work presented here counted with the support of MEIC (Spanish Government) project 3D4LIFE (TIN-2017-88515-C2), PRECA II of the Universitat de Barcelona (HAR2017-84451-P) and EHEM (JPICH0127 Conservation, Protection and Use, PCI2020-111979). The high-resolution photographs of the paintings are by Gaetano Alfano (Università degli Studi della Tuscia).Peer ReviewedPostprint (published version

APPLICATION IN CULTURAL HERITAGE OF STEREOLITOGRAPHY TECHNIQUES FOR THE 3D REPRODUCTION

With the use of stereolithography, one of the most widespread and old rapid prototyping techniques, it is possible to reproduce objects, damaged or missing parts, faithful replicas of interest in the field of Cultural Heritage. In this work, some of the applications of the stereolithography technique to replicate objects or missing parts of artistic or cultural interest, are illustrated. The aim of the Authors is to highlight the potential, the advantages, and also the limits if any, of this technique when applied in the field of Cultural Heritage for various needs, reporting some real cases of objects obtained in the laboratory with stereolithography

Fuentes de color mejoradas para el modelado tridimensional de artefactos arqueológicos de tamaño medio localizados in situ.

[EN] The paper describes a color enhanced processing system - applied as case study on an artifact of the Pompeii archaeological area - developed in order to enhance different techniques for reality-based 3D models construction and visualization of archaeological artifacts. This processing allows rendering reflectance properties with perceptual fidelity on a consumer display and presents two main improvements over existing techniques: a. the color definition of the archaeological artifacts; b. the comparison between the range-based and photogrammetry-based pipelines to understand the limits of use and suitability to specific objects.[ES] El documento describe un sistema mejorado de procesamiento de color, aplicado como caso de estudio sobre un artefacto de la zona arqueológica de Pompeya. Este sistema se ha desarrollado con la finalidad de mejorar las diferentes técnicas para la construcción de modelos 3D basados sobre datos de la realidad y para la visualización de artefactos arqueológicos. Este proceso permite visualizar las propiedades de reflectancia con fidelidad perceptible en una pantalla de usuario y presenta dos mejoras principales respecto a las técnicas existentes:a. la definición del color de los artefactos arqueológicos;b. la comparación entre los flujos de trabajo basados en range-based-modeling y en fotogrametría, para entender los límites de uso y la adecuación a los objetos específicos.Apollonio, FI.; Ballabeni, M.; Gaiani, M. (2014). Color enhanced pipelines for reality-based 3D modeling of on site medium sized archeological artifacts. Virtual Archaeology Review. 5(10):59-76. https://doi.org/10.4995/var.2014.4218OJS5976510AGISOFT PHOTOSCAN (2014), http://www.agisoft.ru.ALLEN P., FEINER S., et al. (2004): "Seeing into the past: Creating a 3D modeling pipeline for archaeological visualization", in Proceedings of 3DPVT '04, 2004, pp. 751-758.BERALDIN J.-A., PICARD M., et al. (2002): "Virtualizing a byzantine crypt by combining high-resolution textures with laser scanner 3D data", in Proceedings of VSMM 2002, pp. 3-14.BERNARDINI F., RUSHMEIER H. (2000): "The 3D model acquisition pipeline", in Eurographics 2000 State of the Art Reports.BLAIS F. (2004): "A review of 20 years of Range Sensors Development", in Journal of Electronic Imaging, Vol. 13, N. 1, pp. 231-40. http://dx.doi.org/10.1117/1.1631921BLAIS F., BERALDIN J.A. (2006): "Recent Developments in 3D Multi-modal Laser Imaging Applied to Cultural Heritage, in Machine Vision and Applications, Vol. 17, N. 6, pp. 395-409. http://dx.doi.org/10.1007/s00138-006-0025-3BOEHLER W. (2005): "Comparison of 3D scanning and other 3D measurement techniques", in Baltsavias E., Gruen, A., et al. (eds), Recording, Modeling and Visualization of Cultural Heritage, Taylor & Francis.BOOCHS F., BENTKOWSKA-KAFEL A., et al. (2013): "Towards optimal spectral and spatial documentation of Cultural Heritage. COSCH - an interdisciplinary action in the COST framework", in ISPRS Arch., Vol. XL-5/W2, 2013, pp. 109-113.CALLIERI M., CIGNONI P., et al. (2008): "Masked photo blending: mapping dense photographic dataset on high-resolution 3D models", in Computer & Graphics, Vol. 32, N. 4, 2008, pp. 464 - 473.CALLIERI M., DELLEPIANE M., et al. (2011): "Processing Sampled 3D Data: Reconstruction and Visualization Technologies", in F. Stanco, S. Battiato, G. Gallo (eds.), Digital Imaging for Cultural Heritage Preservation: Analysis, Restoration and Reconstruction of Ancient Artworks, Taylor and Francis, pp. 105-136.CORSINI M., DELLEPIANE M., et al. (2009):"Image-to-geometry registration: a mutual information method exploiting illumination-related geometric properties", in Computer Graphics Forum, Vol. 28, N. 7, 2009, pp. 1755-1764. http://dx.doi.org/10.1111/j.1467-8659.2009.01552.xDANA K.J., VAN GINNEKEN B., et al.. (1999): "Reflectance and texture of real-world surfaces", in ACM Transaction on Graphics, Vol. 18, N. 1, 1999, pp. 1-34. http://dx.doi.org/10.1145/300776.300778DE LUCA L., VERON P., FLORENZANO M. (2006): "Reverse engineering of architectural buildings based on a hybrid modeling approach", Computer & Graphics, Vol. 30, N. 2, pp. 160-76. http://dx.doi.org/10.1016/j.cag.2006.01.020DEBEVEC P. et al. (2004): "Estimating surface reflectance properties of a complex scene under captured natural illumination", in USC ICT Technical Report ICT-TR, 06/2004.DELLEPIANE M., MARROQUIM R., et al. (2012): "Flow-Based Local Optimization for Image-to-Geometry Projection", in IEEE Transactions on Visualization and Computer Graphics, Vol. 18, N. 3, 2012, pp. 463-474. http://dx.doi.org/10.1109/TVCG.2011.75DELLEPIANE M., DELL'UNTO N., et al. (2013a): "Archeological excavation monitoring using dense stereo matching techniques", in Journal of Cultural Heritage, Vol. 14, N. 3, 2013, pp. 201-210. http://dx.doi.org/10.1016/j.culher.2012.01.011DELLEPIANE M., SCOPIGNO R. (2013b): "Global refinement of image-to-geometry registration for color projection", in DigitalHeritage 2013 Proceedings, 2013, Vol. 1, pp. 39-46.DXO (2014), http://www.dxo.com/intl/photography/dxo-optics-pro/EL-HAKIM S.F., BRENNER C., ROTH G. (1998): "A multi-sensor approach to creating accurate virtual environments", in ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 53, N. 6, pp. 379-391. http://dx.doi.org/10.1016/S0924-2716(98)00021-5EL-HAKIM S.F., BERALDIN J.-A., et al. (2004): "Detailed 3D reconstruction of large-scale heritage sites with integrated techniques", in Computer Graphics and Applications, Vol. 24, N. 3, 2004, pp. 21-29. http://dx.doi.org/10.1109/MCG.2004.1318815EL-HAKIM S.F., BERALDIN J.-A. (2007): "Sensor integration and visualization", in Fryer, Mitchell & Chandler (eds.), Applications of 3D Measurement from Images, Whittles Publishing, pp. 259-298.ENGLISH HERITAGE (2005): Metric Survey Specifications for English Heritage. English Heritage Report.ENGLISH HERITAGE (2011), 3D Laser Scanning for Heritage (second edition), English Heritage Publishing.FURUKAWA Y., PONCE J. (2010): "Accurate, dense, and robust multi-view stereopsis", in IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, N. 8, pp. 1362-1376. http://dx.doi.org/10.1109/TPAMI.2009.161GAIANI M., MICOLI L.L. (2005): "A framework to build and visualize 3D models from real world data for historical architecture and archaeology as a base for a 3D information system", in Forte M. (a cura di), The reconstruction of Archaeological Landscapes through Digital Technologies, BAR International series, 1379, pp. 103-125.GAIANI M., ROSSI M., RIZZI A. (2003): "Percezione delle immagini virtuali", in M. Gaiani (ed.), Metodi di Prototipazione Digitale e Visualizzazione per il Disegno Industriale, l'Architettura degli Interni e i Beni Culturali, Polidesign, Milano, 2003.GAIANI M., BENEDETTI B., REMONDINO F. (eds) (2010): Modelli digitali 3D in archeologia: il caso di Pompei, Edizioni della Normale, Pisa, 2010.GAŠPAROVIC M., MALARIC I. (2012): "Increase of readability and accuracy of 3D models using fusion of Close Range Photogrammetry and Laser Scanning", in ISPRS Arch. Photogramm. Remote Sens., Vol. XXXIX-B5, pp. 93-98.GODIN G., BORGEAT L., et al. (2010): "Issues in Acquiring, Processing and Visualizing Large and Detailed 3D Models", in Information Sciences and Systems (CISS), 44th Annual Conference on, pp.1-6. http://dx.doi.org/10.1109/ciss.2010.5464966GONIZZI BARSANTI S., MICOLI L.L., GUIDI G. (2013a): "Quick textured mesh generation for massive 3D digitization of museum artifacts", in DigitalHeritage 2013, Vol. 1, pp. 197-200.GONIZZI BARSANTI S., REMONDINO F., VISINTINI D. (2013b): "3D surveying and modeling of archaeological sites - some critical issues", in ISPRS Ann. Photogramm. Remote Sens., Vol. II-5/W1, 2013, pp. 145-150.GRUSSENMEYER P., LANDES T., et al. (2008): "Comparison methods of terrestrial laser scanning, photogrammetry and tacheometry data for recording of cultural heritage buildings", in ISPRS Arch. Photogramm. Remote Sens., Vol. XXXVII/W5, pp. 213-218.GUARNIERI A., REMONDINO F., VETTORE A. (2006): "Digital photogrammetry and TLS data fusion applied to Cultural Heritage 3D modeling", in ISPRS Arch., Vol. XXXVI/W6, pp. 6.HAPPA J., BASHFORD-ROGERS T., et al. (2012): "Cultural Heritage Predictive Rendering", in Computer Graphics Forum, Vol. 31, N. 6, 2012, pp. 1823-1836. http://dx.doi.org/10.1111/j.1467-8659.2012.02098.xHIRSCHMÜLLER H. (2005): "Accurate and efficient stereo processing by semi-global matching and mututal information", in CVPR 2005 proceedings, Vol. 2, pp. 807-814.HIRSCHMUELLER H. (2008): "Stereo processing by semi- global matching and mutual information", in IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, N. 2, pp. 328-41. http://dx.doi.org/10.1109/TPAMI.2007.1166KARSIDAG G., ALKAN R.M. (2012): "Analysis of The Accuracy of Terrestrial Laser Scanning Measurements", in FIG Working Week 2012 - Knowing to manage the territory, protect the environment, evaluate the cultural heritage proceedings, TS07A - Laser Scanners I, 6097.KAWAKAMI R., IKEUCHI K., TAN R.T. (2005): "Consistent surface color for texturing large objects in outdoor scenes", in ICCV 2005 proceedings, Vol. 2, 2005, pp. 1200-1207.IMATEST (2014), http://www.imatest.com/homeIMAGENOMIC LLC (2012): Noiseware 5 Plug-In User's Guide, 2012INNOVMETRIC POLYWORKS (2014): http://www.innovmetric.com/polyworks/Surveying/LENSCH H.P.A., KAUTZ J., et al. (2003): "Image-based reconstruction of spatial appearance and geometric detail", in ACM Trans. Graph., Vol. 22, N. 2, 2003, pp. 234-257. http://dx.doi.org/10.1145/636886.636891LOWE D. (2004): "Distinctive image features from scale-invariant keypoints", in IJCV, Vol. 60, N. 2, 2004, pp. 91-110.MESHLAB (2014): http://meshlab.sourceforge.net/MUDGE M., SCHROER C., et al. (2010): "Principles and Practices of Robust, Photography based Digital Imaging Techniques for Museums", in VAST 2010 Proceedings, 2010, pp. 111-137.NICODEMUS F. (1965): "Directional reflectance and emissivity of an opaque surface", in Applied Optics, Vol. 4, N. 7, 1965, pp. 767-775. http://dx.doi.org/10.1364/AO.4.000767OPENGL (2014): http://www.opengl.orgPASCALE D. (2006): "RGB coordinates of the Macbeth ColorChecker", in Technical report, The BabelColor Company, Jun 2006.PETROSYAN A., GHAZARYAN A. (2006): "Method and System for Digital Image Enhancement", in US Patent Application, #11/116, 408, 2006.PIERROT-DESEILLIGNY M., PAPARODITIS N. (2006): A multiresolution and optimization-based image matching approach: an application to surface reconstruction from SPOT5-HRS stereo imagery, in ISPRS Arch., Vol. XXXVI-1/W41.PIETRONI N., TARINI M., CIGNONI P. (2010): "Almost isometric mesh parameterization through abstract domains", in IEEE Trans. on Visualization and Computer Graphics, Vol. 16, N. 4, 2010, pp. 621-635. http://dx.doi.org/10.1109/TVCG.2009.96REINHARD E., ARIF KHAN E., OGUZ AKYÜZ A., JOHNSON G. (2008): Color Imaging Fundamentals and Applications, A. K. Peters, Wellesley.REMONDINO F., EL-HAKIM S. (2006): "Image-based 3D modelling: a review", in The Photogrammetric Record, Vol. 21, N.115, 2006, pp. 269-291. http://dx.doi.org/10.1111/j.1477-9730.2006.00383.xREMONDINO F., CAMPANA S., (eds.) (2014): 3D Recording and Modelling in Archaeology and Cultural Heritage, BAR International Series 2598, Archaeopress.REMONDINO F., GUARNIERI A., VETTORE A. (2005): "3D modeling of close-range objects: photogrammetry or laser scanning?", in Procceedings of Videometrics VIII, SPIE-IS&T Electronic Imaging, Vol. 5665, pp. 216-225. http://dx.doi.org/10.1117/12.586294REMONDINO F., EL-HAKIM S., et al. (2008a): "Development and performance analysis of image matching for detailed surface reconstruction of heritage objects", in IEEE Signal Processing Magazine, Vol. 25, N.4, pp. 55-65. http://dx.doi.org/10.1109/MSP.2008.923093REMONDINO F., EL-HAKIM S., GRUEN A., ZHANG L. (2008b): " Turning images into 3D models - Development and performance analysis of image matching for detailed surface reconstruction of heritage objects", in IEEE Signal Processing Magazine, Vol. 25, N.4, 2008, pp. 55-65. http://dx.doi.org/10.1109/MSP.2008.923093REMONDINO F., et al. (2012): "Low-Cost and Open-Source Solutions for Automated Image Orientation - A Critical Overview", in Euromed 2012 Proceedings, pp. 40-54. http://dx.doi.org/10.1007/978-3-642-34234-9_5RUSHMEIER H., BERNARDINI F. (1999): "Computing consistent normals and colors from photometric data," in 3DIM 1999 proceedings, pp. 99-108. http://dx.doi.org/10.1109/im.1999.805339SANTOPUOLI N., SECCIA L. (2008): "Il rilievo del colore nel campo dei beni culturali", in Trattato di Restauro Architettonico, UTET, 2008, Vol. X, pp. 141-163.SCOPIGNO R., CALLIERI M., et al. (2011): "3D Models for Cultural Heritage: Beyond Plain Visualization", in Computer , Vol. 44, N. 7, pp. 48-55.SCHWARTZ C., WEINMANN M., RUITERS R., KLEIN R. (2011): "Integrated High-Quality Acquisition of Geometry and Appearance for Cultural Heritage", in VAST 2011 Proceedings, 2011, pp. 25-32.SEITZ S.M., et al. (2006): "A comparison and evaluation of multi-view stereo reconstruction algorithms", in CVPR Proceedings, Vol. 1, 2006, pp. 519-528. http://dx.doi.org/10.1109/cvpr.2006.19SINHA S.N., POLLEFEYS M. (2005): "Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation", in Proc. 10th ICCV, pp. 349-356. http://dx.doi.org/10.1109/iccv.2005.159STAMOS I., LIU L., et al. (2008): "Integrating automated range registration with multiview geometry for the photorealistic modelling of large-scale scenes", in International Journal of Computer Vision Vol. 78, N. 2-3, pp. 237-60. http://dx.doi.org/10.1007/s11263-007-0089-1STUMPFEL J., TCHOU C., et al. (2003): "Digital reunification of the Parthenon and its sculptures", in Proceedings of Virtual Reality, Archaeology and Cultural Heritage (VAST) 2003, pp. 41-50.TOMASI C., KANADE T. (1992): "Shape and motion from image streams under orthography - a factorization method", in IJCV, Vol. 9, N. 2, 1992, pp. 137-154.TROCCOLI A., ALLEN P.K. (2005): "Relighting acquired models of outdoor scenes", in 3DIM 2005 proceedings, pp. 245-252. http://dx.doi.org/10.1109/3dim.2005.69VOSSELMAN G., MAAS H. (2010): Airborne and Terrestrial Laser Scanning, CRC Press.VOGIATZIS G., HERNANDEZ C., et al. (2007): "Multi-view stereo via volumetric graph-cuts and occlusion robust photo-consistency", in IEEE Trans. PAMI, Vol. 29, N. 12, pp. 2241-2246. http://dx.doi.org/10.1109/TPAMI.2007.70712VU H.H., LABATUT P., et al. (2012): "High accuracy and visibility-consistent dense multiview stereo", in IEEE Trans. PAMI, Vol. 34, N. 5, pp. 889-901. http://dx.doi.org/10.1109/TPAMI.2011.172WENZEL K., ABDEL-WAHAB M., et al., (2012): "High-Resolution Surface Reconstruction from Imagery for Close Range Cultural Heritage Applications", in ISPRS Arch., Vol. XXXIX-B5, pp. 133-138.WU C. (2013): "Towards Linear-Time Incremental Structure from Motion", in 3D Vision, 2013 International Conference on, 2013, pp.127-134. http://dx.doi.org/10.1109/3dv.2013.25XU C., A. GEORGHIADES S., et al. (2006): "A System for Reconstructing Integrated Texture Maps for Large Structures", in 3DPVT'06 proceedings, 2006, pp.822-829.YU Y., MALIK J. (1998): "Recovering photometric properties of architectural scenes from photographs", in SIGGRAPH '98 proceedings, 1998, pp. 207-217.ZHANG L. (2005): "Automatic Digital Surface Model (DSM) generation from linear array images", Ph.D. Thesis, Institute of Geodesy and Photogrammetry, ETH Zurich, Switzerland.ZHAO X., ZHOU Z., WU W. (2012): "Radiance-based color calibration for image-based modeling with multiple cameras", in Science China Information Sciences, Vol. 55, N. 7, 2012, pp. 1509-1519

Material Recognition Meets 3D Reconstruction : Novel Tools for Efficient, Automatic Acquisition Systems

For decades, the accurate acquisition of geometry and reflectance properties has represented one of the major objectives in computer vision and computer graphics with many applications in industry, entertainment and cultural heritage. Reproducing even the finest details of surface geometry and surface reflectance has become a ubiquitous prerequisite in visual prototyping, advertisement or digital preservation of objects. However, today's acquisition methods are typically designed for only a rather small range of material types. Furthermore, there is still a lack of accurate reconstruction methods for objects with a more complex surface reflectance behavior beyond diffuse reflectance. In addition to accurate acquisition techniques, the demand for creating large quantities of digital contents also pushes the focus towards fully automatic and highly efficient solutions that allow for masses of objects to be acquired as fast as possible. This thesis is dedicated to the investigation of basic components that allow an efficient, automatic acquisition process. We argue that such an efficient, automatic acquisition can be realized when material recognition "meets" 3D reconstruction and we will demonstrate that reliably recognizing the materials of the considered object allows a more efficient geometry acquisition. Therefore, the main objectives of this thesis are given by the development of novel, robust geometry acquisition techniques for surface materials beyond diffuse surface reflectance, and the development of novel, robust techniques for material recognition. In the context of 3D geometry acquisition, we introduce an improvement of structured light systems, which are capable of robustly acquiring objects ranging from diffuse surface reflectance to even specular surface reflectance with a sufficient diffuse component. We demonstrate that the resolution of the reconstruction can be increased significantly for multi-camera, multi-projector structured light systems by using overlappings of patterns that have been projected under different projector poses. As the reconstructions obtained by applying such triangulation-based techniques still contain high-frequency noise due to inaccurately localized correspondences established for images acquired under different viewpoints, we furthermore introduce a novel geometry acquisition technique that complements the structured light system with additional photometric normals and results in significantly more accurate reconstructions. In addition, we also present a novel method to acquire the 3D shape of mirroring objects with complex surface geometry. The aforementioned investigations on 3D reconstruction are accompanied by the development of novel tools for reliable material recognition which can be used in an initial step to recognize the present surface materials and, hence, to efficiently select the subsequently applied appropriate acquisition techniques based on these classified materials. In the scope of this thesis, we therefore focus on material recognition for scenarios with controlled illumination as given in lab environments as well as scenarios with natural illumination that are given in photographs of typical daily life scenes. Finally, based on the techniques developed in this thesis, we provide novel concepts towards efficient, automatic acquisition systems