An approach for precise 2D/3D semantic annotation of spatially-oriented images for in-situ visualization applications

Thanks to nowadays technologies, innovative tools afford to increase our knowledge of historic monuments, in the field of preservation and valuation of cultural heritage. These tools are aimed to help experts to create, enrich and share information on historical buildings. Among the various documentary sources, photographs contain a high level of details about shapes and colors. With the development of image analysis and image-based-modeling techniques, large sets of images can be spatially oriented towards a digital mock-up. For these reasons, digital photographs prove to be an easy to use, affordable and flexible support, for heritage documentation. This article presents, in a first step, an approach for 2D/3D semantic annotations in a set of spatially-oriented photographs (whose positions and orientations in space are automatically estimated). In a second step, we will focus on a method for displaying those annotations on new images acquired by mobile devices in situ. Firstly, an automated image-based reconstruction method produces 3D information (specifically 3D coordinates) by processing a large images set. Then, images are semantically annotated and a process uses the previously generated 3D information inherent to images for the annotations transfer. As a consequence, this protocol provides a simple way to finely annotate a large quantity of images at once instead of one by one. As those images annotations are directly inherent to 3D information, they can be stored as 3D files. To bring up on screen the information related to a building, the user takes a picture in situ. An image processing method allows estimating the orientation parameters of this new photograph inside the already oriented large images base. Then the annotations can be precisely projected on the oriented picture and send back to the user. In this way a continuity of information could be established from the initial acquisition to the in situ visualization

An approach for precise 2D/3D semantic annotation of spatially-oriented images for in-situ visualization applications

Thanks to nowadays technologies, innovative tools afford to increase our knowledge of historic monuments, in the field of preservation and valuation of cultural heritage. These tools are aimed to help experts to create, enrich and share information on historical buildings. Among the various documentary sources, photographs contain a high level of details about shapes and colors. With the development of image analysis and image-based-modeling techniques, large sets of images can be spatially oriented towards a digital mock-up. For these reasons, digital photographs prove to be an easy to use, affordable and flexible support, for heritage documentation. This article presents, in a first step, an approach for 2D/3D semantic annotations in a set of spatially-oriented photographs (whose positions and orientations in space are automatically estimated). In a second step, we will focus on a method for displaying those annotations on new images acquired by mobile devices in situ. Firstly, an automated image-based reconstruction method produces 3D information (specifically 3D coordinates) by processing a large images set. Then, images are semantically annotated and a process uses the previously generated 3D information inherent to images for the annotations transfer. As a consequence, this protocol provides a simple way to finely annotate a large quantity of images at once instead of one by one. As those images annotations are directly inherent to 3D information, they can be stored as 3D files. To bring up on screen the information related to a building, the user takes a picture in situ. An image processing method allows estimating the orientation parameters of this new photograph inside the already oriented large images base. Then the annotations can be precisely projected on the oriented picture and send back to the user. In this way a continuity of information could be established from the initial acquisition to the in situ visualization

Analyzing the evolution of deterioration patterns: A first step of an image-based approach for comparing multitemporal data sets

International audienceWhen documenting and analyzing cultural heritage, the monument states can be described by multitemporal data sets, which however present a complication for the elaboration and examination process. This difficulty leads to the necessity to improve the analyze process in order to expand the documentation process and help experts to enrich and share information about the historical buildings. Therefore an approach of change measurement, which supports the chronical comprehension of a building by visualizing and quantifying the dimensional temporal effects was elaborated, where the analyze process of a multitemporal data set was based on the interpretation of depth map images. These maps were obtained by generating ortho images of an object that was created on purpose by setting up an experimentation to acquire a multitemporal data set. This approach is the first step of a wider ongoing research about change detection processes on multitemporal data sets. Index Terms-Change measurements, multitemporal data set, depth map, ortho image, evolution of deterioration patterns

An approach for precise 2D/3D semantic annotation of spatially-oriented images for in-situ visualization applications

Thanks to nowadays technologies, innovative tools afford to increase our knowledge of historic monuments, in the field of preservation and valuation of cultural heritage. These tools are aimed to help experts to create, enrich and share information on historical buildings. Among the various documentary sources, photographs contain a high level of details about shapes and colors. With the development of image analysis and image-based-modeling techniques, large sets of images can be spatially oriented towards a digital mock-up. For these reasons, digital photographs prove to be an easy to use, affordable and flexible support, for heritage documentation. This article presents, in a first step, an approach for 2D/3D semantic annotations in a set of spatially-oriented photographs (whose positions and orientations in space are automatically estimated). In a second step, we will focus on a method for displaying those annotations on new images acquired by mobile devices in situ. Firstly, an automated image-based reconstruction method produces 3D information (specifically 3D coordinates) by processing a large images set. Then, images are semantically annotated and a process uses the previously generated 3D information inherent to images for the annotations transfer. As a consequence, this protocol provides a simple way to finely annotate a large quantity of images at once instead of one by one. As those images annotations are directly inherent to 3D information, they can be stored as 3D files. To bring up on screen the information related to a building, the user takes a picture in situ. An image processing method allows estimating the orientation parameters of this new photograph inside the already oriented large images base. Then the annotations can be precisely projected on the oriented picture and send back to the user. In this way a continuity of information could be established from the initial acquisition to the in situ visualization

Laying the foundations for an information system dedicated to heritage building degradation monitoring based on the 2D/3D semantic annotation of photographs

In the last decade many 3D digitization techniques have emerged allowing the generation of dense and precise digital representations of historical building. However, regardless their level of geometric accuracy or visual realism, 3D models are not yet fully adapted to the conservation analysis purposes. In fact, even if a 3D model can be considered an efficient way to accurately record the state of a building, its potentiality in terms of semantic annotation and spatial distribution of heterogeneous data still remain almost unexplored today. Since several years, photographs appear to be a flexible and well diffused portable support for the heritage documentation. They are a natural medium to annotate and compare temporal states. Thanks to the recent advances in photogrammetry computer vision and augmented reality, photographs can be also considered as an excellent support for accurate spatial localization. This article presents the first principles for the development of an information system to monitor the historic building degradation based on three main components: a high dynamic range (HDR) imagebased automatic pipeline, an hybrid (2D/3D) semantic annotation method and a domain ontology describing knowledge related to degradation phenomena. The innovative integration of these main components allows us to introduce the notion of ”informative continuum” as a key for interconnecting spatialized and semanticallyenriched photographs to populate a knowledge base on the building degradation. The first steps of this on-going project are illustrated by an experimentation carried out on the Caromb church in the south of France

Photogrammétrie multispectrale dans l’étude des peintures murales : Acquisition et traitement de données

International audienceL’apport de la photogrammétrie numérique et plus particulièrement d'orthophotomosaïque nous a conduit à développer un système de fusion de ces données avec des techniques d'imagerie technique déjà largement utilisées dans l'étude des biens culturels

Laying the foundations for an information system dedicated to heritage building degradation monitoring based on the 2D/3D semantic annotation of photographs

International audienceIn the last decade many 3D digitization techniques have emerged allowing the generation of dense and precise digital representations of historical building. However, regardless their level of geometric accuracy or visual realism, 3D models are not yet fully adapted to the conservation analysis purposes. In fact, even if a 3D model can be considered an efficient way to accurately record the state of a building, its potentiality in terms of semantic annotation and spatial distribution of heterogeneous data still remain almost unexplored today. Since several years, photographs appear to be a flexible and well diffused portable support for the heritage documentation. They are a natural medium to annotate and compare temporal states. Thanks to the recent advances in photogrammetry computer vision and augmented reality, photographs can be also considered as an excellent support for accurate spatial localization. This article presents the first principles for the development of an information system to monitor the historic building degradation based on three main components: a high dynamic range (HDR) imagebased automatic pipeline, an hybrid (2D/3D) semantic annotation method and a domain ontology describing knowledge related to degradation phenomena. The innovative integration of these main components allows us to introduce the notion of ”informative continuum” as a key for interconnecting spatialized and semanticallyenriched photographs to populate a knowledge base on the building degradation. The first steps of this on-going project are illustrated by an experimentation carried out on the Caromb church in the south of France

A Shape-Adjusted Tridimensional Reconstruction of Cultural Heritage Artifacts Using a Miniature Quadrotor

International audienceThe innovative automated 3D modeling procedure presented here was used to reconstruct a Cultural Heritage (CH) object by means of an unmanned aerial vehicle. Using a motion capture system, a small low-cost quadrotor equipped with a miniature low-resolution Raspberry Pi camera module was accurately controlled in the closed loop mode and made to follow a trajectory around the artifact. A two-stage process ensured the accuracy of the 3D reconstruction process. The images taken during the first circular trajectory were used to draw the artifact's shape. The second trajectory was smartly and autonomously adjusted to match the artifact's shape, then it provides new pictures taken close to the artifact and, thus, greatly improves the final 3D reconstruction in terms of the completeness, accuracy and quickness, in particular where the artifact's shape is complex. The results obtained here using close-range photogrammetric methods show that the process of automated 3D model reconstruction based on a robotized quadrotor using a motion capture system is a realistic approach, which could provide a suitable new digital conservation tool in the cultural heritage field

A complete methodology for the mechanical diagnosis of statue provided by innovative uses of 3D model. Application to the imperial marble statue of Alba-la-Romaine (France)

International audienceA multidisciplinary methodology is presented to assess the mechanical behaviour of a marble statue with complex fracture plans and localized cracks. Firstly, a 3D model is generated by photogrammetry. Its underlying geometrical data provide valuable insight for the physical characterisation and the numerical analysis. Indeed, the ultrasound analysis, which is usually impossible on such a complex shape, is achieved thanks to the accurate measures of distance between transmitter and receiver obtained from the 3D geometrical model. Finally, an innovative use of FEM/DEM analysis is proposed to evaluate the mechanical relevance of a non-destructive basing system. Reflecting the advances of this collective work, the resulting pedestal solution is non-conventional since it is safe, not invasive and totally reversible. A particular attention is paid to use mainly open-source numerical tools from 3D acquisition through mechanical analysis in order to enable the reproducibility of the process