Annotation sémantique 2D/3D d'images spatialisées pour la documentation et l'analyse d'objets patrimoniaux

In the field of architecture and historic preservation , the information and communication technologies enable the acquisition of large amounts of data introducing analysis media for different purposes and at different levels of details ( photographs, point cloud, scientific imaging, ...). The organization and the structure of these resources is now a major problem for the description, the analysis and the understanding of cultural heritage objects. However the existing solutions in semantic annotations on images or on 3D model are insufficient, especially in the linking of different analysis media.This thesis proposes an approach for conducting annotations on different two-dimensional media while allowing the propagation of these annotations between different representations (2D or 3D) of the object. The objective is to identify solutions to correlate (from a spatial, temporal and semantic point of view) sets of annotations within sets of images. Thus, the system is based on the principle of data spatialization for establishing a relationship between the 3D representations, incorporating all the geometric complexity of the object and therefore to the metric information extraction, and 2D representations of object. The approach seeks to the establishment of an information continuity from the image acquisition to the construction of 3D representations semantically enhanced by incorporating multi-media and multi-temporal aspects. This work resulted in the definition and the development of a set of software modules that can be used by specialists of conservation of architectural heritage as by the general public.Dans le domaine de l’architecture et de la conservation du patrimoine historique, les technologies de l’information et de la communication permettent l’acquisition de grandes quantités de données introduisant des supports d’analyses pour différentes finalités et à différents niveaux de détails (photographies, nuages de points, imagerie scientifique, …). L’organisation et la structuration de ces ressources est aujourd’hui un problème majeur pour la description, l’analyse et la compréhension d’objets patrimoniaux. Cependant les solutions existantes d’annotations sémantiques d’images ou de modèle 3D se révèlent insuffisantes notamment sur l’aspect de mise en relation des différents supports d’analyse.Cette thèse propose une approche permettant de conduire des annotations sur les différents supports bidimensionnels tout en permettant la propagation de ces annotations entre les différentes représentations (2D ou 3D) de l’objet. L’objectif est d’identifier des solutions pour corréler (d’un point de vue spatial, temporel et sémantique) des jeux d’annotations au sein d’un jeu d’images. Ainsi le système repose sur le principe de spatialisation des données permettant d’établir une relation entre les représentations 3D, intégrant toute la complexité géométrique de l’objet et par conséquent permettant l’extraction d’informations métriques, et les représentations 2D de l’objet. L’approche cherche donc à la mise en place d’une continuité informationnelle depuis l’acquisition d’images jusqu’à la construction de représentations 3D sémantiquement enrichies en intégrant des aspects multi-supports et multi-temporels. Ce travail a abouti à la définition et le développement d’un ensemble de modules informatiques pouvant être utilisés par des spécialistes de la conservation d’un patrimoine architectural comme par le grand public

An approach for precise 2D/3D semantic annotation of spatially-oriented images for in-situ visualization applications

Thanks to nowadays technologies, innovative tools afford to increase our knowledge of historic monuments, in the field of preservation and valuation of cultural heritage. These tools are aimed to help experts to create, enrich and share information on historical buildings. Among the various documentary sources, photographs contain a high level of details about shapes and colors. With the development of image analysis and image-based-modeling techniques, large sets of images can be spatially oriented towards a digital mock-up. For these reasons, digital photographs prove to be an easy to use, affordable and flexible support, for heritage documentation. This article presents, in a first step, an approach for 2D/3D semantic annotations in a set of spatially-oriented photographs (whose positions and orientations in space are automatically estimated). In a second step, we will focus on a method for displaying those annotations on new images acquired by mobile devices in situ. Firstly, an automated image-based reconstruction method produces 3D information (specifically 3D coordinates) by processing a large images set. Then, images are semantically annotated and a process uses the previously generated 3D information inherent to images for the annotations transfer. As a consequence, this protocol provides a simple way to finely annotate a large quantity of images at once instead of one by one. As those images annotations are directly inherent to 3D information, they can be stored as 3D files. To bring up on screen the information related to a building, the user takes a picture in situ. An image processing method allows estimating the orientation parameters of this new photograph inside the already oriented large images base. Then the annotations can be precisely projected on the oriented picture and send back to the user. In this way a continuity of information could be established from the initial acquisition to the in situ visualization

A semi-automatic 2D/3D annotation framework for the geometric analysis of heritage artefacts

International audienceDocumentation and monitoring of heritage objects involve many actors on multidisciplinary aspects. The progress made over the years in the field of digital technologies has enabled many tools for analysis, management and dissemination of information gathered around an object. These tools must allow users to semantically describe the object while allowing them to grasp its morphological complexity and the heterogeneity of the available analysis supports. This article introduces an approach for the semantic annotation of heritage objects by using the bijective relationship that can be established between a 3D representation of an object and the set of oriented images towards it, while maintaining a continuum of information between all phases of observation and description, from acquisition to visualization of semantically enriched representations. The main idea is to offer a versatile environment to help extraction of relevant information from images using geometric descriptors and semi-automatic point cloud processing methods

A Hybrid Approach for the Semantic Annotation of Spatially Oriented Images

Thanks to recent developments, actual information technologies offer tools for the access to information about cultural heritage buildings. These tools are aimed to help experts to enrich and share information about historic monuments. This article describes a hybrid process for the semantic annotation of a set of spatially-oriented images (whose positions and orientations in space are automatically estimated). First of all, a large set of images is processed with an automated image based reconstruction method so as to generate 3D information (specifically 3D coordinates). This 3D information is then used in an automated transfer process of annotations through the set of images. At last, new images can be oriented within the set of already-oriented and already-annotated images. Then, annotations can be precisely projected on newly-oriented images. As a consequence, this process provides a simple way to establish a continuity of information inside a set of images

2D/3D Semantic Annotation of Spatialized Images for the Documentation and Analysis of Cultural Heritage

The introduction of digital technologies in the documentation methods for cultural heritage has helped to design new tools for the acquisition and management of information collected for multidisciplinary studies. These tools are beginning to emerge as the preferred media for describing, analyzing and understanding the objects of study. Existing solutions for semantic annotation on images, on 3D models or with 2D/3D hybrid methods still reveal themselves today insufficient to tackle the complex problem of annotating heritage artifacts. In this field, the semantic description of the studied objects must be able to rely on a rich and structured representation by on one hand making explicit the morphological complexity of the object and on the other hand by reflecting all aspects conveyed by the acquisitions of scientific imaging. This paper introduces an approach for conducting semantic annotations on 2D images (photography, scientific imaging ...) while facilitating the annotation work with an automatic propagation of these annotations between other correlated representations (2D or 3D) of the object. It is based on a spatial referencing method aiming at the establishment of a continuous 2D/3D projective relationship. The goal of the approach is to define an informative continuum between all phases of observation and description processes ranging from the acquisition of images and spatial data up to the building of semantically-enriched 3D representations. The idea is to insert semantics at all phases of 2D/3D data processing while ensuring a continuous correlation of annotations from a spatial, temporal and morphological point of view

An approach for precise 2D/3D semantic annotation of spatially-oriented images for in-situ visualization applications

Thanks to nowadays technologies, innovative tools afford to increase our knowledge of historic monuments, in the field of preservation and valuation of cultural heritage. These tools are aimed to help experts to create, enrich and share information on historical buildings. Among the various documentary sources, photographs contain a high level of details about shapes and colors. With the development of image analysis and image-based-modeling techniques, large sets of images can be spatially oriented towards a digital mock-up. For these reasons, digital photographs prove to be an easy to use, affordable and flexible support, for heritage documentation. This article presents, in a first step, an approach for 2D/3D semantic annotations in a set of spatially-oriented photographs (whose positions and orientations in space are automatically estimated). In a second step, we will focus on a method for displaying those annotations on new images acquired by mobile devices in situ. Firstly, an automated image-based reconstruction method produces 3D information (specifically 3D coordinates) by processing a large images set. Then, images are semantically annotated and a process uses the previously generated 3D information inherent to images for the annotations transfer. As a consequence, this protocol provides a simple way to finely annotate a large quantity of images at once instead of one by one. As those images annotations are directly inherent to 3D information, they can be stored as 3D files. To bring up on screen the information related to a building, the user takes a picture in situ. An image processing method allows estimating the orientation parameters of this new photograph inside the already oriented large images base. Then the annotations can be precisely projected on the oriented picture and send back to the user. In this way a continuity of information could be established from the initial acquisition to the in situ visualization

Distribution of semantic annotations towards a set of spatially-oriented photographs

Today with the development of new information technologies, the domain of conservation and promotion of cultural heritage benefits from new tools to disseminate knowledge on heritage buildings. These tools are intended to help experts to enrich and access information about buildings. This paper describes this critical issue and proposes an approach permitting the automatic transfer of annotations in a set of oriented photographs (whose positions and orientations in space are known) by using 3D information. First of all, photographs are processed with an automated image-based 3D reconstruction method to produce 3D information (specifically 3D coordinates). Then, 3D information is used in the process of annotations’ transfer between images. Finally, with this process, the modification of annotations can be easily implemented. As a consequence, this process provides a simple way to annotate blocs of images all at once instead of one by one

An approach for precise 2D/3D semantic annotation of spatially-oriented images for in-situ visualization applications

Thanks to nowadays technologies, innovative tools afford to increase our knowledge of historic monuments, in the field of preservation and valuation of cultural heritage. These tools are aimed to help experts to create, enrich and share information on historical buildings. Among the various documentary sources, photographs contain a high level of details about shapes and colors. With the development of image analysis and image-based-modeling techniques, large sets of images can be spatially oriented towards a digital mock-up. For these reasons, digital photographs prove to be an easy to use, affordable and flexible support, for heritage documentation. This article presents, in a first step, an approach for 2D/3D semantic annotations in a set of spatially-oriented photographs (whose positions and orientations in space are automatically estimated). In a second step, we will focus on a method for displaying those annotations on new images acquired by mobile devices in situ. Firstly, an automated image-based reconstruction method produces 3D information (specifically 3D coordinates) by processing a large images set. Then, images are semantically annotated and a process uses the previously generated 3D information inherent to images for the annotations transfer. As a consequence, this protocol provides a simple way to finely annotate a large quantity of images at once instead of one by one. As those images annotations are directly inherent to 3D information, they can be stored as 3D files. To bring up on screen the information related to a building, the user takes a picture in situ. An image processing method allows estimating the orientation parameters of this new photograph inside the already oriented large images base. Then the annotations can be precisely projected on the oriented picture and send back to the user. In this way a continuity of information could be established from the initial acquisition to the in situ visualization

Laying the foundations for an information system dedicated to heritage building degradation monitoring based on the 2D/3D semantic annotation of photographs

In the last decade many 3D digitization techniques have emerged allowing the generation of dense and precise digital representations of historical building. However, regardless their level of geometric accuracy or visual realism, 3D models are not yet fully adapted to the conservation analysis purposes. In fact, even if a 3D model can be considered an efficient way to accurately record the state of a building, its potentiality in terms of semantic annotation and spatial distribution of heterogeneous data still remain almost unexplored today. Since several years, photographs appear to be a flexible and well diffused portable support for the heritage documentation. They are a natural medium to annotate and compare temporal states. Thanks to the recent advances in photogrammetry computer vision and augmented reality, photographs can be also considered as an excellent support for accurate spatial localization. This article presents the first principles for the development of an information system to monitor the historic building degradation based on three main components: a high dynamic range (HDR) imagebased automatic pipeline, an hybrid (2D/3D) semantic annotation method and a domain ontology describing knowledge related to degradation phenomena. The innovative integration of these main components allows us to introduce the notion of ”informative continuum” as a key for interconnecting spatialized and semanticallyenriched photographs to populate a knowledge base on the building degradation. The first steps of this on-going project are illustrated by an experimentation carried out on the Caromb church in the south of France

Share - Publish - Store - Preserve. Methodologies, Tools and Challenges for 3D Use in Social Sciences and Humanities

Through this White Paper, which gathers contributions from experts of 3D data as well as professionals concerned with the interoperability and sustainability of 3D research data, the PARTHENOS project aims at highlighting some of the current issues they have to face, with possible specific points according to the discipline, and potential practices and methodologies to deal with these issues. During the workshop, several tools to deal with these issues have been introduced and confronted with the participants experiences, this White Paper now intends to go further by also integrating participants feedbacks and suggestions of potential improvements. Therefore, even if the focus is put on specific tools, the main goal is to contribute to the development of standardized good practices related to the sharing, publication, storage and long-term preservation of 3D data