51,430 research outputs found
Recommended from our members
Commonsense Knowledge and Conceptual Structure in Container Metaphors
Cognitive grammar provides an analytic framework in which the semantic value of linguistic expressions is characterized relative to domains of presupposed knowledge. Cognitive metaphor theory holds that metaphorical language involves a mapping of conceptual structure from a source domain to a target domain. Containers are one such pervasive structure. This investigation proposes a detailed representation for the domain CONTAINER and applies it in the analysis of metaphorical expressions mapping CONTAINER onto target domains ARGUMENT and linguistic expression. Each source domain word is analyzed with respect to which aspects of the CONTAINER domain structure it refers, and whether it refers to a 2D or 3D bounded region. The pattern of aspects mapped suggest that spatial containment, content, and material container object comprise major aspects of the 3D CONTAINER domain. The target domains are demonstrated to be structured according this container organization. The results demonstrate that cognitive semantic analysis can reveal specific structures of commonsense knowledge which are prerequisite for language use
Open-Fusion: Real-time Open-Vocabulary 3D Mapping and Queryable Scene Representation
Precise 3D environmental mapping is pivotal in robotics. Existing methods
often rely on predefined concepts during training or are time-intensive when
generating semantic maps. This paper presents Open-Fusion, a groundbreaking
approach for real-time open-vocabulary 3D mapping and queryable scene
representation using RGB-D data. Open-Fusion harnesses the power of a
pre-trained vision-language foundation model (VLFM) for open-set semantic
comprehension and employs the Truncated Signed Distance Function (TSDF) for
swift 3D scene reconstruction. By leveraging the VLFM, we extract region-based
embeddings and their associated confidence maps. These are then integrated with
3D knowledge from TSDF using an enhanced Hungarian-based feature-matching
mechanism. Notably, Open-Fusion delivers outstanding annotation-free 3D
segmentation for open-vocabulary without necessitating additional 3D training.
Benchmark tests on the ScanNet dataset against leading zero-shot methods
highlight Open-Fusion's superiority. Furthermore, it seamlessly combines the
strengths of region-based VLFM and TSDF, facilitating real-time 3D scene
comprehension that includes object concepts and open-world semantics. We
encourage the readers to view the demos on our project page:
https://uark-aicv.github.io/OpenFusio
SSR-2D: Semantic 3D Scene Reconstruction from 2D Images
Most deep learning approaches to comprehensive semantic modeling of 3D indoor
spaces require costly dense annotations in the 3D domain. In this work, we
explore a central 3D scene modeling task, namely, semantic scene reconstruction
without using any 3D annotations. The key idea of our approach is to design a
trainable model that employs both incomplete 3D reconstructions and their
corresponding source RGB-D images, fusing cross-domain features into volumetric
embeddings to predict complete 3D geometry, color, and semantics with only 2D
labeling which can be either manual or machine-generated. Our key technical
innovation is to leverage differentiable rendering of color and semantics to
bridge 2D observations and unknown 3D space, using the observed RGB images and
2D semantics as supervision, respectively. We additionally develop a learning
pipeline and corresponding method to enable learning from imperfect predicted
2D labels, which could be additionally acquired by synthesizing in an augmented
set of virtual training views complementing the original real captures,
enabling more efficient self-supervision loop for semantics. In this work, we
propose an end-to-end trainable solution jointly addressing geometry
completion, colorization, and semantic mapping from limited RGB-D images,
without relying on any 3D ground-truth information. Our method achieves
state-of-the-art performance of semantic scene reconstruction on two
large-scale benchmark datasets MatterPort3D and ScanNet, surpasses baselines
even with costly 3D annotations. To our knowledge, our method is also the first
2D-driven method addressing completion and semantic segmentation of real-world
3D scans
From survey to semantic representation for Cultural Heritage: the 3D modeling of recurring architectural elements
The digitization of Cultural Heritage paves the way for new approaches to surveying and restitution of historical sites. With a view to the management of integrated programs of documentation and conservation, the research is now focusing on the creation of information systems where to link the digital representation of a building to semantic knowledge. With reference to the emblematic case study of the Calci Charterhouse, also known as Pisa Charterhouse, this contribution illustrates an approach to be followed in the transition from 3D survey information, derived from laser scanner and photogrammetric techniques, to the creation of semantically enriched 3D models. The proposed approach is based on the recognition -segmentation and classification- of elements on the original raw point cloud, and on the manual mapping of NURBS elements on it. For this shape recognition process, reference to architectural treatises and vocabularies of classical architecture is a key step. The created building components are finally imported in a H-BIM environment, where they are enriched with semantic information related to historical knowledge, documentary sources and restoration activities
A Proposal for Semantic Map Representation and Evaluation
Semantic mapping is the incremental process of “mapping” relevant information of the world (i.e., spatial information, temporal events, agents and actions) to a formal description supported by a reasoning engine. Current research focuses on learning the semantic of environments based on their spatial location, geometry and appearance. Many methods to tackle this problem have been proposed, but the lack of a uniform representation, as well as standard benchmarking suites, prevents their direct comparison. In this paper, we propose a standardization in the representation of semantic maps, by defining an easily extensible formalism to be used on top of metric maps of the environments. Based on this, we describe the procedure to build a dataset (based on real sensor data) for benchmarking semantic mapping techniques, also hypothesizing some possible evaluation metrics. Nevertheless, by providing a tool for the construction of a semantic map ground truth, we aim at the contribution of the scientific community in acquiring data for populating the dataset
- …