1,125 research outputs found
Furniture models learned from the WWW: using web catalogs to locate and categorize unknown furniture pieces in 3D laser scans
In this article, we investigate how autonomous robots can exploit the high quality information already available from the WWW concerning 3-D models of office furniture. Apart from the hobbyist effort in Google 3-D Warehouse, many companies providing office furnishings already have the models for considerable portions of the objects found in our workplaces and homes. In particular, we present an approach that allows a robot to learn generic models of typical office furniture using examples found in the Web. These generic models are then used by the robot to locate and categorize unknown furniture in real indoor environments
VQ-NeRF: Neural Reflectance Decomposition and Editing with Vector Quantization
We propose VQ-NeRF, a two-branch neural network model that incorporates
Vector Quantization (VQ) to decompose and edit reflectance fields in 3D scenes.
Conventional neural reflectance fields use only continuous representations to
model 3D scenes, despite the fact that objects are typically composed of
discrete materials in reality. This lack of discretization can result in noisy
material decomposition and complicated material editing. To address these
limitations, our model consists of a continuous branch and a discrete branch.
The continuous branch follows the conventional pipeline to predict decomposed
materials, while the discrete branch uses the VQ mechanism to quantize
continuous materials into individual ones. By discretizing the materials, our
model can reduce noise in the decomposition process and generate a segmentation
map of discrete materials. Specific materials can be easily selected for
further editing by clicking on the corresponding area of the segmentation
outcomes. Additionally, we propose a dropout-based VQ codeword ranking strategy
to predict the number of materials in a scene, which reduces redundancy in the
material segmentation process. To improve usability, we also develop an
interactive interface to further assist material editing. We evaluate our model
on both computer-generated and real-world scenes, demonstrating its superior
performance. To the best of our knowledge, our model is the first to enable
discrete material editing in 3D scenes.Comment: Accepted by TVCG. Project Page:
https://jtbzhl.github.io/VQ-NeRF.github.io
TwinTex: Geometry-aware Texture Generation for Abstracted 3D Architectural Models
Coarse architectural models are often generated at scales ranging from
individual buildings to scenes for downstream applications such as Digital Twin
City, Metaverse, LODs, etc. Such piece-wise planar models can be abstracted as
twins from 3D dense reconstructions. However, these models typically lack
realistic texture relative to the real building or scene, making them
unsuitable for vivid display or direct reference. In this paper, we present
TwinTex, the first automatic texture mapping framework to generate a
photo-realistic texture for a piece-wise planar proxy. Our method addresses
most challenges occurring in such twin texture generation. Specifically, for
each primitive plane, we first select a small set of photos with greedy
heuristics considering photometric quality, perspective quality and facade
texture completeness. Then, different levels of line features (LoLs) are
extracted from the set of selected photos to generate guidance for later steps.
With LoLs, we employ optimization algorithms to align texture with geometry
from local to global. Finally, we fine-tune a diffusion model with a multi-mask
initialization component and a new dataset to inpaint the missing region.
Experimental results on many buildings, indoor scenes and man-made objects of
varying complexity demonstrate the generalization ability of our algorithm. Our
approach surpasses state-of-the-art texture mapping methods in terms of
high-fidelity quality and reaches a human-expert production level with much
less effort. Project page: https://vcc.tech/research/2023/TwinTex.Comment: Accepted to SIGGRAPH ASIA 202
Augmented reality over maps
Dissertação de mestrado integrado em Engenharia InformåticaMaps and Geographic Information System (GIS) play a major role in modern society,
particularly on tourism, navigation and personal guidance. However, providing geographical
information of interest related to individual queries remains a strenuous task. The main
constraints are (1) the several information scales available, (2) the large amount of information
available on each scale, and (3) difficulty in directly infer a meaningful geographical context
from text, pictures, or diagrams that are used by most user-aiding systems. To that extent,
and to overcome the aforementioned difficulties, we develop a solution which allows the
overlap of visual information over the maps being queried â a method commonly referred
to as Augmented Reality (AR).
With that in mind, the object of this dissertation is the research and implementation of a
method for the delivery of visual cartographic information over physical (analogue) and
digital two-dimensional (2D) maps utilizing AR. We review existing state-of-art solutions and
outline their limitations across different use cases. Afterwards, we provide a generic modular
solution for a multitude of real-life applications, to name a few: museums, fairs, expositions,
and public street maps. During the development phase, we take into consideration the
trade-off between speed and accuracy in order to develop an accurate and real-time solution.
Finally, we demonstrate the feasibility of our methods with an application on a real use case
based on a map of the city of Oporto, in Portugal.Mapas e Sistema de Informação Geogråfica (GIS) desempenham um papel importante na
sociedade, particularmente no turismo, navegação e orientação pessoal. No entanto, fornecer
informaçÔes geogråficas de interesse a consultas dos utilizadores é uma tarefa årdua. Os
principais dificuldades sĂŁo (1) as vĂĄrias escalas de informaçÔes disponĂveis, (2) a grande
quantidade de informação disponĂvel em cada escala e (3) dificuldade em inferir diretamente
um contexto geogrĂĄfico significativo a partir dos textos, figuras ou diagramas usados. Assim,
e para superar as dificuldades mencionadas, desenvolvemos uma solução que permite a
sobreposição de informaçÔes visuais sobre os mapas que estão a ser consultados - um
método geralmente conhecido como Realidade Aumentada (AR).
Neste sentido, o objetivo desta dissertação é a pesquisa e implementação de um método para
a visualização de informaçÔes cartogrĂĄficas sobre mapas 2D fĂsicos (analĂłgicos) e digitais
utilizando AR. Em primeiro lugar, analisamos o estado da arte juntamente com as soluçÔes
existentes e tambĂ©m as suas limitaçÔes nas diversas utilizaçÔes possĂveis. Posteriormente,
fornecemos uma solução modular genérica para uma vårias aplicaçÔes reais tais como:
museus, feiras, exposiçÔes e mapas pĂșblicos de ruas. Durante a fase de desenvolvimento,
tivemos em consideração o compromisso entre velocidade e precisão, a fim de desenvolver
uma solução precisa que funciona em tempo real. Por fim, demonstramos a viabilidade de
nossos métodos com uma aplicação num caso de uso real baseado num mapa da cidade do
Porto (Portugal)
Automatic Reconstruction of Parametric, Volumetric Building Models from 3D Point Clouds
Planning, construction, modification, and analysis of buildings requires means of representing a building's physical structure and related semantics in a meaningful way. With the rise of novel technologies and increasing requirements in the architecture, engineering and construction (AEC) domain, two general concepts for representing buildings have gained particular attention in recent years. First, the concept of Building Information Modeling (BIM) is increasingly used as a modern means for representing and managing a building's as-planned state digitally, including not only a geometric model but also various additional semantic properties. Second, point cloud measurements are now widely used for capturing a building's as-built condition by means of laser scanning techniques. A particular challenge and topic of current research are methods for combining the strengths of both point cloud measurements and Building Information Modeling concepts to quickly obtain accurate building models from measured data. In this thesis, we present our recent approaches to tackle the intermeshed challenges of automated indoor point cloud interpretation using targeted segmentation methods, and the automatic reconstruction of high-level, parametric and volumetric building models as the basis for further usage in BIM scenarios. In contrast to most reconstruction methods available at the time, we fundamentally base our approaches on BIM principles and standards, and overcome critical limitations of previous approaches in order to reconstruct globally plausible, volumetric, and parametric models.Automatische Rekonstruktion von parametrischen, volumetrischen GebĂ€udemodellen aus 3D Punktwolken FĂŒr die Planung, Konstruktion, Modifikation und Analyse von GebĂ€uden werden Möglichkeiten zur sinnvollen ReprĂ€sentation der physischen GebĂ€udestruktur sowie dazugehöriger Semantik benötigt. Mit dem Aufkommen neuer Technologien und steigenden Anforderungen im Bereich von Architecture, Engineering and Construction (AEC) haben zwei Konzepte fĂŒr die ReprĂ€sentation von GebĂ€uden in den letzten Jahren besondere Aufmerksamkeit erlangt. Erstens wird das Konzept des Building Information Modeling (BIM) zunehmend als ein modernes Mittel zur digitalen Abbildung und Verwaltung "As-Planned"-Zustands von GebĂ€uden verwendet, welches nicht nur ein geometrisches Modell sondern auch verschiedene zusĂ€tzliche semantische Eigenschaften beinhaltet. Zweitens werden Punktwolkenmessungen inzwischen hĂ€ufig zur Aufnahme des "As-Built"-Zustands mittels Laser-Scan-Techniken eingesetzt. Eine besondere Herausforderung und Thema aktueller Forschung ist die Entwicklung von Methoden zur Vereinigung der StĂ€rken von Punktwolken und Konzepten des Building Information Modeling um schnell akkurate GebĂ€udemodelle aus den gemessenen Daten zu erzeugen. In dieser Dissertation prĂ€sentieren wir unsere aktuellen AnsĂ€tze um die miteinander verwobenen Herausforderungen anzugehen, Punktwolken mithilfe geeigneter Segmentierungsmethoden automatisiert zu interpretieren, sowie hochwertige, parametrische und volumetrische GebĂ€udemodelle als Basis fĂŒr die Verwendung im BIM-Umfeld zu rekonstruieren. Im Gegensatz zu den meisten derzeit verfĂŒgbaren Rekonstruktionsverfahren basieren unsere AnsĂ€tze grundlegend auf Prinzipien und Standards aus dem BIM-Umfeld und ĂŒberwinden kritische EinschrĂ€nkungen bisheriger AnsĂ€tze um vollstĂ€ndig plausible, volumetrische und parametrische Modelle zu erzeugen.</p
GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts
For years, researchers have been devoted to generalizable object perception
and manipulation, where cross-category generalizability is highly desired yet
underexplored. In this work, we propose to learn such cross-category skills via
Generalizable and Actionable Parts (GAParts). By identifying and defining 9
GAPart classes (lids, handles, etc.) in 27 object categories, we construct a
large-scale part-centric interactive dataset, GAPartNet, where we provide rich,
part-level annotations (semantics, poses) for 8,489 part instances on 1,166
objects. Based on GAPartNet, we investigate three cross-category tasks: part
segmentation, part pose estimation, and part-based object manipulation. Given
the significant domain gaps between seen and unseen object categories, we
propose a robust 3D segmentation method from the perspective of domain
generalization by integrating adversarial learning techniques. Our method
outperforms all existing methods by a large margin, no matter on seen or unseen
categories. Furthermore, with part segmentation and pose estimation results, we
leverage the GAPart pose definition to design part-based manipulation
heuristics that can generalize well to unseen object categories in both the
simulator and the real world. Our dataset, code, and demos are available on our
project page.Comment: To appear in CVPR 2023 (Highlight
Unmanned Ground Robots for Rescue Tasks
This chapter describes two unmanned ground vehicles that can help search and rescue teams in their difficult, but life-saving tasks. These robotic assets have been developed within the framework of the European project ICARUS. The large unmanned ground vehicle is intended to be a mobile base station. It is equipped with a powerful manipulator arm and can be used for debris removal, shoring operations, and remote structural operations (cutting, welding, hammering, etc.) on very rough terrain. The smaller unmanned ground vehicle is also equipped with an array of sensors, enabling it to search for victims inside semi-destroyed buildings. Working together with each other and the human search and rescue workers, these robotic assets form a powerful team, increasing the effectiveness of search and rescue operations, as proven by operational validation tests in collaboration with end users
- âŠ