Voxel-Based Indoor Reconstruction From HoloLens Triangle Meshes

Current mobile augmented reality devices are often equipped with range sensors. The Microsoft HoloLens for instance is equipped with a Time-Of-Flight (ToF) range camera providing coarse triangle meshes that can be used in custom applications. We suggest to use the triangle meshes for the automatic generation of indoor models that can serve as basis for augmenting their physical counterpart with location-dependent information. In this paper, we present a novel voxel-based approach for automated indoor reconstruction from unstructured three-dimensional geometries like triangle meshes. After an initial voxelization of the input data, rooms are detected in the resulting voxel grid by segmenting connected voxel components of ceiling candidates and extruding them downwards to find floor candidates. Semantic class labels like 'Wall', 'Wall Opening', 'Interior Object' and 'Empty Interior' are then assigned to the room voxels in-between ceiling and floor by a rule-based voxel sweep algorithm. Finally, the geometry of the detected walls and their openings is refined in voxel representation. The proposed approach is not restricted to Manhattan World scenarios and does not rely on room surfaces being planar.Comment: 8 pages, 4 figure

Normal classification of 3D occupancy grids for voxel-based indoor reconstruction from point clouds

In this paper, we present an automated method for classification of binary voxel occupancy grids of discretized indoor mapping data such as point clouds or triangle meshes according to normal vector directions. Filled voxels get assigned normal class labels distinguishing between horizontal and vertical building structures. The horizontal building structures are further differentiated into those with normal directions pointing upwards or downwards with respect to the building interior. The derived normal grids can be deployed in the context of an existing voxel-based indoor reconstruction pipeline, which so far was only applicable to indoor mapping triangle meshes that already contain normal vectors consistently oriented with respect to the building interior. By means of quantitative evaluation against reference data, we demonstrate the performance of the proposed method and its applicability in the context of voxel-based indoor reconstruction from indoor mapping point clouds without normal vectors. The code of our implementation is made available to the public at https://github.com/huepat/voxir

MARKER-BASED LOCALIZATION OF THE MICROSOFT HOLOLENS IN BUILDING MODELS

Mobile augmented reality devices for indoor environments like the Microsoft HoloLens hold potential for the in-situ visualization of building model data. While the HoloLens has sufficient real-time inside-out tracking capacity to provide a spatially correct and stable visualization of virtual content relative to its surroundings, the placement of virtual objects normally has to be done actively by the user. Beyond reliable tracking capacity, augmenting an indoor environment with corresponding building model data requires a one-time localization of the AR platform inside the local coordinate frame of the building model to be visualized. To this aim, we present a simple marker-based localization method for the HoloLens that is sufficient to overlay indoor environments with virtual room-scale model data with a spatial accuracy of few centimeters. Furthermore, an evaluation method suitable for the proposed scenario is presented, that does not rely on the HoloLens “Live Capture” camera which turned out to produce deviant placements of virtual content in relation to the perception of the user wearing the HoloLens device

SELF-LOCALIZATION OF A MULTI-FISHEYE CAMERA BASED AUGMENTED REALITY SYSTEM IN TEXTURELESS 3D BUILDING MODELS

Georeferenced images help planners to compare and document the progress of underground construction sites. As underground positioning can not rely on GPS/GNSS, we introduce a solely vision based localization method, that makes use of a textureless 3D CAD model of the construction site. In our analysis-by-synthesis approach, depth and normal fisheye images are rendered from presampled positions and gradient orientations are extracted to build a high dimensional synthetic feature space. Acquired camera images are then matched to those features by using a robust distance metric and fast nearest neighbor search. In this manner, initial poses can be obtained on a laptop in real-time using concurrent processing and the graphics processing unit

Projector-Based Augmented Reality for Quality Inspection of Scanned Objects

After scanning or reconstructing the geometry of objects, we need to inspect the result of our work. Are there any parts missing? Is every detail covered in the desired quality? We typically do this by looking at the resulting point clouds or meshes of our objects on-screen. What, if we could see the information directly visualized on the object itself? Augmented reality is the generic term for bringing virtual information into our real environment. In our paper, we show how we can project any 3D information like thematic visualizations or specific monitoring information with reference to our object onto the object’s surface itself, thus augmenting it with additional information. For small objects that could for instance be scanned in a laboratory, we propose a low-cost method involving a projector-camera system to solve this task. The user only needs a calibration board with coded fiducial markers to calibrate the system and to estimate the projector’s pose later on for projecting textures with information onto the object’s surface. Changes within the projected 3D information or of the projector’s pose will be applied in real-time. Our results clearly reveal that such a simple setup will deliver a good quality of the augmented information

DETECTION AND EVALUATION OF TOPOLOGICAL CONSISTENCY IN CITYGML DATASETS

The topological consistency of Boundary-Representation models, meaning here that the incidence graph is homeomorphic with the underlying topology of geographical data, is checked for several CityGML datasets, and a first classification of topological inconsistencies is performed. The analysis is carried out on a spatial database system into which the datasets have been imported. It is found that real-world datasets contain many topologically inconsistent pairs of intersecting polygons. Also data satisfying the ISO/OGC standards can still be topologically inconsistent. In the case when the intersection is a point, topological inconsistency occurs because a vertex lies on a line segment. However, the most frequent topological inconsistencies seem to arise when the intersection of two polygons is a line segment. Consequently, topological queries in present CityGML data cannot rely on the incidence graph only, but must always make costly geometric computations if correct results are to be expected

Uncertainty-aware Cross-Entropy for Semantic Segmentation

Deep neural networks have shown exceptional performance in various tasks, but their lack of robustness, reliability, and tendency to be overconfident pose challenges for their deployment in safety-critical applications like autonomous driving. In this regard, quantifying the uncertainty inherent to a model’s prediction is a promising endeavour to address these shortcomings. In this work, we present a novel Uncertainty-aware Cross-Entropy loss (U-CE) that incorporates dynamic predictive uncertainties into the training process by pixel-wise weighting of the well-known cross-entropy loss (CE). Through extensive experimentation, we demonstrate the superiority of U-CE over regular CE training on two benchmark datasets, Cityscapes and ACDC, using two common backbone architectures, ResNet-18 and ResNet-101. With U-CE, we manage to train models that not only improve their segmentation performance but also provide meaningful uncertainties after training. Consequently, we contribute to the development of more robust and reliable segmentation models, ultimately advancing the state-of-the-art in safety-critical applications and beyond

The G1613A Mutation in the HBV Genome Affects HBeAg Expression and Viral Replication through Altered Core Promoter Activity

Infection of hepatitis B virus (HBV) causes acute and chronic hepatitis and is closely associated with the development of cirrhosis and hepatocellular carcinoma (HCC). Previously, we demonstrated that the G1613A mutation in the HBV negative regulatory element (NRE) is a hotspot mutation in HCC patients. In this study, we further investigated the functional consequences of this mutation in the context of the full length HBV genome and its replication. We showed that the G1613A mutation significantly suppresses the secretion of e antigen (HBeAg) and enhances the synthesis of viral DNA, which is in consistence to our clinical result that the G1613A mutation associates with high viral load in chronic HBV carriers. To further investigate the molecular mechanism of the mutation, we performed the electrophoretic mobility shift assay with the recombinant RFX1 protein, a trans-activator that was shown to interact with the NRE of HBV. Intriguingly, RFX1 binds to the G1613A mutant with higher affinity than the wild-type sequence, indicating that the mutation possesses the trans-activating effect to the core promoter via NRE. The trans-activating effect was further validated by the enhancement of the core promoter activity after overexpression of RFX1 in liver cell line. In summary, our results suggest the functional consequences of the hotspot G1613A mutation found in HBV. We also provide a possible molecular mechanism of this hotspot mutation to the increased viral load of HBV carriers, which increases the risk to HCC

The Role of Interferon in Hepatitis B Therapy

Despite the introduction of new nucleos(t)ide analogues in recent years, peginterferon is still recommended as a potential first-line treatment option by current practice guidelines for the management of chronic hepatitis B. Peginterferon offers the advantage of higher sustained off-treatment response rates compared to nucleos(t)ide analogues because of its immunomodulatory effects. Sustained transition to the inactive hepatitis B surface antigen (HBsAg) carrier state can be achieved in about 30% of hepatitis B e antigen (HBeAg)–positive patients and 20% of HBeAg-negative patients. Recent studies have focused on identification of pretreatment and on-treatment factors that allow the selection of patients who are likely to achieve a sustained response to peginterferon therapy in order to avoid the side-effects and costs associated with unnecessary treatment. Future studies need to address whether specific virologic benchmarks can guide individualized decisions concerning therapy continuation and whether peginterferon combined with new potent nucleos(t)ide analogues improves treatment outcomes