Dense Piecewise Planar RGB-D SLAM for Indoor Environments

The paper exploits weak Manhattan constraints to parse the structure of indoor environments from RGB-D video sequences in an online setting. We extend the previous approach for single view parsing of indoor scenes to video sequences and formulate the problem of recovering the floor plan of the environment as an optimal labeling problem solved using dynamic programming. The temporal continuity is enforced in a recursive setting, where labeling from previous frames is used as a prior term in the objective function. In addition to recovery of piecewise planar weak Manhattan structure of the extended environment, the orthogonality constraints are also exploited by visual odometry and pose graph optimization. This yields reliable estimates in the presence of large motions and absence of distinctive features to track. We evaluate our method on several challenging indoors sequences demonstrating accurate SLAM and dense mapping of low texture environments. On existing TUM benchmark we achieve competitive results with the alternative approaches which fail in our environments.Comment: International Conference on Intelligent Robots and Systems (IROS) 201

Multi-view urban scene reconstruction in non-uniform volume

This paper presents a new fully automatic approach for multi-view urban scene reconstruction. Our algorithm is based on the Manhattan-World assumption, which can provide compact models while preserving fidelity of synthetic architectures. Starting from a dense point cloud, we extract its main axes by global optimization, and construct a nonuniform volume based on them. A graph model is created from volume facets rather than voxels. Appropriate edge weights are defined to ensure the validity and quality of the surface reconstruction. Compared with the common pointcloud- to-model methods, the proposed methodology exploits image information to unveil the real structures of holes in the point cloud. Experiments demonstrate the encouraging performance of the algorithm. © 2013 SPIE

PlaceRaider: Virtual Theft in Physical Spaces with Smartphones

As smartphones become more pervasive, they are increasingly targeted by malware. At the same time, each new generation of smartphone features increasingly powerful onboard sensor suites. A new strain of sensor malware has been developing that leverages these sensors to steal information from the physical environment (e.g., researchers have recently demonstrated how malware can listen for spoken credit card numbers through the microphone, or feel keystroke vibrations using the accelerometer). Yet the possibilities of what malware can see through a camera have been understudied. This paper introduces a novel visual malware called PlaceRaider, which allows remote attackers to engage in remote reconnaissance and what we call virtual theft. Through completely opportunistic use of the camera on the phone and other sensors, PlaceRaider constructs rich, three dimensional models of indoor environments. Remote burglars can thus download the physical space, study the environment carefully, and steal virtual objects from the environment (such as financial documents, information on computer monitors, and personally identifiable information). Through two human subject studies we demonstrate the effectiveness of using mobile devices as powerful surveillance and virtual theft platforms, and we suggest several possible defenses against visual malware

Comparative analysis of technologies and methods for automatic construction of building information models for existing buildings

Building Information Modelling (BIM) provides an intelligent and parametric digital platform to support activities throughout the life-cycle of a building and has been used for new building construction projects nowadays. However, most existing buildings today do not have complete as-built information documents after the construction phase, nor existed meaningful BIM models. Despite the growing use of BIM models and the improvement in as-built records, missing or incomplete building information is still one of the main reasons for the low-level efficiency of building project management. Furthermore, as-built BIM modelling for existing buildings is considered to be a time-consuming process in real projects. Researchers have paid attention to systems and technologies for automated creation of as-built BIM models, but no system has achieved full automation yet. With the ultimate goal of developing a fully automated BIM model creation system, this paper summarises the state-of-the-art techniques and methods for creating as-built BIM models as the starting point, which include data capturing technologies, data processing technologies, object recognition approaches and creating as-built BIM models. Merits and limitations of each technology and method are evaluated based on intensive literature review. This paper also discusses key challenges and gaps remained unaddressed, which are identified through comparative analysis of technologies and methods currently available to support fully automated creation of as-built BIM models.published_or_final_versio

Shape Anchors for Data-Driven Multi-view Reconstruction

We present a data-driven method for building dense 3D reconstructions using a combination of recognition and multi-view cues. Our approach is based on the idea that there are image patches that are so distinctive that we can accurately estimate their latent 3D shapes solely using recognition. We call these patches shape anchors, and we use them as the basis of a multi-view reconstruction system that transfers dense, complex geometry between scenes. We "anchor" our 3D interpretation from these patches, using them to predict geometry for parts of the scene that are relatively ambiguous. The resulting algorithm produces dense reconstructions from stereo point clouds that are sparse and noisy, and we demonstrate it on a challenging dataset of real-world, indoor scenes.American Society for Engineering Education. National Defense Science and Engineering Graduate FellowshipUnited States. Office of Naval Research. Multidisciplinary University Research Initiative (N000141010933)National Science Foundation (U.S.) (Grant CGV-1212928

Towards automatic reconstruction of indoor scenes from incomplete point clouds: door and window detection and regularization

In the last years, point clouds have become the main source of information for building modelling. Although a considerable amount of methodologies addressing the automated generation of 3D models from point clouds have been developed, indoor modelling is still a challenging task due to complex building layouts and the high presence of severe clutters and occlusions. Most of methodologies are highly dependent on data quality, often producing irregular and non-consistent models. Although manmade environments generally exhibit some regularities, they are not commonly considered. This paper presents an optimization-based approach for detecting regularities (i.e., same shape, same alignment and same spacing) in building indoor features. The methodology starts from the detection of openings based on a voxel-based visibility analysis to distinguish ‘occluded’ from ‘empty’ regions in wall surfaces. The extraction of regular patterns in windows is addressed from studying the point cloud from an outdoor perspective. The layout is regularized by minimizing deformations while respecting the detected constraints. The methodology applies for elements placed in the same planeXunta de Galicia | Ref. ED481B 2016/079-

Real-time manhattan world rotation estimation in 3D

Drift of the rotation estimate is a well known problem in visual odometry systems as it is the main source of positioning inaccuracy. We propose three novel algorithms to estimate the full 3D rotation to the surrounding Manhattan World (MW) in as short as 20 ms using surface-normals derived from the depth channel of a RGB-D camera. Importantly, this rotation estimate acts as a structure compass which can be used to estimate the bias of an odometry system, such as an inertial measurement unit (IMU), and thus remove its angular drift. We evaluate the run-time as well as the accuracy of the proposed algorithms on groundtruth data. They achieve zerodrift rotation estimation with RMSEs below 3.4° by themselves and below 2.8° when integrated with an IMU in a standard extended Kalman filter (EKF). Additional qualitative results show the accuracy in a large scale indoor environment as well as the ability to handle fast motion. Selected segmentations of scenes from the NYU depth dataset demonstrate the robustness of the inference algorithms to clutter and hint at the usefulness of the segmentation for further processing.United States. Office of Naval Research. Multidisciplinary University Research Initiative6 (Awards N00014-11-1-0688 and N00014-10-1-0936)National Science Foundation (U.S.) (Award IIS-1318392