Applied surveying education : documenting cultural heritage in 3D in the city of Ghent (Belgium) using laser scanning and photo modelling

For several years the city of Ghent (Belgium) and the Ghent University, Department of Geography have been working together to document and measure important cultural heritage sites in 3D. The partnership enables master students in Geomatics and Surveying at the Ghent University to take part in a project driven measuring campaign. During the project, students use and compare several 3D data acquisition methods. This allows the students to implement their theoretical knowledge in the field. The used methods are analysed and critically compared by the students. Through this hands-on-training, students are encouraged to think “outside the box”. When problems occur, they are stimulated to think how these problems could have happened and most importantly how they can solve them. The documentation of these historic monuments in Ghent will be used during future renovation works and archaeological research. This paper will discuss the measurements in the Ghent City Museum (Stadsmuseum or STAM). The following methods are applied during the extensive field work: engineering surveying using total station and GNSS, photo modelling and laser scanning. The deliverables are created in a CAD or GIS environment. After successful completion of the course, students have gained a significant expertise concerning the processing of topographic data, 3D point clouds and imagery in an integrated way. This knowledge can be used after their studies to assess which equipment is most suitable for any given survey project. The final products of the photo modelling and the laser scanning process is a 3D model. Furthermore, digital elevation models and orthorectified images of the historic monument can be created. The orthorectified images are visualised and processed into high resolution orthophoto plans, in a CAD or GIS environment

IM-3D: iterative multiview diffusion and reconstruction for high-quality 3D generation

Most text-to-3D generators build upon off-the-shelf text-to-image models trained on billions of images. They use variants of Score Distillation Sampling (SDS), which is slow, somewhat unstable, and prone to artifacts. A mitigation is to fine-tune the 2D generator to be multi-view aware, which can help distillation or can be combined with reconstruction networks to output 3D objects directly. In this paper, we further explore the design space of text-to-3D models. We significantly improve multi-view generation by considering video instead of image generators. Combined with a 3D reconstruction algorithm which, by using Gaussian splatting, can optimize a robust image-based loss, we directly produce high-quality 3D outputs from the generated views. Our new method, IM-3D, reduces the number of evaluations of the 2D generator network 10-100x, resulting in a much more efficient pipeline, better quality, fewer geometric inconsistencies, and higher yield of usable 3D assets

Computer Vision and Graphics for Heritage Preservation and Digital Archaeology

The goal of this work is to provide attendees with a survey of topics related to Heritage Preservation and Digital Archeology, which are challenging and motivating subjects to both computer vision and graphics community. These issues have been gaining increasing attention and priority within the scientific scenario and among funding agencies and development organizations over the last years. Motivations to this work are the recent efforts in the digital preservation of cultural heritage objects and sites before degradation or damage caused by environmental factors or human development. One of the main focuses of these researches is the development of new techniques for realistic 3D model building from images, preserving as much information as possible. We intend to introduce and discuss several emerging topics in computer vision and graphics related to the proposed theme while highlighting the major contributions and advances in these fields

Holoscopic 3D imaging and display technology: Camera/ processing/ display

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonHoloscopic 3D imaging “Integral imaging” was first proposed by Lippmann in 1908. It has become an attractive technique for creating full colour 3D scene that exists in space. It promotes a single camera aperture for recording spatial information of a real scene and it uses a regularly spaced microlens arrays to simulate the principle of Fly’s eye technique, which creates physical duplicates of light field “true 3D-imaging technique”. While stereoscopic and multiview 3D imaging systems which simulate human eye technique are widely available in the commercial market, holoscopic 3D imaging technology is still in the research phase. The aim of this research is to investigate spatial resolution of holoscopic 3D imaging and display technology, which includes holoscopic 3D camera, processing and display. Smart microlens array architecture is proposed that doubles spatial resolution of holoscopic 3D camera horizontally by trading horizontal and vertical resolutions. In particular, it overcomes unbalanced pixel aspect ratio of unidirectional holoscopic 3D images. In addition, omnidirectional holoscopic 3D computer graphics rendering techniques are proposed that simplify the rendering complexity and facilitate holoscopic 3D content generation. Holoscopic 3D image stitching algorithm is proposed that widens overall viewing angle of holoscopic 3D camera aperture and pre-processing of holoscopic 3D image filters are proposed for spatial data alignment and 3D image data processing. In addition, Dynamic hyperlinker tool is developed that offers interactive holoscopic 3D video content search-ability and browse-ability. Novel pixel mapping techniques are proposed that improves spatial resolution and visual definition in space. For instance, 4D-DSPM enhances 3D pixels per inch from 44 3D-PPIs to 176 3D-PPIs horizontally and achieves spatial resolution of 1365 × 384 3D-Pixels whereas the traditional spatial resolution is 341 × 1536 3D-Pixels. In addition distributed pixel mapping is proposed that improves quality of holoscopic 3D scene in space by creating RGB-colour channel elemental images

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding

Research on depth-based human activity analysis achieved outstanding performance and demonstrated the effectiveness of 3D representation for action recognition. The existing depth-based and RGB+D-based action recognition benchmarks have a number of limitations, including the lack of large-scale training samples, realistic number of distinct class categories, diversity in camera views, varied environmental conditions, and variety of human subjects. In this work, we introduce a large-scale dataset for RGB+D human action recognition, which is collected from 106 distinct subjects and contains more than 114 thousand video samples and 8 million frames. This dataset contains 120 different action classes including daily, mutual, and health-related activities. We evaluate the performance of a series of existing 3D activity analysis methods on this dataset, and show the advantage of applying deep learning methods for 3D-based human action recognition. Furthermore, we investigate a novel one-shot 3D activity recognition problem on our dataset, and a simple yet effective Action-Part Semantic Relevance-aware (APSR) framework is proposed for this task, which yields promising results for recognition of the novel action classes. We believe the introduction of this large-scale dataset will enable the community to apply, adapt, and develop various data-hungry learning techniques for depth-based and RGB+D-based human activity understanding. [The dataset is available at: http://rose1.ntu.edu.sg/Datasets/actionRecognition.asp]Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI