Multi-modal Sensor Registration for Vehicle Perception via Deep Neural Networks

The ability to simultaneously leverage multiple modes of sensor information is critical for perception of an automated vehicle's physical surroundings. Spatio-temporal alignment of registration of the incoming information is often a prerequisite to analyzing the fused data. The persistence and reliability of multi-modal registration is therefore the key to the stability of decision support systems ingesting the fused information. LiDAR-video systems like on those many driverless cars are a common example of where keeping the LiDAR and video channels registered to common physical features is important. We develop a deep learning method that takes multiple channels of heterogeneous data, to detect the misalignment of the LiDAR-video inputs. A number of variations were tested on the Ford LiDAR-video driving test data set and will be discussed. To the best of our knowledge the use of multi-modal deep convolutional neural networks for dynamic real-time LiDAR-video registration has not been presented.Comment: 7 pages, double column, IEEE format, accepted at IEEE HPEC 201

Realistic texture synthesis for point-based fruitage phenotype.

Although current 3D scanner technology can acquire textural images from a point model, visible seams in the image, inconvenient data acquisition and occupancy of a large space during use are points of concern for outdoor fruit models. In this paper, an SPSDW (simplification and perception based subdivision followed by down-sampling weighted average) method is proposed to balance memory usage and texture synthesis quality using a crop fruit, such as apples, as a research subject for a point-based fruit model. First, the quadtree method is improved to make splitting more efficient, and a reasonable texton descriptor is defined to promote query efficiency. Then, the color perception feature is extracted from the image for all pixels. Next, an advanced sub-division scheme and down-sampling strategy are designed to optimize memory space. Finally, a weighted oversampling method is proposed for high-quality texture mixing. This experiment demonstrates that the SPSDW method preserves the mixed texture more realistically and smoothly and preserves color memory up to 94%, 84.7% and 85.7% better than the two-dimesional processing, truncating scalar quantitative and color vision model methods, respectively

SURVEY PLANNING FOR DOCUMENTATION OF A MONUMENT FOR THE UNDERSTANDING, PRESERVATION AND RESTORATION

The three-dimensional (3D) preservation and repair of historic sites are increasingly in practice using modelling and digital documentation. This study focuses on replacing conventional techniques of historical documentation by creating a digital documentation procedure employing laser scanning for 3D mapping of a monument located in Prayagraj, India. To quickly record the entire monument structure, four scanning stations were planned, where three for the facades and one for the interior. A 3D structure of the monument and its elements dimension that included structural, architectural, historical, and non-engineering information was the end product. Researchers, architects, and conservationists can use this laser scanning-based technique to analyze data in great detail to identify weaknesses and conservation requirements. In order to preserve the monument's cultural relevance, it can also be used for virtual tours. Digital documentation can also provide an accurate monument record for restoration needs, protecting the monument from human- or natural-caused damage. Overall, 3D Modelling and digital documentation are valuable tools in heritage conservation, providing comprehensive records of heritage sites and aiding in practical conservation and restoration plans while making cultural heritage accessible to a broader societies

A Shadow Based Method for Image to Model Registration

This paper presents a novel method for 2D to 3D texture mapping using shadows as cues. This work is part of a larger set of methods that address the entire 3D modeling pipeline to create geometrically and photometrically accurate models using a variety of data sources. The focus is on building models of large outdoor, urban, historic and archaeological sites. We pose registration of 2D images with the 3D model as an optimization problem that uses knowledge of the Sun’s position to estimate shadows in a scene, and use the shadows produced as a cue to refine the registration parameters. Results are presented for registration where ground truth is known and also for a large scale model consisting of 14 3D scans and 10 images on a large archaeological site in Sicily. I