470 research outputs found
CNN for IMU Assisted Odometry Estimation using Velodyne LiDAR
We introduce a novel method for odometry estimation using convolutional
neural networks from 3D LiDAR scans. The original sparse data are encoded into
2D matrices for the training of proposed networks and for the prediction. Our
networks show significantly better precision in the estimation of translational
motion parameters comparing with state of the art method LOAM, while achieving
real-time performance. Together with IMU support, high quality odometry
estimation and LiDAR data registration is realized. Moreover, we propose
alternative CNNs trained for the prediction of rotational motion parameters
while achieving results also comparable with state of the art. The proposed
method can replace wheel encoders in odometry estimation or supplement missing
GPS data, when the GNSS signal absents (e.g. during the indoor mapping). Our
solution brings real-time performance and precision which are useful to provide
online preview of the mapping results and verification of the map completeness
in real time
A Joint 3D-2D based Method for Free Space Detection on Roads
In this paper, we address the problem of road segmentation and free space
detection in the context of autonomous driving. Traditional methods either use
3-dimensional (3D) cues such as point clouds obtained from LIDAR, RADAR or
stereo cameras or 2-dimensional (2D) cues such as lane markings, road
boundaries and object detection. Typical 3D point clouds do not have enough
resolution to detect fine differences in heights such as between road and
pavement. Image based 2D cues fail when encountering uneven road textures such
as due to shadows, potholes, lane markings or road restoration. We propose a
novel free road space detection technique combining both 2D and 3D cues. In
particular, we use CNN based road segmentation from 2D images and plane/box
fitting on sparse depth data obtained from SLAM as priors to formulate an
energy minimization using conditional random field (CRF), for road pixels
classification. While the CNN learns the road texture and is unaffected by
depth boundaries, the 3D information helps in overcoming texture based
classification failures. Finally, we use the obtained road segmentation with
the 3D depth data from monocular SLAM to detect the free space for the
navigation purposes. Our experiments on KITTI odometry dataset, Camvid dataset,
as well as videos captured by us, validate the superiority of the proposed
approach over the state of the art.Comment: Accepted for publication at IEEE WACV 201
3D objects and scenes classification, recognition, segmentation, and reconstruction using 3D point cloud data: A review
Three-dimensional (3D) point cloud analysis has become one of the attractive
subjects in realistic imaging and machine visions due to its simplicity,
flexibility and powerful capacity of visualization. Actually, the
representation of scenes and buildings using 3D shapes and formats leveraged
many applications among which automatic driving, scenes and objects
reconstruction, etc. Nevertheless, working with this emerging type of data has
been a challenging task for objects representation, scenes recognition,
segmentation, and reconstruction. In this regard, a significant effort has
recently been devoted to developing novel strategies, using different
techniques such as deep learning models. To that end, we present in this paper
a comprehensive review of existing tasks on 3D point cloud: a well-defined
taxonomy of existing techniques is performed based on the nature of the adopted
algorithms, application scenarios, and main objectives. Various tasks performed
on 3D point could data are investigated, including objects and scenes
detection, recognition, segmentation and reconstruction. In addition, we
introduce a list of used datasets, we discuss respective evaluation metrics and
we compare the performance of existing solutions to better inform the
state-of-the-art and identify their limitations and strengths. Lastly, we
elaborate on current challenges facing the subject of technology and future
trends attracting considerable interest, which could be a starting point for
upcoming research studie
Development of Mining Sector Applications for Emerging Remote Sensing and Deep Learning Technologies
This thesis uses neural networks and deep learning to address practical, real-world problems in the mining sector. The main focus is on developing novel applications in the area of object detection from remotely sensed data. This area has many potential mining applications and is an important part of moving towards data driven strategic decision making across the mining sector. The scientific contributions of this research are twofold; firstly, each of the three case studies demonstrate new applications which couple remote sensing and neural network based technologies for improved data driven decision making. Secondly, the thesis presents a framework to guide implementation of these technologies in the mining sector, providing a guide for researchers and professionals undertaking further studies of this type. The first case study builds a fully connected neural network method to locate supporting rock bolts from 3D laser scan data. This method combines input features from the remote sensing and mobile robotics research communities, generating accuracy scores up to 22% higher than those found using either feature set in isolation. The neural network approach also is compared to the widely used random forest classifier and is shown to outperform this classifier on the test datasets. Additionally, the algorithms’ performance is enhanced by adding a confusion class to the training data and by grouping the output predictions using density based spatial clustering. The method is tested on two datasets, gathered using different laser scanners, in different types of underground mines which have different rock bolting patterns. In both cases the method is found to be highly capable of detecting the rock bolts with recall scores of 0.87-0.96. The second case study investigates modern deep learning for LiDAR data. Here, multiple transfer learning strategies and LiDAR data representations are examined for the task of identifying historic mining remains. A transfer learning approach based on a Lunar crater detection model is used, due to the task similarities between both the underlying data structures and the geometries of the objects to be detected. The relationship between dataset resolution and detection accuracy is also examined, with the results showing that the approach is capable of detecting pits and shafts to a high degree of accuracy with precision and recall scores between 0.80-0.92, provided the input data is of sufficient quality and resolution. Alongside resolution, different LiDAR data representations are explored, showing that the precision-recall balance varies depending on the input LiDAR data representation. The third case study creates a deep convolutional neural network model to detect artisanal scale mining from multispectral satellite data. This model is trained from initialisation without transfer learning and demonstrates that accurate multispectral models can be built from a smaller training dataset when appropriate design and data augmentation strategies are adopted. Alongside the deep learning model, novel mosaicing algorithms are developed both to improve cloud cover penetration and to decrease noise in the final prediction maps. When applied to the study area, the results from this model provide valuable information about the expansion, migration and forest encroachment of artisanal scale mining in southwestern Ghana over the last four years. Finally, this thesis presents an implementation framework for these neural network based object detection models, to generalise the findings from this research to new mining sector deep learning tasks. This framework can be used to identify applications which would benefit from neural network approaches; to build the models; and to apply these algorithms in a real world environment. The case study chapters confirm that the neural network models are capable of interpreting remotely sensed data to a high degree of accuracy on real world mining problems, while the framework guides the development of new models to solve a wide range of related challenges
MLF-DET: Multi-Level Fusion for Cross-Modal 3D Object Detection
In this paper, we propose a novel and effective Multi-Level Fusion network,
named as MLF-DET, for high-performance cross-modal 3D object DETection, which
integrates both the feature-level fusion and decision-level fusion to fully
utilize the information in the image. For the feature-level fusion, we present
the Multi-scale Voxel Image fusion (MVI) module, which densely aligns
multi-scale voxel features with image features. For the decision-level fusion,
we propose the lightweight Feature-cued Confidence Rectification (FCR) module
which further exploits image semantics to rectify the confidence of detection
candidates. Besides, we design an effective data augmentation strategy termed
Occlusion-aware GT Sampling (OGS) to reserve more sampled objects in the
training scenes, so as to reduce overfitting. Extensive experiments on the
KITTI dataset demonstrate the effectiveness of our method. Notably, on the
extremely competitive KITTI car 3D object detection benchmark, our method
reaches 82.89% moderate AP and achieves state-of-the-art performance without
bells and whistles
- …