90 research outputs found
Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift
This paper first answers the question "why do the two most powerful
techniques Dropout and Batch Normalization (BN) often lead to a worse
performance when they are combined together?" in both theoretical and
statistical aspects. Theoretically, we find that Dropout would shift the
variance of a specific neural unit when we transfer the state of that network
from train to test. However, BN would maintain its statistical variance, which
is accumulated from the entire learning procedure, in the test phase. The
inconsistency of that variance (we name this scheme as "variance shift") causes
the unstable numerical behavior in inference that leads to more erroneous
predictions finally, when applying Dropout before BN. Thorough experiments on
DenseNet, ResNet, ResNeXt and Wide ResNet confirm our findings. According to
the uncovered mechanism, we next explore several strategies that modifies
Dropout and try to overcome the limitations of their combination by avoiding
the variance shift risks.Comment: 9 pages, 7 figure
SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving
In this paper, we introduce a deep encoder-decoder network, named SalsaNet,
for efficient semantic segmentation of 3D LiDAR point clouds. SalsaNet segments
the road, i.e. drivable free-space, and vehicles in the scene by employing the
Bird-Eye-View (BEV) image projection of the point cloud. To overcome the lack
of annotated point cloud data, in particular for the road segments, we
introduce an auto-labeling process which transfers automatically generated
labels from the camera to LiDAR. We also explore the role of imagelike
projection of LiDAR data in semantic segmentation by comparing BEV with
spherical-front-view projection and show that SalsaNet is projection-agnostic.
We perform quantitative and qualitative evaluations on the KITTI dataset, which
demonstrate that the proposed SalsaNet outperforms other state-of-the-art
semantic segmentation networks in terms of accuracy and computation time. Our
code and data are publicly available at
https://gitlab.com/aksoyeren/salsanet.git
A Framework for Evaluating Land Use and Land Cover Classification Using Convolutional Neural Networks
Analyzing land use and land cover (LULC) using remote sensing (RS) imagery is essential
for many environmental and social applications. The increase in availability of RS data has led to the
development of new techniques for digital pattern classification. Very recently, deep learning (DL)
models have emerged as a powerful solution to approach many machine learning (ML) problems.
In particular, convolutional neural networks (CNNs) are currently the state of the art for many image
classification tasks. While there exist several promising proposals on the application of CNNs to
LULC classification, the validation framework proposed for the comparison of different methods
could be improved with the use of a standard validation procedure for ML based on cross-validation
and its subsequent statistical analysis. In this paper, we propose a general CNN, with a fixed
architecture and parametrization, to achieve high accuracy on LULC classification over RS data
from different sources such as radar and hyperspectral. We also present a methodology to perform
a rigorous experimental comparison between our proposed DL method and other ML algorithms
such as support vector machines, random forests, and k-nearest-neighbors. The analysis carried out
demonstrates that the CNN outperforms the rest of techniques, achieving a high level of performance
for all the datasets studied, regardless of their different characteristics.Ministerio de Economía y Competitividad TIN2014-55894-C2-1-RMinisterio de Economía y Competitividad TIN2017-88209-C2-2-
- …