See the Difference: Direct Pre-Image Reconstruction and Pose Estimation by Differentiating HOG

The Histogram of Oriented Gradient (HOG) descriptor has led to many advances in computer vision over the last decade and is still part of many state of the art approaches. We realize that the associated feature computation is piecewise differentiable and therefore many pipelines which build on HOG can be made differentiable. This lends to advanced introspection as well as opportunities for end-to-end optimization. We present our implementation of $\nabla$HOG based on the auto-differentiation toolbox Chumpy and show applications to pre-image visualization and pose estimation which extends the existing differentiable renderer OpenDR pipeline. Both applications improve on the respective state-of-the-art HOG approaches

Spectral Analysis for Semantic Segmentation with Applications on Feature Truncation and Weak Annotation

We propose spectral analysis to investigate the correlation between the accuracy and the resolution of segmentation maps for semantic segmentation. The current networks predict segmentation maps on the down-sampled grid of images to alleviate the computational cost. Moreover, these networks can be trained by weak annotations that utilize only the coarse contour of segmentation maps. Despite the successful achievement of these works utilizing the low-frequency information of segmentation maps, however, the accuracy of resultant segmentation maps may also be degraded in the regions near object boundaries. It is yet unclear for a theoretical guideline to determine an optimal down-sampled grid to strike the balance between the cost and the accuracy of segmentation. We analyze the objective function (cross-entropy) and network back-propagation process in frequency domain. We discover that cross-entropy and key features of CNN are mainly contributed by the low-frequency components of segmentation maps. This further provides us quantitative results to determine the efficacy of down-sampled grid of segmentation maps. The analysis is then validated on the two applications: the feature truncation method and the block-wise annotation that limit the high-frequency components of the CNN features and annotation, respectively. The results agree with our analysis. Thus the success of the existing work utilizing low-frequency information of segmentation maps now has theoretical foundation.Comment: 21 page