COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning

Neural network compression empowers the effective yet unwieldy deep convolutional neural networks (CNN) to be deployed in resource-constrained scenarios. Most state-of-the-art approaches prune the model in filter-level according to the "importance" of filters. Despite their success, we notice they suffer from at least two of the following problems: 1) The redundancy among filters is not considered because the importance is evaluated independently. 2) Cross-layer filter comparison is unachievable since the importance is defined locally within each layer. Consequently, we must manually specify layer-wise pruning ratios. 3) They are prone to generate sub-optimal solutions because they neglect the inequality between reducing parameters and reducing computational cost. Reducing the same number of parameters in different positions in the network may reduce different computational cost. To address the above problems, we develop a novel algorithm named as COP (correlation-based pruning), which can detect the redundant filters efficiently. We enable the cross-layer filter comparison through global normalization. We add parameter-quantity and computational-cost regularization terms to the importance, which enables the users to customize the compression according to their preference (smaller or faster). Extensive experiments have shown COP outperforms the others significantly. The code is released at https://github.com/ZJULearning/COP.Comment: 7 pages, 4 figures, has been accepted by IJCAI201

SelFLoc: Selective Feature Fusion for Large-scale Point Cloud-based Place Recognition

Point cloud-based place recognition is crucial for mobile robots and autonomous vehicles, especially when the global positioning sensor is not accessible. LiDAR points are scattered on the surface of objects and buildings, which have strong shape priors along different axes. To enhance message passing along particular axes, Stacked Asymmetric Convolution Block (SACB) is designed, which is one of the main contributions in this paper. Comprehensive experiments demonstrate that asymmetric convolution and its corresponding strategies employed by SACB can contribute to the more effective representation of point cloud feature. On this basis, Selective Feature Fusion Block (SFFB), which is formed by stacking point- and channel-wise gating layers in a predefined sequence, is proposed to selectively boost salient local features in certain key regions, as well as to align the features before fusion phase. SACBs and SFFBs are combined to construct a robust and accurate architecture for point cloud-based place recognition, which is termed SelFLoc. Comparative experimental results show that SelFLoc achieves the state-of-the-art (SOTA) performance on the Oxford and other three in-house benchmarks with an improvement of 1.6 absolute percentages on mean average recall@1

General Rotation Invariance Learning for Point Clouds via Weight-Feature Alignment

Compared to 2D images, 3D point clouds are much more sensitive to rotations. We expect the point features describing certain patterns to keep invariant to the rotation transformation. There are many recent SOTA works dedicated to rotation-invariant learning for 3D point clouds. However, current rotation-invariant methods lack generalizability on the point clouds in the open scenes due to the reliance on the global distribution, \ie the global scene and backgrounds. Considering that the output activation is a function of the pattern and its orientation, we need to eliminate the effect of the orientation.In this paper, inspired by the idea that the network weights can be considered a set of points distributed in the same 3D space as the input points, we propose Weight-Feature Alignment (WFA) to construct a local Invariant Reference Frame (IRF) via aligning the features with the principal axes of the network weights. Our WFA algorithm provides a general solution for the point clouds of all scenes. WFA ensures the model achieves the target that the response activity is a necessary and sufficient condition of the pattern matching degree. Practically, we perform experiments on the point clouds of both single objects and open large-range scenes. The results suggest that our method almost bridges the gap between rotation invariance learning and normal methods.Comment: 4 figure

A Study of Unsupervised Evaluation Metrics for Practical and Automatic Domain Adaptation

Unsupervised domain adaptation (UDA) methods facilitate the transfer of models to target domains without labels. However, these methods necessitate a labeled target validation set for hyper-parameter tuning and model selection. In this paper, we aim to find an evaluation metric capable of assessing the quality of a transferred model without access to target validation labels. We begin with the metric based on mutual information of the model prediction. Through empirical analysis, we identify three prevalent issues with this metric: 1) It does not account for the source structure. 2) It can be easily attacked. 3) It fails to detect negative transfer caused by the over-alignment of source and target features. To address the first two issues, we incorporate source accuracy into the metric and employ a new MLP classifier that is held out during training, significantly improving the result. To tackle the final issue, we integrate this enhanced metric with data augmentation, resulting in a novel unsupervised UDA metric called the Augmentation Consistency Metric (ACM). Additionally, we empirically demonstrate the shortcomings of previous experiment settings and conduct large-scale experiments to validate the effectiveness of our proposed metric. Furthermore, we employ our metric to automatically search for the optimal hyper-parameter set, achieving superior performance compared to manually tuned sets across four common benchmarks. Codes will be available soon

M3^3CS: Multi-Target Masked Point Modeling with Learnable Codebook and Siamese Decoders

Masked point modeling has become a promising scheme of self-supervised pre-training for point clouds. Existing methods reconstruct either the original points or related features as the objective of pre-training. However, considering the diversity of downstream tasks, it is necessary for the model to have both low- and high-level representation modeling capabilities to capture geometric details and semantic contexts during pre-training. To this end, M$^3$CS is proposed to enable the model with the above abilities. Specifically, with masked point cloud as input, M$^3$CS introduces two decoders to predict masked representations and the original points simultaneously. While an extra decoder doubles parameters for the decoding process and may lead to overfitting, we propose siamese decoders to keep the amount of learnable parameters unchanged. Further, we propose an online codebook projecting continuous tokens into discrete ones before reconstructing masked points. In such way, we can enforce the decoder to take effect through the combinations of tokens rather than remembering each token. Comprehensive experiments show that M$^3$CS achieves superior performance at both classification and segmentation tasks, outperforming existing methods

Berberine Inhibits Intestinal Polyps Growth in Apc (min/+) Mice via Regulation of Macrophage Polarization

Antitumor effect of berberine has been reported in a wide spectrum of cancer, however, the mechanisms of which are not fully understood. The aim of this study was to investigate the hypothesis that berberine suppresses tumorigenesis in the familial adenomatous polyposis (FAP) by regulating the macrophage polarization in Apc (min/+) mouse model. Berberine was given to Apc (min/+) mice for 12 weeks. Primary macrophages were isolated; after berberine treatment, the change in signaling cascade was determined. The total number and size of polyps were reduced remarkably in berberine group, compared with control group. A significant decrease in protein levels of F4/80, mannose receptor (MR), and COX-2 in stroma of intestinal polyps and an increase in the level of iNOS were observed after berberine treatment. The mRNA level of MR and Arg-1 in berberine group was significantly lower than those in IL-10 or IL-4 group, while no significant difference in mRNA levels of iNOS and CXCL10 was observed. The migration and invasiveness assays in vitro showed that berberine could reduce the capability of migration and invasiveness. These findings suggest that berberine attenuates intestinal tumorigenesis by inhibiting the migration and invasion of colorectal tumor cells via regulation of macrophage polarization

Estimation and uncertainty analyses of grassland biomass in Northern China: Comparison of multiple remote sensing data sources and modeling approaches

Accurate estimation of grassland biomass and its dynamics are crucial not only for the biogeochemical dynamics of terrestrial ecosystems, but also for the sustainable use of grassland resources. However, estimations of grassland biomass on large spatial scale usually suffer from large variability and mostly lack quantitative uncertainty analyses. In this study, the spatial grassland biomass estimation and its uncertainty were assessed based on 265 field measurements and remote sensing data across Northern China during 2001-2005. Potential sources of uncertainty, including remote sensing data sources (DATsrc), model forms (MODfrm) and model parameters (biomass allocation, BMallo, e.g. root:shoot ratio), were determined and their relative contribution was quantified. The results showed that the annual grassland biomass in Northern China was 1268.37 +/- 180.84Tg (i.e., 532.02 +/- 99.71 g/m(2)) during 2001-2005, increasing from western to eastern area, with a mean relative uncertainty of 19.8%. There were distinguishable differences among the uncertainty contributions of three sources (BMallo >DATsrc>MODfrm), which contributed 52%, 27% and 13%, respectively. This study highlighted the need to concern the uncertainty in grassland biomass estimation, especially for the uncertainty related to BMallo. (C) 2015 Elsevier Ltd. All rights reserved