155 research outputs found
Beyond Single Instance Multi-view Unsupervised Representation Learning
Recent unsupervised contrastive representation learning follows a Single
Instance Multi-view (SIM) paradigm where positive pairs are usually constructed
with intra-image data augmentation. In this paper, we propose an effective
approach called Beyond Single Instance Multi-view (BSIM). Specifically, we
impose more accurate instance discrimination capability by measuring the joint
similarity between two randomly sampled instances and their mixture, namely
spurious-positive pairs. We believe that learning joint similarity helps to
improve the performance when encoded features are distributed more evenly in
the latent space. We apply it as an orthogonal improvement for unsupervised
contrastive representation learning, including current outstanding methods
SimCLR, MoCo, and BYOL. We evaluate our learned representations on many
downstream benchmarks like linear classification on ImageNet-1k and PASCAL VOC
2007, object detection on MS COCO 2017 and VOC, etc. We obtain substantial
gains with a large margin almost on all these tasks compared with prior arts.Comment: A plug-in approach with minimal modification to existing methods
based on instance discriminatio
ROME: Robustifying Memory-Efficient NAS via Topology Disentanglement and Gradients Accumulation
Single-path based differentiable neural architecture search has great
strengths for its low computational cost and memory-friendly nature. However,
we surprisingly discover that it suffers from severe searching instability
which has been primarily ignored, posing a potential weakness for a wider
application. In this paper, we delve into its performance collapse issue and
propose a new algorithm called RObustifying Memory-Efficient NAS (ROME).
Specifically, 1) for consistent topology in the search and evaluation stage, we
involve separate parameters to disentangle the topology from the operations of
the architecture. In such a way, we can independently sample connections and
operations without interference; 2) to discount sampling unfairness and
variance, we enforce fair sampling for weight update and apply a gradient
accumulation mechanism for architecture parameters. Extensive experiments
demonstrate that our proposed method has strong performance and robustness,
where it mostly achieves state-of-the-art results on a large number of standard
benchmarks.Comment: Observe new collapse in memory efficient NAS and address it using
ROM
Stepped Fault Line Selection Method Based on Spectral Kurtosis and Relative Energy Entropy of Small Current to Ground System
This paper proposes a stepped selection method based on spectral kurtosis relative energy entropy. Firstly, the length and type of window function are set; then when fault occurs, enter step 1: the polarity of first half-wave extremes is analyzed; if the ratios of extremes between neighboring lines are positive, the bus bar is the fault line, else, the SK relative energy entropies are calculated, and then enter step 2: if the obtained entropy multiple is bigger than the threshold or equal to the threshold, the overhead line of max entropy corresponding is the fault line, if not, enter step 3: the line of max entropy corresponding is the fault line. At last, the applicability of the proposed algorithm is presented, and the comparison results are discussed
Involvement of human chorionic gonadotropin in regulating vasculogenic mimicry and hypoxia-inducible factor-1α expression in ovarian cancer cells
SegViT: Semantic Segmentation with Plain Vision Transformers
We explore the capability of plain Vision Transformers (ViTs) for semantic
segmentation and propose the SegVit. Previous ViT-based segmentation networks
usually learn a pixel-level representation from the output of the ViT.
Differently, we make use of the fundamental component -- attention mechanism,
to generate masks for semantic segmentation. Specifically, we propose the
Attention-to-Mask (ATM) module, in which the similarity maps between a set of
learnable class tokens and the spatial feature maps are transferred to the
segmentation masks. Experiments show that our proposed SegVit using the ATM
module outperforms its counterparts using the plain ViT backbone on the ADE20K
dataset and achieves new state-of-the-art performance on COCO-Stuff-10K and
PASCAL-Context datasets. Furthermore, to reduce the computational cost of the
ViT backbone, we propose query-based down-sampling (QD) and query-based
up-sampling (QU) to build a Shrunk structure. With the proposed Shrunk
structure, the model can save up to computations while maintaining
competitive performance.Comment: 9 Pages, NeurIPS 202
- …