791 research outputs found

    UniNeXt: Exploring A Unified Architecture for Vision Recognition

    Full text link
    Vision Transformers have shown great potential in computer vision tasks. Most recent works have focused on elaborating the spatial token mixer for performance gains. However, we observe that a well-designed general architecture can significantly improve the performance of the entire backbone, regardless of which spatial token mixer is equipped. In this paper, we propose UniNeXt, an improved general architecture for the vision backbone. To verify its effectiveness, we instantiate the spatial token mixer with various typical and modern designs, including both convolution and attention modules. Compared with the architecture in which they are first proposed, our UniNeXt architecture can steadily boost the performance of all the spatial token mixers, and narrows the performance gap among them. Surprisingly, our UniNeXt equipped with naive local window attention even outperforms the previous state-of-the-art. Interestingly, the ranking of these spatial token mixers also changes under our UniNeXt, suggesting that an excellent spatial token mixer may be stifled due to a suboptimal general architecture, which further shows the importance of the study on the general architecture of vision backbone. All models and codes will be publicly available

    AxWin Transformer: A Context-Aware Vision Transformer Backbone with Axial Windows

    Full text link
    Recently Transformer has shown good performance in several vision tasks due to its powerful modeling capabilities. To reduce the quadratic complexity caused by the attention, some outstanding work restricts attention to local regions or extends axial interactions. However, these methos often lack the interaction of local and global information, balancing coarse and fine-grained information. To address this problem, we propose AxWin Attention, which models context information in both local windows and axial views. Based on the AxWin Attention, we develop a context-aware vision transformer backbone, named AxWin Transformer, which outperforming the state-of-the-art methods in both classification and downstream segmentation and detection tasks

    RegionBLIP: A Unified Multi-modal Pre-training Framework for Holistic and Regional Comprehension

    Full text link
    In this work, we investigate extending the comprehension of Multi-modal Large Language Models (MLLMs) to regional objects. To this end, we propose to extract features corresponding to regional objects as soft prompts for LLM, which provides a straightforward and scalable approach and eliminates the need for LLM fine-tuning. To effectively extract regional features from regular image features and irregular point cloud features, we present a novel and unified position-assisted feature extraction module. Furthermore, training an MLLM from scratch is highly time-consuming. Thus, we propose incrementally extending existing pre-trained MLLMs to comprehend more modalities and the regional objects of those modalities. Specifically, we freeze the Q-Former from BLIP-2, an impressive MLLM, and optimize the modality-specific Lora parameters in Q-Former and LLM for each newly introduced modality. The freezing of the Q-Former eliminates the need for extensive pre-training on massive image-text data. The freezed Q-Former pre-trained from massive image-text data is also beneficial for the pre-training on image-region-text data. We name our framework RegionBLIP. We pre-train RegionBLIP on image-region-text, point-cloud-text, and point-cloud-region-text data. Experimental results verify that \Ours{} can preserve the image comprehension capability of BILP-2 and further gain a comprehension of the newly introduced point cloud modality and regional objects. The Data, Code, and Pre-trained models will be available at https://github.com/mightyzau/RegionBLIP

    Data Pruning via Moving-one-Sample-out

    Full text link
    In this paper, we propose a novel data-pruning approach called moving-one-sample-out (MoSo), which aims to identify and remove the least informative samples from the training set. The core insight behind MoSo is to determine the importance of each sample by assessing its impact on the optimal empirical risk. This is achieved by measuring the extent to which the empirical risk changes when a particular sample is excluded from the training set. Instead of using the computationally expensive leaving-one-out-retraining procedure, we propose an efficient first-order approximator that only requires gradient information from different training stages. The key idea behind our approximation is that samples with gradients that are consistently aligned with the average gradient of the training set are more informative and should receive higher scores, which could be intuitively understood as follows: if the gradient from a specific sample is consistent with the average gradient vector, it implies that optimizing the network using the sample will yield a similar effect on all remaining samples. Experimental results demonstrate that MoSo effectively mitigates severe performance degradation at high pruning ratios and achieves satisfactory performance across various settings.Comment: Accepted by the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023

    Genome-scale identification of Caenorhabditis elegans regulatory elements by tiling-array mapping of DNase I hypersensitive sites

    Get PDF
    <p>Abstract</p> <p>Background</p> <p>A major goal of post-genomics research is the integrated analysis of genes, regulatory elements and the chromatin architecture on a genome-wide scale. Mapping DNase I hypersensitive sites within the nuclear chromatin is a powerful and well-established method of identifying regulatory element candidates.</p> <p>Results</p> <p>Here, we report the first genome-wide analysis of DNase I hypersensitive sites (DHSs) in <it>Caenorhabditis elegans</it>. The data was obtained by hybridizing DNase I-treated and end-captured material from young adult worms to a high-resolution tiling microarray. The data show that <it>C. elegans </it>DHSs were significantly enriched within intergenic regions located 2 kb upstream and downstream of coding genes, and also that a considerable fraction of all DHSs mapped to intergenic positions distant to annotated coding genes. Annotated transcribed loci were generally depleted in DHSs relative to intergenic regions, but DHSs were nonetheless enriched in coding exons and UTRs, whereas introns were significantly depleted in DHSs. Many DHSs appeared to be associated with annotated non-coding RNAs and recently detected transcripts of unknown function. It has been reported that nematode highly conserved non-coding elements were associated with cis-regulatory elements, and we also found that DHSs, particularly distal intergenic DHSs, were significantly enriched in regions that were conserved between the <it>C. elegans </it>and <it>C. briggsae </it>genomes.</p> <p>Conclusion</p> <p>We describe the first genome-wide analysis of <it>C. elegans </it>DHSs, and show that the distribution of DHSs is strongly associated with functional elements in the genome.</p

    Robust Super-Resolution Imaging Based on a Ring Core Fiber with Orbital Angular Momentum

    Full text link
    Single fiber imaging technology offers unique insights for research and inspection in difficult to reach and narrow spaces. In particular, ultra-compact multimode fiber (MMF) imaging, has received increasing interest over the past decade. However, MMF imaging will be seriously distorted when subjected to dynamic perturbations due to time-varying mode coupling, and the imaging of space objects via Gaussian beam will be relatively degraded at the edge due to insufficient contrast. Here, a robust super-resolution imaging method based on a ring core fiber (RCF) with orbital angular momentum (OAM) has been proposed and experimentally demonstrated. The OAM modes propagating in the RCF form a series of weakly-coupled mode groups, making our imaging system robust to external perturbations. In addition, a spiral phase plate is used as a vortex filter to produce OAM for edge enhancement, thus improving the image resolution. Furthermore, a few-shot U-Transformer neural network is proposed to enhance the resilience of the developed RCF-OAM imaging system against environmental perturbations. Finally, the developed RCF-OAM imaging system achieves biological image transmission, demonstrating the practicality of our scheme. This pioneering RCF OAM imaging system may have broad applications, potentially revolutionising fields such as biological imaging and industrial non-destructive testing

    Genome-Wide Association Mapping for Cold Tolerance in a Core Collection of Rice (Oryza sativa L.) Landraces by Using High-Density Single Nucleotide Polymorphism Markers From Specific-Locus Amplified Fragment Sequencing

    Get PDF
    Understanding the genetic mechanism of cold tolerance in rice is important to mine elite genes from rice landraces and breed excellent cultivars for this trait. In this study, a genome-wide association study (GWAS) was performed using high-density single nucleotide polymorphisms (SNPs) obtained using specific-locus amplified fragment sequencing (SLAF-seq) technology from a core collection of landraces of rice. A total of 67,511 SNPs obtained from 116,643 SLAF tags were used for genotyping the 150 accessions of rice landraces in the Ting’s rice core collection. A compressed mixed liner model was used to perform GWAS by using the high-density SNPs for cold tolerance in rice landraces at the seedling stage. A total of 26 SNPs were found to be significantly (P &lt; 1.48 × 10-7) associated with cold tolerance, which could explained phenotypic variations ranging from 26 to 33%. Among them, two quantitative trait loci (QTLs) were mapped closely to the previously cloned/mapped genes or QTLs for cold tolerance. A newly identified QTL for cold tolerance in rice was further characterized by sequencing, real time-polymerase chain reaction, and bioinformatics analyses. One candidate gene, i.e., Os01g0620100, showed different gene expression levels between the cold tolerant and sensitive landraces under cold stress. We found the difference of coding amino acid in Os01g0620100 between cold tolerant and sensitive landraces caused by polymorphism within the coding domain sequence. In addition, the prediction of Os01g0620100 protein revealed a WD40 domain that was frequently found in cold tolerant landraces. Therefore, we speculated that Os01g0620100 was highly important for the response to cold stress in rice. These results indicated that rice landraces are important sources for investigating rice cold tolerance, and the mapping results might provide important information to breed cold-tolerant rice cultivars by using marker-assisted selection
    • …
    corecore