Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers

Although vision transformers (ViTs) have shown promising results in various computer vision tasks recently, their high computational cost limits their practical applications. Previous approaches that prune redundant tokens have demonstrated a good trade-off between performance and computation costs. Nevertheless, errors caused by pruning strategies can lead to significant information loss. Our quantitative experiments reveal that the impact of pruned tokens on performance should be noticeable. To address this issue, we propose a novel joint Token Pruning & Squeezing module (TPS) for compressing vision transformers with higher efficiency. Firstly, TPS adopts pruning to get the reserved and pruned subsets. Secondly, TPS squeezes the information of pruned tokens into partial reserved tokens via the unidirectional nearest-neighbor matching and similarity-based fusing steps. Compared to state-of-the-art methods, our approach outperforms them under all token pruning intensities. Especially while shrinking DeiT-tiny&small computational budgets to 35%, it improves the accuracy by 1%-6% compared with baselines on ImageNet classification. The proposed method can accelerate the throughput of DeiT-small beyond DeiT-tiny, while its accuracy surpasses DeiT-tiny by 4.78%. Experiments on various transformers demonstrate the effectiveness of our method, while analysis experiments prove our higher robustness to the errors of the token pruning policy. Code is available at https://github.com/megvii-research/TPS-CVPR2023.Comment: Accepted to CVPR202

Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification

RGB-Infrared (IR) person re-identification is very challenging due to the large cross-modality variations between RGB and IR images. The key solution is to learn aligned features to the bridge RGB and IR modalities. However, due to the lack of correspondence labels between every pair of RGB and IR images, most methods try to alleviate the variations with set-level alignment by reducing the distance between the entire RGB and IR sets. However, this set-level alignment may lead to misalignment of some instances, which limits the performance for RGB-IR Re-ID. Different from existing methods, in this paper, we propose to generate cross-modality paired-images and perform both global set-level and fine-grained instance-level alignments. Our proposed method enjoys several merits. First, our method can perform set-level alignment by disentangling modality-specific and modality-invariant features. Compared with conventional methods, ours can explicitly remove the modality-specific features and the modality variation can be better reduced. Second, given cross-modality unpaired-images of a person, our method can generate cross-modality paired images from exchanged images. With them, we can directly perform instance-level alignment by minimizing distances of every pair of images. Extensive experimental results on two standard benchmarks demonstrate that the proposed model favourably against state-of-the-art methods. Especially, on SYSU-MM01 dataset, our model can achieve a gain of 9.2% and 7.7% in terms of Rank-1 and mAP. Code is available at https://github.com/wangguanan/JSIA-ReID.Comment: accepted by AAAI'2

Precise and Rapid Validation of Candidate Gene by Allele Specific Knockout With CRISPR/Cas9 in Wild Mice

It is a tempting goal to identify causative genes underlying phenotypic differences among inbred strains of mice, which is a huge reservoir of genetic resources to understand mammalian pathophysiology. In particular, the wild-derived mouse strains harbor enormous genetic variations that have been acquired during evolutionary divergence over 100s of 1000s of years. However, validating the genetic variation in non-classical strains was extremely difficult, until the advent of CRISPR/Cas9 genome editing tools. In this study, we first describe a T cell phenotype in both wild-derived PWD/PhJ parental mice and F1 hybrids, from a cross to C57BL/6 (B6) mice, and we isolate a genetic locus on Chr2, using linkage mapping and chromosome substitution mice. Importantly, we validate the identification of the functional gene controlling this T cell phenotype, Cd44, by allele specific knockout of the PWD copy, leaving the B6 copy completely intact. Our experiments using F1 mice with a dominant phenotype, allowed rapid validation of candidate genes by designing sgRNA PAM sequences that only target the DNA of the PWD genome. We obtained 10 animals derived from B6 eggs fertilized with PWD sperm cells which were subjected to microinjection of CRISPR/Cas9 gene targeting machinery. In the newborns of F1 hybrids, 80% (n = 10) had allele specific knockout of the candidate gene Cd44 of PWD origin, and no mice showed mistargeting of the B6 copy. In the resultant allele-specific knockout F1 mice, we observe full recovery of T cell phenotype. Therefore, our study provided a precise and rapid approach to functionally validate genes that could facilitate gene discovery in classic mouse genetics. More importantly, as we succeeded in genetic manipulation of mice, allele specific knockout could provide the possibility to inactivate disease alleles while keeping the normal allele of the gene intact in human cells

A coupled filtering method to solve feature-motion decorrelation in speckle tracking

In speckle tracking, feature-motion decorrelation is a problem that makes feature tracking results unable to represent the underlying tissue deformation. In this paper, we propose a coupled filtering method to address the feature-motion decorrelation problem. After explicitly modeling image variations caused by tissue deformation, we filter the image before tissue deformation and the warped image after tissue deformation with a pair of filters, respectively. Theoretical derivation show that the two filtered images are identical to each other. Since the coupled filtering method keeps tissue deformation parameters, feature-based tracking is able to accurately estimate the parameters of underlying tissue deformation. Experiments in elastography studies show that our method is superior over previous methods. ©2010 IEEE

Managing urban energy system: A case of Suzhou in China

Managing urban energy system is vital for energy conservation and CO2 reduction. Integrating energy input-output model with carbon emission pinch analysis, we propose a framework for managing urban energy system. This framework could analyze current energy demands and CO2 emissions, predict their future possibilities and optimize energy mix of key sectors under CO2 emission constraints. Key sectors are identified by the energy input-output table from both direct and accumulative perspectives. Moreover, taking Suzhou, a typical manufacturing center and export-oriented city in China, as a case example, energy metabolism of Suzhou in 2020 is predicted using energy input-output model. And three sectors named Coking, Smelting and pressing of metals and Production and supply of electric power are identified to have big effects on CO2 emissions. Subsequently, energy mix of three identified key sectors is optimized under CO2 emission constraints by the carbon emission pinch analysis. According to the results, clean energy sources will occupy a great position in Suzhou's future energy demands. And the reuse of wastes as energy sources should be limited to achieve CO2 mitigation targets. Finally, policy implications of results and future work are discussed.Carbon emission pinch analysis Input-output model Energy mix

Community Detection Through Optimal Density Contrast of Adjacency Matrix

Detecting communities in real world networks is an important problem for data analysis in science and engineering. By clustering nodes intelligently, a recursive algorithm is designed to detect community. Since the relabeling of nodes does not alter the topology of the network, the problem of community detection corresponds to the finding of a good labeling of nodes so that the adjacency matrix form blocks. By putting a fictitious interaction between nodes, the relabeling problem becomes one of energy minimization, where the total energy of the network is defined by putting interaction between the labels of nodes so that clustering nodes that arc in the same community will decrease the total energy. A greedy method is used for the computation of minimum energy. The method shows efficient detection of community in artificial as well as real world network. The result is illustrated in a tree showing hierarchical structure of communities on the basis of sub-matrix density. Applications of the method to weighted and directed networks are discussed

Accurate segmentation of ultrasound images using the motion cue

Accurate segmentation of ultrasound images is desired to measure tissue shapes and sizes. But in some cases, it is difficult to segment ultrasound images purely based on static intensity values. In this paper, we propose a novel segmentation framework by using the motion cue. We integrate a new speckle tracking algorithm into a level set framework to iteratively refine the segmentation and motion estimation at the boundary regions of two different tissues. The balance between local motion contrast of different tissue regions and the smoothness constraint will drive the active contour to the boundaries of different tissues. Experiments on both simulated data and phantom data show that our method has achieved better results than previous methods. ©2010 IEEE

Community Detection using Intelligent Clustering Technique and Sub-matrix Density Ordering

Detecting communities in real world networks is an important problem for data analysis in science and engineering. By clustering nodes intelligently, a recursive algorithm is designed to detect community. Since the relabeling of nodes does not alter the topology of the network, the problem of community detection corresponds to the finding of a good labelling of nodes so that the adjacency matrix form blocks. By putting a fictitious interaction between nodes, the relabeling problem becomes one of energy minimization, where the total energy of the network is defined by putting interaction between the labels of nodes so that the clustering of nodes in the same community will decrease the total energy. A greedy algorithm is used for the computation of minimum energy. The method shows efficient detection of community in artifical as well as real world network. The result is illustrated in a tree showing hierarchical structure of communities on the basis of sub-matrix density

Rozpowszechnianie poza granicami kraju starożytnej chińskiej kultury kostiumowej z perspektywy zaufania kulturowego

Since the reform and opening-up, China has actively followed overseas costume trends and bidden farewell to the era of uniforms. It has gone through evolution from collective imitation to physical liberation to the advocacy of diversified individuality to this new age, where the Chinese culture of costume is radiant with self-confidence. The culture of ancient Chinese costume has a long history with extensive and profound connotations. This paper, from the perspective of cultural confidence, adopts the propagation mode of '5W' to discuss the overseas dissemination of ancient Chinese costume culture, the specific content of Chinese-foreign exchange, and theoretical approaches in the theoretical framework of communication based on the case of 'American Tour of Innovative Design Works of Ancient Chinese Costume Culture', organised by Jiangnan University in 2018, in a bid to serve as a reference for the narration of the national story, the communication of a national voice and the creation of a national image.Od czasu reformy i otwarcia Chiny aktywnie podążały za zagranicznymi trendami kostiumowymi i pożegnały się z erą uniformów. Chiny przeszły ewolucję od zbiorowego naśladowania, przez wyzwolenie i popieranie zróżnicowanej indywidualności, aż do tej nowej ery, w której chińska kultura kostiumowa promieniuje pewnością siebie. Kultura starożytnych chińskich strojów ma długą historię z rozległymi i głębokimi konotacjami. W pracy, z punktu widzenia zaufania kulturowego, przyjęto tryb propagacji "5W". W artykule omówiono proces zagranicznego rozpowszechniania starożytnej chińskiej kultury kostiumowej, specyficznej treści chińsko-zagranicznej wymiany oraz teoretycznych podejść w ramach komunikacji w oparciu o wystawę "American Tour of Innovative Design Works of Ancient Chinese Costume Culture", zorganizowaną przez Uniwersytet Jiangnan w 2018 roku. Wydarzenie to posłużyło, jako punkt odniesienia dla narracji narodowej historii, przekazywania głosu narodowego i tworzenia narodowego wizerunku