TransTailor: Pruning the Pre-trained Model for Improved Transfer Learning

The increasing of pre-trained models has significantly facilitated the performance on limited data tasks with transfer learning. However, progress on transfer learning mainly focuses on optimizing the weights of pre-trained models, which ignores the structure mismatch between the model and the target task. This paper aims to improve the transfer performance from another angle - in addition to tuning the weights, we tune the structure of pre-trained models, in order to better match the target task. To this end, we propose TransTailor, targeting at pruning the pre-trained model for improved transfer learning. Different from traditional pruning pipelines, we prune and fine-tune the pre-trained model according to the target-aware weight importance, generating an optimal sub-model tailored for a specific target task. In this way, we transfer a more suitable sub-structure that can be applied during fine-tuning to benefit the final performance. Extensive experiments on multiple pre-trained models and datasets demonstrate that TransTailor outperforms the traditional pruning methods and achieves competitive or even better performance than other state-of-the-art transfer learning methods while using a smaller model. Notably, on the Stanford Dogs dataset, TransTailor can achieve 2.7% accuracy improvement over other transfer methods with 20% fewer FLOPs.Comment: This paper has been accepted by AAAI202

Understanding Programs by Exploiting (Fuzzing) Test Cases

Semantic understanding of programs has attracted great attention in the community. Inspired by recent successes of large language models (LLMs) in natural language understanding, tremendous progress has been made by treating programming language as another sort of natural language and training LLMs on corpora of program code. However, programs are essentially different from texts after all, in a sense that they are normally heavily structured and syntax-strict. In particular, programs and their basic units (i.e., functions and subroutines) are designed to demonstrate a variety of behaviors and/or provide possible outputs, given different inputs. The relationship between inputs and possible outputs/behaviors represents the functions/subroutines and profiles the program as a whole. Therefore, we propose to incorporate such a relationship into learning, for achieving a deeper semantic understanding of programs. To obtain inputs that are representative enough to trigger the execution of most part of the code, we resort to fuzz testing and propose fuzz tuning to boost the performance of program understanding and code representation learning, given a pre-trained LLM. The effectiveness of the proposed method is verified on two program understanding tasks including code clone detection and code classification, and it outperforms current state-of-the-arts by large margins. Code is available at https://github.com/rabbitjy/FuzzTuning.Comment: Findings of the Association for Computational Linguistics: ACL 202

Characterization of anti-leukemia components from Indigo naturalis using comprehensive two-dimensional K562/cell membrane chromatography and in silico target identification.

Traditional Chinese Medicine (TCM) has been developed for thousands of years and has formed an integrated theoretical system based on a large amount of clinical practice. However, essential ingredients in TCM herbs have not been fully identified, and their precise mechanisms and targets are not elucidated. In this study, a new strategy combining comprehensive two-dimensional K562/cell membrane chromatographic system and in silico target identification was established to characterize active components from Indigo naturalis, a famous TCM herb that has been widely used for the treatment of leukemia in China, and their targets. Three active components, indirubin, tryptanthrin and isorhamnetin, were successfully characterized and their anti-leukemia effects were validated by cell viability and cell apoptosis assays. Isorhamnetin, with undefined cancer related targets, was selected for in silico target identification. Proto-oncogene tyrosine-protein kinase (Src) was identified as its membrane target and the dissociation constant (Kd) between Src and isorhamnetin was 3.81 μM. Furthermore, anti-leukemia effects of isorhamnetin were mediated by Src through inducing G2/M cell cycle arrest. The results demonstrated that the integrated strategy could efficiently characterize active components in TCM and their targets, which may bring a new light for a better understanding of the complex mechanism of herbal medicines

Detecting single molecules inside a carbon nanotube to control molecular sequences using inertia trapping phenomenon

Here we show the detection of single gas molecules inside a carbon nanotube based on the change in resonance frequency and amplitude associated with the inertia trapping phenomenon. As its direct implication, a method for controlling the sequence of small molecule is then proposed to realize the concept of manoeuvring of matter atom by atom in one dimension. The detection as well as the implication is demonstrated numerically with the molecular dynamics method. It is theoretically assessed that it is possible for a physical model to be fabricated in the very near future

Crimson: A Data Management System to Support Evaluating Phylogenetic Tree Reconstruction Algorithms

Evolutionary and systems biology increasingly rely on the construction of large phylogenetic trees which represent the relationships between species of interest. As the number and size of such trees increases, so does the need for efficient data storage and query capabilities. Although much attention has been focused on XML as a tree data model, phylogenetic trees differ from document-oriented applications in their size and depth, and their need for structure based queries rather than path-based queries. This paper focuses on Crimson, a tree storage system for phylogenetic trees used to evaluate phylogenetic tree reconstruction algorithms within the context of the NSF CIPRes project. A goal of the modeling component of the CIPRes project is to construct a huge simulation tree representing a gold standard of evolutionary history against which phylogenetic tree reconstruction algorithms can be tested. In this demonstration, we highlight our storage and indexing strategies and show how Crimson is used for benchmarking phylogenetic tree reconstruction algorithms. We also show how our design can be used to support more general queries over phylogenetic trees