57 research outputs found

    Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancy

    Full text link
    Current state-of-the-art results in computer vision depend in part on fine-tuning large pre-trained vision models. However, with the exponential growth of model sizes, the conventional full fine-tuning, which needs to store a individual network copy for each tasks, leads to increasingly huge storage and transmission overhead. Adapter-based Parameter-Efficient Tuning (PET) methods address this challenge by tuning lightweight adapters inserted into the frozen pre-trained models. In this paper, we investigate how to make adapters even more efficient, reaching a new minimum size required to store a task-specific fine-tuned network. Inspired by the observation that the parameters of adapters converge at flat local minima, we find that adapters are resistant to noise in parameter space, which means they are also resistant to low numerical precision. To train low-precision adapters, we propose a computational-efficient quantization method which minimizes the quantization error. Through extensive experiments, we find that low-precision adapters exhibit minimal performance degradation, and even 1-bit precision is sufficient for adapters. The experimental results demonstrate that 1-bit adapters outperform all other PET methods on both the VTAB-1K benchmark and few-shot FGVC tasks, while requiring the smallest storage size. Our findings show, for the first time, the significant potential of quantization techniques in PET, providing a general solution to enhance the parameter efficiency of adapter-based PET methods. Code: https://github.com/JieShibo/PETL-ViTComment: Accepted to ICCV 202

    Picomolar Dichotomous Activity of Gnidimacrin Against HIV-1

    Get PDF
    Highly active antiretroviral therapy (HAART) has offered a promising approach for controlling HIV-1 replication in infected individuals. However, with HARRT, HIV-1 is suppressed rather than eradicated due to persistence of HIV-1 in latent viral reservoirs. Thus, purging the virus from latent reservoirs is an important strategy toward eradicating HIV-1 infection. In this study, we discovered that the daphnane diterpene gnidimacrin, which was previously reported to have potent anti-cancer cell activity, activated HIV-1 replication and killed persistently-infected cells at picomolar concentrations. In addition to its potential to purge HIV-1 from latently infected cells, gnidimacrin potently inhibited a panel of HIV-1 R5 virus infection of peripheral blood mononuclear cells (PBMCs) at an average concentration lower than 10 pM. In contrast, gnidimacrin only partially inhibited HIV-1 ×4 virus infection of PBMCs. The strong anti-HIV-1 R5 virus activity of gnidimacrin was correlated with its effect on down-regulation of the HIV-1 coreceptor CCR5. The anti-R5 virus activity of gnidimacrin was completely abrogated by a selective protein kinase C beta inhibitor enzastaurin, which suggests that protein kinase C beta plays a key role in the potent anti-HIV-1 activity of gnidimacrin in PBMCs. In summary, these results suggest that gnidimacrin could activate latent HIV-1, specifically kill HIV-1 persistently infected cells, and inhibit R5 viruses at picomolar concentrations

    Maleic anhydride-modified chicken ovalbumin as an effective and inexpensive anti-HIV microbicide candidate for prevention of HIV sexual transmission

    Get PDF
    <p>Abstract</p> <p>Background</p> <p>Previous studies have shown that 3-hydroxyphthalic anhydride (HP)-modified bovine milk protein, β-lactoglobulin (β-LG), is a promising microbicide candidate. However, concerns regarding the potential risk of prion contamination in bovine products and carcinogenic potential of phthalate derivatives were raised. Here we sought to replace bovine protein with an animal protein of non-bovine origin and substitute HP with another anhydride for the development of anti-HIV microbicide for preventing HIV sexual transmission.</p> <p>Results</p> <p>Maleic anhydride (ML), succinic anhydride (SU) and HP at different conditions and variable pH values were used for modification of proteins. All the anhydrate-modified globulin-like proteins showed potent anti-HIV activity, which is correlated with the percentage of modified lysine and arginine residues in the modified protein. We selected maleic anhydride-modified ovalbumin (ML-OVA) for further study because OVA is easier to obtain than β-LG, and ML is safer than HP. Furthermore, ML-OVA exhibited broad antiviral activities against HIV-1, HIV-2, SHIV and SIV. This modified protein has no or low <it>in vitro </it>cytotoxicity to human T cells and vaginal epithelial cells. It is resistant to trypsin hydrolysis, possibly because the lysine and arginine residues in OVA are modified by ML. Mechanism studies suggest that ML-OVA inhibits HIV-1 entry by targeting gp120 on HIV-1 virions and also the CD4 receptor on the host cells.</p> <p>Conclusion</p> <p>ML-OVA is a potent HIV fusion/entry inhibitor with the potential to be developed as an effective, safe and inexpensive anti-HIV microbicide.</p

    An M2e-based multiple antigenic peptide vaccine protects mice from lethal challenge with divergent H5N1 influenza viruses

    Get PDF
    <p>Abstract</p> <p>Background</p> <p>A growing concern has raised regarding the pandemic potential of the highly pathogenic avian influenza (HPAI) H5N1 viruses. Consequently, there is an urgent need to develop an effective and safe vaccine against the divergent H5N1 influenza viruses. In the present study, we designed a tetra-branched multiple antigenic peptide (MAP)-based vaccine, designated M2e-MAP, which contains the sequence overlapping the highly conserved extracellular domain of matrix protein 2 (M2e) of a HPAI H5N1 virus, and investigated its immune responses and cross-protection against different clades of H5N1 viruses.</p> <p>Results</p> <p>Our results showed that M2e-MAP vaccine induced strong M2e-specific IgG antibody responses following 3-dose immunization of mice with M2e-MAP in the presence of Freunds' or aluminium (alum) adjuvant. M2e-MAP vaccination limited viral replication and attenuated histopathological damage in the challenged mouse lungs. The M2e-MAP-based vaccine protected immunized mice against both clade1: VN/1194 and clade2.3.4: SZ/406H H5N1 virus challenge, being able to counteract weight lost and elevate survival rate following lethal challenge of H5N1 viruses.</p> <p>Conclusions</p> <p>These results suggest that M2e-MAP presenting M2e of H5N1 virus has a great potential to be developed into an effective subunit vaccine for the prevention of infection by a broad spectrum of HPAI H5N1 viruses.</p

    A Recombinant Vaccine of H5N1 HA1 Fused with Foldon and Human IgG Fc Induced Complete Cross-Clade Protection against Divergent H5N1 Viruses

    Get PDF
    Development of effective vaccines to prevent influenza, particularly highly pathogenic avian influenza (HPAI) caused by influenza A virus (IAV) subtype H5N1, is a challenging goal. In this study, we designed and constructed two recombinant influenza vaccine candidates by fusing hemagglutinin 1 (HA1) fragment of A/Anhui/1/2005(H5N1) to either Fc of human IgG (HA1-Fc) or foldon plus Fc (HA1-Fdc), and evaluated their immune responses and cross-protection against divergent strains of H5N1 virus. Results showed that these two recombinant vaccines induced strong immune responses in the vaccinated mice, which specifically reacted with HA1 proteins and an inactivated heterologous H5N1 virus. Both proteins were able to cross-neutralize infections by one homologous strain (clade 2.3) and four heterologous strains belonging to clades 0, 1, and 2.2 of H5N1 pseudoviruses as well as three heterologous strains (clades 0, 1, and 2.3.4) of H5N1 live virus. Importantly, immunization with these two vaccine candidates, especially HA1-Fdc, provided complete cross-clade protection against high-dose lethal challenge of different strains of H5N1 virus covering clade 0, 1, and 2.3.4 in the tested mouse model. This study suggests that the recombinant fusion proteins, particularly HA1-Fdc, could be developed into an efficacious universal H5N1 influenza vaccine, providing cross-protection against infections by divergent strains of highly pathogenic H5N1 virus

    A Search for Light Fermionic Dark Matter Absorption on Electrons in PandaX-4T

    Full text link
    We report a search on a sub-MeV fermionic dark matter absorbed by electrons with an outgoing active neutrino using the 0.63 tonne-year exposure collected by PandaX-4T liquid xenon experiment. No significant signals are observed over the expected background. The data are interpreted into limits to the effective couplings between such dark matter and electrons. For axial-vector or vector interactions, our sensitivity is competitive in comparison to existing astrophysical bounds on the decay of such dark matter into photon final states. In particular, we present the first direct detection limits for an axial-vector (vector) interaction which are the strongest in the mass range from 25 to 45 (35 to 50) keV/c2^2

    FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer

    No full text
    Recent work has explored the potential to adapt a pre-trained vision transformer (ViT) by updating only a few parameters so as to improve storage efficiency, called parameter-efficient transfer learning (PETL). Current PETL methods have shown that by tuning only 0.5% of the parameters, ViT can be adapted to downstream tasks with even better performance than full fine-tuning. In this paper, we aim to further promote the efficiency of PETL to meet the extreme storage constraint in real-world applications. To this end, we propose a tensorization-decomposition framework to store the weight increments, in which the weights of each ViT are tensorized into a single 3D tensor, and their increments are then decomposed into lightweight factors. In the fine-tuning process, only the factors need to be updated and stored, termed Factor-Tuning (FacT). On VTAB-1K benchmark, our method performs on par with NOAH, the state-of-the-art PETL method, while being 5x more parameter-efficient. We also present a tiny version that only uses 8K (0.01% of ViT's parameters) trainable parameters but outperforms full fine-tuning and many other PETL methods such as VPT and BitFit. In few-shot settings, FacT also beats all PETL baselines using the fewest parameters, demonstrating its strong capability in the low-data regime

    Alleviating Representational Shift for Continual Fine-tuning

    Full text link
    We study a practical setting of continual learning: fine-tuning on a pre-trained model continually. Previous work has found that, when training on new tasks, the features (penultimate layer representations) of previous data will change, called representational shift. Besides the shift of features, we reveal that the intermediate layers' representational shift (IRS) also matters since it disrupts batch normalization, which is another crucial cause of catastrophic forgetting. Motivated by this, we propose ConFiT, a fine-tuning method incorporating two components, cross-convolution batch normalization (Xconv BN) and hierarchical fine-tuning. Xconv BN maintains pre-convolution running means instead of post-convolution, and recovers post-convolution ones before testing, which corrects the inaccurate estimates of means under IRS. Hierarchical fine-tuning leverages a multi-stage strategy to fine-tune the pre-trained network, preventing massive changes in Conv layers and thus alleviating IRS. Experimental results on four datasets show that our method remarkably outperforms several state-of-the-art methods with lower storage overhead
    corecore