Approximated Prompt Tuning for Vision-Language Pre-trained Models

Prompt tuning is a parameter-efficient way to deploy large-scale pre-trained models to downstream tasks by adding task-specific tokens. In terms of vision-language pre-trained (VLP) models, prompt tuning often requires a large number of learnable tokens to bridge the gap between the pre-training and downstream tasks, which greatly exacerbates the already high computational overhead. In this paper, we revisit the principle of prompt tuning for Transformer-based VLP models and reveal that the impact of soft prompt tokens can be actually approximated via independent information diffusion steps, thereby avoiding the expensive global attention modeling and reducing the computational complexity to a large extent. Based on this finding, we propose a novel Approximated Prompt Tuning (APT) approach towards efficient VL transfer learning. To validate APT, we apply it to two representative VLP models, namely ViLT and METER, and conduct extensive experiments on a bunch of downstream tasks. Meanwhile, the generalization of APT is also validated on CLIP for image classification. The experimental results not only show the superior performance gains and computation efficiency of APT against the conventional prompt tuning methods, e.g., +6.6% accuracy and -64.62% additional computation overhead on METER, but also confirm its merits over other parameter-efficient transfer learning approaches

Effect of animal-sourced bioactive peptides on the in vitro development of mouse preantral follicles

Abstract The aim of this study was to investigate the effect of bioactive peptides (BAPT) from animal sources on the development of mouse preantral follicles in vitro. Preantral follicles were isolated and randomly divided into the following groups: an untreated group (control) and three groups supplemented with 20, 40 and 60 μg/mL BAPT, respectively. After establishing the in vitro follicle culture, the gene expression levels and hormone levels were quantified. After in vitro maturation, the developmental rates, reactive oxygen species (ROS) production levels and mitochondrial distributions of MII oocytes were investigated, followed by the analyses of embryonic developmental rates after in vitro fertilization. The results showed that BAPT promoted the growth of mouse preantral follicles. Notably, after 14 d of in vitro culture, the levels of 17 β-estradiol and progesterone were up-regulated with BAPT treatments. Moreover, the expression levels of Oct4, Bmp15, GDF9, FOXO3, Zp3, FOXL2, Inhibin alpha, SOD2, Catalase, GPx and Bcl-2 in the developing follicles were significantly up-regulated after BAPT treatments (P < 0.05), while BAPT significantly inhibited the expression levels of BAX (P < 0.05). Following BAPT treatments, the ROS production levels of MII oocytes were decreased while the mitochondrial distributions were significantly enhanced. Furthermore, increased maturation rates, fertilization and embryonic developmental rates were found in these BAPT-treated groups (P < 0.05). These results demonstrated that BAPT significantly improved the development of preantral follicles in vitro by reducing ROS-dependent cellular damages and by enhancing mitochondrial distributions, thereby promoting the further applications of animal-derived BAPT in biomedical research

Isolation and Characterization of Aphidicolin Derivatives from Tolypocladium inflatum

Inflatin G (1), a new aphidicolin analogue, together with seven known compounds inflatin A (2), inflatin B (3), aphidicolin (4), aphidicolin-17-monoacetate (5), gulypyrone A (6), pyridoxatin rotamers A (7) and B (8), were isolated from the ascomycete fungus Tolypocladium inflatum. Their structures were determined through NMR analyses and the circular dichroism data of the in situ formed [Rh2(OCOCF3)4] complexes. Compounds 1, 4, 5, 7, and 8 showed modest cytotoxicity against four human cancer cell lines A549, CNE1-MP1, A375, and MCF-7

Ameliorative effect of recombinant human lactoferrin on the premature ovarian failure in rats after cyclophosphamide treatments

Abstract This study investigated the effect of recombinant human lactoferrin (rhLF) on the premature ovarian failure (POF) of rats. After cyclophosphamide treatments, the POF rats were divided into the following groups: normal control group (NC), low-dose group (LD), medium-dose group (MD) and high-dose group (HD) of rhLF. After drug administrations, the ovarian indexes and hormonal levels were detected. After follicle number count, the proliferation and apoptosis were analyzed with the expressions of genes related with oogenesis, reactive oxygen species (ROS) production and apoptosis detected, followed by the calculation of oxidative stress and protein expressions. After 4-hydroperoxy cyclophosphamide (4-HC) treatments, the effect of rhLF on the proliferation, ROS production and gene expressions of primary rat granulosa cells (GCs) cultured in vitro were detected. After mating, the fertilities of POF rats were recorded. The result showed that the rhLF administrations up-regulated the ovarian index with the number of developing follicles increased and the decreases of hormonal levels conferred. The Ki-67 intensities of the MD and HD groups were up-regulated with the Tunnel intensities decreased. The rhLF treatments significantly promoted the expression of oogenesis, antioxidant and anti-apoptosis related genes. The expression of Bax and Caspase 3 were decreased with the expression of Bcl-2 up-regulated after rhLF administrations. The in vitro treatments of rhLF effectively conferred the toxicity of 4-HC on primary rat GCs. The fertility assessment showed the rhLF treatments up-regulated the offspring’s’ folliculogenesis, which confirmed the ameliorative role of rhLF on the POF damages via the inhibition of ROS production in GCs

Antitumor Response of Anti-B7-H3 CAR-T Cells with Humanized scFv in Solid Tumors

We prepared a humanized alternative B7-H3 CAR (B7-H3-haCAR-T) targeting B7-H3 which contained the humanized singl-chain varinlde fragment (scFv) from humanized 8H9 Monoclonal Antibody (hu8H9mAb). The antitumor effects of B7-H3-haCAR-T cells were evaluated in B7-H3 overexpressed tumor cells (in vitro) and B7-H3 xenograft models (in vivo). The specific tumour killing ability of B7-H3-haCAR-T in overexpressing B7-H3 tumour cells was verified by cytotoxicity and ELISA tests. In addition, B7-H3 haCAR-T cells were also noted to suppress the tumour growth remarkably well in the xenograft murine models, and the survival time in the haCAR-T cell treatment group was appreciably longer than that in the control group. The specific recognition and highly efficient tumoricidal behavior of B7-H3 haCAR-T provide a basis for future clinical studies with humanized scFv-transduced CAR-T cells targeting solid tumors

Comparative transcriptomics identifies patterns of selection in roses

Abstract Background Roses are important plants for human beings with pivotal economical and biological traits like continuous flowering, flower architecture, color and scent. Due to frequent hybridization and high genome heterozygosity, classification of roses and their relatives remains a big challenge. Results Here, to identify potential markers for phylogenetic reconstruction and to reveal the patterns of natural selection in roses, we generated sets of high quality and comprehensive reference transcriptomes for Rosa chinensis ‘Old Blush’ (OB) and R. wichuriana ‘Basye’s Thornless’ (BT), two species exhibiting contrasted traits of high economical importance. The assembled reference transcriptomes showed transcripts N50 above 2000 bp. Two roses shared about 10,073 transcripts (N50 = 2282 bp), in which a set of 5959 transcripts was conserved within genera of Rosa. Further comparison with species in Rosaceae identified 4447 transcripts being common (Rosaceae-common) in Rosa, Malus, Prunus, Rubus, and Fragaria, while a pool of 164 transcripts being specific for roses (Rosa-specific). Among the Rosaceae-common transcripts, 409 transcripts showed a signature of positive selection and a clustered expression in different tissues. Interestingly, nine of these rapidly evolving genes were related to DNA damage repair and responses to environmental stimulus, a potential associated with genome confliction post hybridization. Coincident with this fast evolution pattern in rose genes, 24 F-box and four TMV resistant proteins were significantly enriched in the Rosa-specific genes. Conclusions We expect that these Rosaceae-common and Rosa-specific transcripts should facilitate the phylogenetic analysis of Rosaceae plants as well as investigations of Rosa-specific biology. The data reported here could provide fundamental genomic tools and knowledge critical for understanding the biology and domestication of roses and for roses breeding