Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval

In text-video retrieval, recent works have benefited from the powerful learning capabilities of pre-trained text-image foundation models (e.g., CLIP) by adapting them to the video domain. A critical problem for them is how to effectively capture the rich semantics inside the video using the image encoder of CLIP. To tackle this, state-of-the-art methods adopt complex cross-modal modeling techniques to fuse the text information into video frame representations, which, however, incurs severe efficiency issues in large-scale retrieval systems as the video representations must be recomputed online for every text query. In this paper, we discard this problematic cross-modal fusion process and aim to learn semantically-enhanced representations purely from the video, so that the video representations can be computed offline and reused for different texts. Concretely, we first introduce a spatial-temporal "Prompt Cube" into the CLIP image encoder and iteratively switch it within the encoder layers to efficiently incorporate the global video semantics into frame representations. We then propose to apply an auxiliary video captioning objective to train the frame representations, which facilitates the learning of detailed video semantics by providing fine-grained guidance in the semantic space. With a naive temporal fusion strategy (i.e., mean-pooling) on the enhanced frame representations, we obtain state-of-the-art performances on three benchmark datasets, i.e., MSR-VTT, MSVD, and LSMDC.Comment: to be appeared in ICCV202

Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models

Conditioned diffusion models have demonstrated state-of-the-art text-to-image synthesis capacity. Recently, most works focus on synthesizing independent images; While for real-world applications, it is common and necessary to generate a series of coherent images for story-stelling. In this work, we mainly focus on story visualization and continuation tasks and propose AR-LDM, a latent diffusion model auto-regressively conditioned on history captions and generated images. Moreover, AR-LDM can generalize to new characters through adaptation. To our best knowledge, this is the first work successfully leveraging diffusion models for coherent visual story synthesizing. Quantitative results show that AR-LDM achieves SoTA FID scores on PororoSV, FlintstonesSV, and the newly introduced challenging dataset VIST containing natural images. Large-scale human evaluations show that AR-LDM has superior performance in terms of quality, relevance, and consistency.Comment: Technical Repor

Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs

Large-scale knowledge graphs (KGs) are shown to become more important in current information systems. To expand the coverage of KGs, previous studies on knowledge graph completion need to collect adequate training instances for newly-added relations. In this paper, we consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions and hence recognize the facts of unseen relations with no examples being seen. For this purpose, we leverage Generative Adversarial Networks (GANs) to establish the connection between text and knowledge graph domain: The generator learns to generate the reasonable relation embeddings merely with noisy text descriptions. Under this setting, zero-shot learning is naturally converted to a traditional supervised classification task. Empirically, our method is model-agnostic that could be potentially applied to any version of KG embeddings, and consistently yields performance improvements on NELL and Wiki dataset

AtomNet-Aided OTUD7B Inhibitor Discovery and Validation

Protein deubiquitinases play critical pathophysiological roles in cancer. Among all deubiquitinases, an oncogenic function for OTUD7B has been established in genetic NSCLC murine models. However, few deubiquitinase inhibitors have been developed due to technical challenges. Here, we report a putative small molecule OTUD7B inhibitor obtained from an AI-aided screen of a 4 million compound library. We validated the effects of the OTUD7B inhibitor (7Bi) in reducing Akt-pS473 signals in multiple NSCLC and HEK293 cells by blocking OTUD7B-governed GβL deubiquitination in cells, as well as inhibiting OTUD7B-mediated cleavage of K11-linked di-ub in an in vitro enzyme assay. Furthermore, we report in leukemia cells, either genetic depletion or 7Bi-mediated pharmacological inhibition of OTUD7B reduces Akt-pS473 via inhibiting the OTUD7B/GβL signaling axis. Together, our study identifies the first putative OTUD7B inhibitor showing activities both in cells and in vitro, with promising applications as a therapeutic agent in treating cancer with OTUD7B overexpression

Differential sensitivity of target genes to translational repression by miR-17~92

MicroRNAs (miRNAs) are thought to exert their functions by modulating the expression of hundreds of target genes and each to a small degree, but it remains unclear how small changes in hundreds of target genes are translated into the specific function of a miRNA. Here, we conducted an integrated analysis of transcriptome and translatome of primary B cells from mutant mice expressing miR-17~92 at three different levels to address this issue. We found that target genes exhibit differential sensitivity to miRNA suppression and that only a small fraction of target genes are actually suppressed by a given concentration of miRNA under physiological conditions. Transgenic expression and deletion of the same miRNA gene regulate largely distinct sets of target genes. miR-17~92 controls target gene expression mainly through translational repression and 5’UTR plays an important role in regulating target gene sensitivity to miRNA suppression. These findings provide molecular insights into a model in which miRNAs exert their specific functions through a small number of key target genesCX is a Pew Scholar in Biomedical Sciences. This study is supported by the PEW Charitable Trusts, Cancer Research Institute, National Institute of Health (R01AI087634, R01AI089854, RC1CA146299, R56AI110403, and R01AI121155 to CX), National Natural Science Foundation of China (31570882 to WHL, 31570883 to NX, 31570911 to GF, 91429301 to JH, 31671428 and 31500665 to YZ), 1000 Young Talents Program of China (K08008 to NX), 100 Talents Program of The Chinese Academy of Sciences (YZ), National Program on Key Basic Research Project of China (2016YFA0501900 to YZ), the Fundamental Research Funds for the Central Universities of China (20720150065 to NX and GF), Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2015R1C1A1A01052387 to SGK, NRF-2016R1A4A1010115 to SGK and PHK), and 2016 Research Grant from Kangwon National University (SGK)

Metabolic Profiles and cDNA-AFLP Analysis of Salvia miltiorrhiza and Salvia castanea Diel f. tomentosa Stib

Plants of the genus Salvia produce various types of phenolic compounds and tanshinones which are effective for treatment of coronary heart disease. Salvia miltiorrhiza and S. castanea Diels f. tomentosa Stib are two important members of the genus. In this study, metabolic profiles and cDNA-AFLP analysis of four samples were employed to identify novel genes potentially involved in phenolic compounds and tanshinones biosynthesis, including the red roots from the two species and two tanshinone-free roots from S. miltiorrhiza. The results showed that the red roots of S. castanea Diels f. tomentosa Stib produced high contents of rosmarinic acid (21.77 mg/g) and tanshinone IIA (12.60 mg/g), but low content of salvianolic acid B (1.45 mg/g). The red roots of S. miltiorrhiza produced high content of salvianolic acid B (18.69 mg/g), while tanshinones accumulation in this sample was much less than that in S. castanea Diels f. tomentosa Stib. Tanshinones were not detected in the two tanshinone-free samples, which produced high contents of phenolic compounds. A cDNA-AFLP analysis with 128 primer pairs revealed that 2300 transcript derived fragments (TDFs) were differentially expressed among the four samples. About 323 TDFs were sequenced, of which 78 TDFs were annotated with known functions through BLASTX searching the Genbank database and 14 annotated TDFs were assigned into secondary metabolic pathways through searching the KEGGPATHWAY database. The quantitative real-time PCR analysis indicated that the expression of 9 TDFs was positively correlated with accumulation of phenolic compounds and tanshinones. These TDFs additionally showed coordinated transcriptional response with 6 previously-identified genes involved in biosynthesis of tanshinones and phenolic compounds in S. miltiorrhiza hairy roots treated with yeast extract. The sequence data in the present work not only provided us candidate genes involved in phenolic compounds and tanshinones biosynthesis but also gave us further insight into secondary metabolism in Salvia

Prostate cancer-associated SPOP mutations confer resistance to BET inhibitors through stabilization of BRD4

The bromodomain and extra-terminal (BET) family of proteins, comprised of four members including BRD2, BRD3, BRD4 and the testis-specific isoform BRDT, largely function as transcriptional co-activators 1–3 and play critical roles in various cellular processes, including cell cycle, apoptosis, migration and invasion 4,5. As such, BET proteins enhance the oncogenic functions of major cancer drivers by either elevating their expression such as c-Myc in leukemia 6,7 or by promoting transcriptional activities of oncogenic factors such as AR and ERG in the prostate cancer setting 8. Pathologically, BET proteins are frequently overexpressed and clinically linked to various types of human cancers 5,9,10, therefore pursued as attractive therapeutic targets for selective inhibition in patients. To this end, a number of bromodomain inhibitors, including JQ1 and I-BET, have been developed 11,12 and shown promising outcomes in early clinical trials. Despite resistance to BET inhibitor has been documented in pre-clinical models 13–15 the molecular mechanisms underlying acquired resistance are largely unknown. Here, we report that Cullin 3SPOP earmarks BET proteins including BRD2, BRD3 and BRD4 for ubiquitination-mediated degradation. Pathologically, prostate cancer-associated SPOP mutants fail to interact with and promote the destruction of BET proteins, leading to their elevated abundance in SPOP-deficient prostate cancer. As a result, prostate cancer cells and prostate cancer patient-derived organoids harboring SPOP mutations are more resistant to BET inhibitor-induced cell growth arrest and apoptosis. Therefore, our results elucidate the tumor suppressor role of SPOP in prostate cancer by negatively controlling BET protein stability, and also provide a molecular mechanism for BET inhibitor resistance in prostate cancer patients bearing SPOP mutations

AI is a viable alternative to high throughput screening: a 318-target study

: High throughput screening (HTS) is routinely used to identify bioactive small molecules. This requires physical compounds, which limits coverage of accessible chemical space. Computational approaches combined with vast on-demand chemical libraries can access far greater chemical space, provided that the predictive accuracy is sufficient to identify useful molecules. Through the largest and most diverse virtual HTS campaign reported to date, comprising 318 individual projects, we demonstrate that our AtomNet® convolutional neural network successfully finds novel hits across every major therapeutic area and protein class. We address historical limitations of computational screening by demonstrating success for target proteins without known binders, high-quality X-ray crystal structures, or manual cherry-picking of compounds. We show that the molecules selected by the AtomNet® model are novel drug-like scaffolds rather than minor modifications to known bioactive compounds. Our empirical results suggest that computational methods can substantially replace HTS as the first step of small-molecule drug discovery