Deep Cross-Modal Correlation Learning for Audio and Lyrics in Music Retrieval

Deep cross-modal learning has successfully demonstrated excellent performance in cross-modal multimedia retrieval, with the aim of learning joint representations between different data modalities. Unfortunately, little research focuses on cross-modal correlation learning where temporal structures of different data modalities such as audio and lyrics should be taken into account. Stemming from the characteristic of temporal structures of music in nature, we are motivated to learn the deep sequential correlation between audio and lyrics. In this work, we propose a deep cross-modal correlation learning architecture involving two-branch deep neural networks for audio modality and text modality (lyrics). Data in different modalities are converted to the same canonical space where inter modal canonical correlation analysis is utilized as an objective function to calculate the similarity of temporal structures. This is the first study that uses deep architectures for learning the temporal correlation between audio and lyrics. A pre-trained Doc2Vec model followed by fully-connected layers is used to represent lyrics. Two significant contributions are made in the audio branch, as follows: i) We propose an end-to-end network to learn cross-modal correlation between audio and lyrics, where feature extraction and correlation learning are simultaneously performed and joint representation is learned by considering temporal structures. ii) As for feature extraction, we further represent an audio signal by a short sequence of local summaries (VGG16 features) and apply a recurrent neural network to compute a compact feature that better learns temporal structures of music audio. Experimental results, using audio to retrieve lyrics or using lyrics to retrieve audio, verify the effectiveness of the proposed deep correlation learning architectures in cross-modal music retrieval

Context-Patch Face Hallucination Based on Thresholding Locality-Constrained Representation and Reproducing Learning

Face hallucination is a technique that reconstruct high-resolution (HR) faces from low-resolution (LR) faces, by using the prior knowledge learned from HR/LR face pairs. Most state-of-the-arts leverage position-patch prior knowledge of human face to estimate the optimal representation coefficients for each image patch. However, they focus only the position information and usually ignore the context information of image patch. In addition, when they are confronted with misalignment or the Small Sample Size (SSS) problem, the hallucination performance is very poor. To this end, this study incorporates the contextual information of image patch and proposes a powerful and efficient context-patch based face hallucination approach, namely Thresholding Locality-constrained Representation and Reproducing learning (TLcR-RL). Under the context-patch based framework, we advance a thresholding based representation method to enhance the reconstruction accuracy and reduce the computational complexity. To further improve the performance of the proposed algorithm, we propose a promotion strategy called reproducing learning. By adding the estimated HR face to the training set, which can simulates the case that the HR version of the input LR face is present in the training set, thus iteratively enhancing the final hallucination result. Experiments demonstrate that the proposed TLcR-RL method achieves a substantial increase in the hallucinated results, both subjectively and objectively. Additionally, the proposed framework is more robust to face misalignment and the SSS problem, and its hallucinated HR face is still very good when the LR test face is from the real-world. The MATLAB source code is available at https://github.com/junjun-jiang/TLcR-RL

High mobility group box-1 in hypothalamic paraventricular nuclei attenuates sympathetic tone in rats at post-myocardial infarction

Background: Inflammation is associated with increased sympathetic drive in cardiovascular diseases. The paraventricular nucleus (PVN) of the hypothalamus is a key regulator of sympathetic nerve activity at post-myocardial infarction (MI). High mobility group box-1 (HMGB1) exhibits inflammatory cytokine like activity in the extracellular space. Inflammation is associated with increased sympathetic drive in cardiovscular diseases. However, the role of HMGB1 in sympathetic nerve activity at post-MI remains unknown. The aim of the present study is to determine the role and mechanism of HMGB1 in the PVN, in terms of sympathetic activity and arrhythmia after MI. Methods: Sprague-Dawley rats underwent left anterior descending coronary artery ligation to induce MI. Anti-HMGB1 polyclonal antibody or control IgG was bilaterally microinjected into the PVN (5 μL every second day for seven consecutive days). Then, renal sympathetic nerve activity (RSNA) was recorded. The association between ventricular arrhythmias (VAs) and MI was evaluated using programmedelectrophysiological stimulation. After performing electrophysiological experiments in vivo, immunohistochemistry was used to detect the distribution of HMGB1, while Western blot was used to detect the expression of HMGB1 and p-ERK in the PVN of MI rats. Results: HMGB1 and p-ERK were upregulated in the PVN in rats at post-MI. Moreover, bilateral PVN microinjection of anti-HMGB1 polyclonal antibody reversed the expression of HMGB1 and p-ERK, and consequently decreased the baseline RSNA and inducible VAs, when compared to those in sham rats. Conclusions: These results suggest that MI causes the translocation of HMGB1 in the PVN, which leads to sympathetic overactivation through the ERK1/2 signaling pathway. The bilateral PVN microinjection of anti-HMGB1 antibody can be an effective therapy for MI-induced arrhythmia

RNA-directed DNA methylation involves co-transcriptional small-RNA-guided slicing of polymerase V transcripts in Arabidopsis.

Small RNAs regulate chromatin modifications such as DNA methylation and gene silencing across eukaryotic genomes. In plants, RNA-directed DNA methylation (RdDM) requires 24-nucleotide small interfering RNAs (siRNAs) that bind to ARGONAUTE 4 (AGO4) and target genomic regions for silencing. RdDM also requires non-coding RNAs transcribed by RNA polymerase V (Pol V) that probably serve as scaffolds for binding of AGO4-siRNA complexes. Here, we used a modified global nuclear run-on protocol followed by deep sequencing to capture Pol V nascent transcripts genome-wide. We uncovered unique characteristics of Pol V RNAs, including a uracil (U) common at position 10. This uracil was complementary to the 5' adenine found in many AGO4-bound 24-nucleotide siRNAs and was eliminated in a siRNA-deficient mutant as well as in the ago4/6/9 triple mutant, suggesting that the +10 U signature is due to siRNA-mediated co-transcriptional slicing of Pol V transcripts. Expression of wild-type AGO4 in ago4/6/9 mutants was able to restore slicing of Pol V transcripts, but a catalytically inactive AGO4 mutant did not correct the slicing defect. We also found that Pol V transcript slicing required SUPPRESSOR OF TY INSERTION 5-LIKE (SPT5L), an elongation factor whose function is not well understood. These results highlight the importance of Pol V transcript slicing in RNA-mediated transcriptional gene silencing, which is a conserved process in many eukaryotes