Speaker-following Video Subtitles

We propose a new method for improving the presentation of subtitles in video (e.g. TV and movies). With conventional subtitles, the viewer has to constantly look away from the main viewing area to read the subtitles at the bottom of the screen, which disrupts the viewing experience and causes unnecessary eyestrain. Our method places on-screen subtitles next to the respective speakers to allow the viewer to follow the visual content while simultaneously reading the subtitles. We use novel identification algorithms to detect the speakers based on audio and visual information. Then the placement of the subtitles is determined using global optimization. A comprehensive usability study indicated that our subtitle placement method outperformed both conventional fixed-position subtitling and another previous dynamic subtitling method in terms of enhancing the overall viewing experience and reducing eyestrain

Look, Listen and Learn - A Multimodal LSTM for Speaker Identification

Speaker identification refers to the task of localizing the face of a person who has the same identity as the ongoing voice in a video. This task not only requires collective perception over both visual and auditory signals, the robustness to handle severe quality degradations and unconstrained content variations are also indispensable. In this paper, we describe a novel multimodal Long Short-Term Memory (LSTM) architecture which seamlessly unifies both visual and auditory modalities from the beginning of each sequence input. The key idea is to extend the conventional LSTM by not only sharing weights across time steps, but also sharing weights across modalities. We show that modeling the temporal dependency across face and voice can significantly improve the robustness to content quality degradations and variations. We also found that our multimodal LSTM is robustness to distractors, namely the non-speaking identities. We applied our multimodal LSTM to The Big Bang Theory dataset and showed that our system outperforms the state-of-the-art systems in speaker identification with lower false alarm rate and higher recognition accuracy.Comment: The 30th AAAI Conference on Artificial Intelligence (AAAI-16

Association of tissue lineage and gene expression: conservatively and differentially expressed genes define common and special functions of tissues

<p>Abstract</p> <p>Background</p> <p>Embryogenesis is the process by which the embryo is formed, develops, and establishes developmental hierarchies of tissues. The recent advance in microarray technology made it possible to investigate the tissue specific patterns of gene expression and their relationship with tissue lineages. This study is focused on how tissue specific functions, tissue lineage, and cell differentiation are correlated, which is essential to understand embryonic development and organism complexity.</p> <p>Results</p> <p>We performed individual gene and gene set based analysis on multiple tissue expression data, in association with the classic topology of mammalian fate maps of embryogenesis. For each sub-group of tissues on the fate map, conservatively, differentially and correlatively expressed genes or gene sets were identified. Tissue distance was found to correlate with gene expression divergence. Tissues of the ectoderm or mesoderm origins from the same segments on the fate map shared more similar expression pattern than those from different origins. Conservatively expressed genes or gene sets define common functions in a tissue group and are related to tissue specific diseases, which is supported by results from Gene Ontology and KEGG pathway analysis. Gene expression divergence is larger in certain human tissues than in the mouse homologous tissues.</p> <p>Conclusion</p> <p>The results from tissue lineage and gene expression analysis indicate that common function features of neighbor tissue groups were defined by the conservatively expressed genes and were related to tissue specific diseases, and differentially expressed genes contribute to the functional divergence of tissues. The difference of gene expression divergence in human and mouse homologous tissues reflected the organism complexity, i.e. distinct neural development levels and different body sizes.</p

Learning conservation laws in unknown quantum dynamics

We present a learning algorithm for discovering conservation laws given as sums of geometrically local observables in quantum dynamics. This includes conserved quantities that arise from local and global symmetries in closed and open quantum many-body systems. The algorithm combines the classical shadow formalism for estimating expectation values of observable and data analysis techniques based on singular value decompositions and robust polynomial interpolation to discover all such conservation laws in unknown quantum dynamics with rigorous performance guarantees. Our method can be directly realized in quantum experiments, which we illustrate with numerical simulations, using closed and open quantum system dynamics in a $\mathbb{Z}_2$-gauge theory and in many-body localized spin-chains.Comment: 22 pages, 3 figure

A Tensor-Based Framework for Studying Eigenvector Multicentrality in Multilayer Networks

Centrality is widely recognized as one of the most critical measures to provide insight in the structure and function of complex networks. While various centrality measures have been proposed for single-layer networks, a general framework for studying centrality in multilayer networks (i.e., multicentrality) is still lacking. In this study, a tensor-based framework is introduced to study eigenvector multicentrality, which enables the quantification of the impact of interlayer influence on multicentrality, providing a systematic way to describe how multicentrality propagates across different layers. This framework can leverage prior knowledge about the interplay among layers to better characterize multicentrality for varying scenarios. Two interesting cases are presented to illustrate how to model multilayer influence by choosing appropriate functions of interlayer influence and design algorithms to calculate eigenvector multicentrality. This framework is applied to analyze several empirical multilayer networks, and the results corroborate that it can quantify the influence among layers and multicentrality of nodes effectively.Comment: 57 pages, 10 figure

Characteristic extraction of rolling bearing compound faults of aero-engine

Rolling bearing’s fault mode usually shows compound faults in aero-engine. The compound faults characteristics are more complex than single one, and many signal analysis methods have rather great limitation for compound fault characteristic extraction which leads to the difficulty to monitor the running state of rolling bearing in aero-engine. Based on above analysis, a method of combining wavelet transform with cyclostationary theory, autocorrelation function and Hilbert transform is proposed and applied to extract characteristic frequency of rolling bearing from compound faults mode only according to single-channel vibration acceleration signal of aero-engine. Meanwhile, a consideration is given to the influence of sensor installation position, compound fault types in the extraction of compound faults characteristics. The result indicates that the proposed new method can effectively monitor rolling bearing running state in four different compound fault modes just according to single-channel vibration acceleration signal no matter sensors are installed in horizontal or vertical direction

Alterations of microbiota and metabolites in the feces of calves with diarrhea associated with rotavirus and coronavirus infections

The changes in the composition of intestinal microbiota and metabolites have been linked to digestive disorders in calves, especially neonatal calf diarrhea. Bovine rotavirus (BRV) and bovine coronavirus (BCoV) are known to be the primary culprits behind neonatal calf diarrhea. In this study, we analyzed changes in the fecal microbiota and metabolites of calves with neonatal diarrhea associated with BRV and BCoV infection using high-throughput 16S rRNA sequencing and metabolomics technology. The microbial diversity in the feces of calves infected with BRV and BCoV with diarrhea decreased significantly, and the composition changed significantly. The significant increase of Fusobacterium and the reductions of some bacteria genera, including Faecalibacterium, Bifidobacterium, Ruminococcus, Subdoligranulum, Parabacteroides, Collinsella, and Olsenella, etc., were closely related to diarrhea associated with BRV and BCoV infection. Metabolites in the feces of BRV and BCoV-infected calves with diarrhea were significantly changed. Phosphatidylcholine [PC; 16:1(9 Z)/16:1(9 Z)], lysophosphatidylethanolamine (LysoPE; 0:0/22:0), lysophosphatidylcholine (LysoPC; P-16:0) and LysoPE (0:0/18:0) were significantly higher in the feces of BRV-infected calves with diarrhea. In contrast, some others, such as desthiobiotin, were significantly lower. BRV infection affects glycerophospholipid metabolism and biotin metabolism in calves. Two differential metabolites were significantly increased, and 67 differential metabolites were significantly reduced in the feces of BCoV-infected calves with diarrhea. Seven significantly reduced metabolites, including deoxythymidylic acid (DTMP), dihydrobiopterin, dihydroneopterin triphosphate, cortexolone, cortisol, pantetheine, and pregnenolone sulfate, were enriched in the folate biosynthesis, pantothenate and CoA biosynthesis, pyrimidine metabolism, and steroid hormone biosynthesis pathway. The decrease in these metabolites was closely associated with increased harmful bacteria and reduced commensal bacteria. The content of short-chain fatty acids (SCFAs) such as acetic acid and propionic acid in the feces of BRV and BCoV-infected calves with diarrhea was lower than that of healthy calves, which was associated with the depletion of SCFAs-producing bacteria such as Parabacteroides, Fournierella, and Collinsella. The present study showed that BRV and BCoV infections changed the composition of the calf fecal microbiota and were associated with changes in fecal metabolites. This study lays the foundation for further revealing the roles of intestinal microbiota in neonatal calf diarrhea associated with BRV and BCoV infection