Generating Video Descriptions with Topic Guidance

Generating video descriptions in natural language (a.k.a. video captioning) is a more challenging task than image captioning as the videos are intrinsically more complicated than images in two aspects. First, videos cover a broader range of topics, such as news, music, sports and so on. Second, multiple topics could coexist in the same video. In this paper, we propose a novel caption model, topic-guided model (TGM), to generate topic-oriented descriptions for videos in the wild via exploiting topic information. In addition to predefined topics, i.e., category tags crawled from the web, we also mine topics in a data-driven way based on training captions by an unsupervised topic mining model. We show that data-driven topics reflect a better topic schema than the predefined topics. As for testing video topic prediction, we treat the topic mining model as teacher to train the student, the topic prediction model, by utilizing the full multi-modalities in the video especially the speech modality. We propose a series of caption models to exploit topic guidance, including implicitly using the topics as input features to generate words related to the topic and explicitly modifying the weights in the decoder with topics to function as an ensemble of topic-aware language decoders. Our comprehensive experimental results on the current largest video caption dataset MSR-VTT prove the effectiveness of our topic-guided model, which significantly surpasses the winning performance in the 2016 MSR video to language challenge.Comment: Appeared at ICMR 201

A New Anelasticity Model for Wave Propagation in Partially Saturated Rocks

Elastic wave propagation in partially saturated reservoir rocks induces fluid flow in multi-scale pore spaces, leading to wave anelasticity (velocity dispersion and attenuation). The propagation characteristics cannot be described by a single-scale flow-induced dissipation mechanism. To overcome this problem, we combine the White patchy-saturation theory and the squirt flow model to obtain a new anelasticity theory for wave propagation. We consider a tight sandstone Qingyang area, Ordos Basin, and perform ultrasonic measurements at partial saturation and different confining pressures, where the rock properties are obtained at full-gas saturation. The comparison between the experimental data and the theoretical results yields a fairly good agreement, indicating the efficacy of the new theory

"GenotypeColour™": colour visualisation of SNPs and CNVs

<p>Abstract</p> <p>Background</p> <p>The volume of data available on genetic variations has increased considerably with the recent development of high-density, single-nucleotide polymorphism (SNP) arrays. Several software programs have been developed to assist researchers in the analysis of this huge amount of data, but few can rely upon a whole genome variability visualisation system that could help data interpretation.</p> <p>Results</p> <p>We have developed <it>GenotypeColour™ </it>as a rapid user-friendly tool able to upload, visualise and compare the huge amounts of data produced by Affymetrix Human Mapping GeneChips without losing the overall view of the data.</p> <p>Some features of <it>GenotypeColour™ </it>include visualising the entire genome variability in a single screenshot for one or more samples, the simultaneous display of the genotype and Copy Number state for thousands of SNPs, and the comparison of large amounts of samples by producing "consensus" images displaying regions of complete or partial identity. The software is also useful for genotype analysis of trios and to show regions of potential uniparental disomy (UPD). All information can then be exported in a tabular format for analysis with dedicated software. At present, the software can handle data from 10 K, 100 K, 250 K, 5.0 and 6.0 Affymetrix chips.</p> <p>Conclusion</p> <p>We have created a software that offers a new way of displaying and comparing SNP and CNV genomic data. The software is available free at <url>http://www.med.unibs.it/~barlati/GenotypeColour</url> and is especially useful for the analysis of multiple samples.</p

Locate and Verify: A Two-Stream Network for Improved Deepfake Detection

Deepfake has taken the world by storm, triggering a trust crisis. Current deepfake detection methods are typically inadequate in generalizability, with a tendency to overfit to image contents such as the background, which are frequently occurring but relatively unimportant in the training dataset. Furthermore, current methods heavily rely on a few dominant forgery regions and may ignore other equally important regions, leading to inadequate uncovering of forgery cues. In this paper, we strive to address these shortcomings from three aspects: (1) We propose an innovative two-stream network that effectively enlarges the potential regions from which the model extracts forgery evidence. (2) We devise three functional modules to handle the multi-stream and multi-scale features in a collaborative learning scheme. (3) Confronted with the challenge of obtaining forgery annotations, we propose a Semi-supervised Patch Similarity Learning strategy to estimate patch-level forged location annotations. Empirically, our method demonstrates significantly improved robustness and generalizability, outperforming previous methods on six benchmarks, and improving the frame-level AUC on Deepfake Detection Challenge preview dataset from 0.797 to 0.835 and video-level AUC on CelebDF$\_$v1 dataset from 0.811 to 0.847. Our implementation is available at https://github.com/sccsok/Locate-and-Verify.Comment: 10 pages, 8 figures, 60 references. This paper has been accepted for ACM MM 202

Associations between Sleep Quality and Heart Rate Variability: Implications for a Biological Model of Stress Detection Using Wearable Technology.

INTRODUCTION: The autonomic nervous system plays a vital role in the modulation of many vital bodily functions, one of which is sleep and wakefulness. Many studies have investigated the link between autonomic dysfunction and sleep cycles; however, few studies have investigated the links between short-term sleep health, as determined by the Pittsburgh Quality of Sleep Index (PSQI), such as subjective sleep quality, sleep latency, sleep duration, habitual sleep efficiency, sleep disturbances, use of sleeping medication, and daytime dysfunction, and autonomic functioning in healthy individuals. AIM: In this cross-sectional study, the aim was to investigate the links between short-term sleep quality and duration, and heart rate variability in 60 healthy individuals, in order to provide useful information about the effects of stress and sleep on heart rate variability (HRV) indices, which in turn could be integrated into biological models for wearable devices. METHODS: Sleep parameters were collected from participants on commencement of the study, and HRV was derived using an electrocardiogram (ECG) during a resting and stress task (Trier Stress Test). RESULT: Low-frequency to high-frequency (LF:HF) ratio was significantly higher during the stress task than during the baseline resting phase, and very-low-frequency and high-frequency HRV were inversely related to impaired sleep during stress tasks. CONCLUSION: Given the ubiquitous nature of wearable technologies for monitoring health states, in particular HRV, it is important to consider the impacts of sleep states when using these technologies to interpret data. Very-low-frequency HRV during the stress task was found to be inversely related to three negative sleep indices: sleep quality, daytime dysfunction, and global sleep score

Stress Watch: The Use of Heart Rate and Heart Rate Variability to Detect Stress: A Pilot Study Using Smart Watch Wearables.

Stress is an inherent part of the normal human experience. Although, for the most part, this stress response is advantageous, chronic, heightened, or inappropriate stress responses can have deleterious effects on the human body. It has been suggested that individuals who experience repeated or prolonged stress exhibit blunted biological stress responses when compared to the general population. Thus, when assessing whether a ubiquitous stress response exists, it is important to stratify based on resting levels in the absence of stress. Research has shown that stress that causes symptomatic responses requires early intervention in order to mitigate possible associated mental health decline and personal risks. Given this, real-time monitoring of stress may provide immediate biofeedback to the individual and allow for early self-intervention. This study aimed to determine if the change in heart rate variability could predict, in two different cohorts, the quality of response to acute stress when exposed to an acute stressor and, in turn, contribute to the development of a physiological algorithm for stress which could be utilized in future smartwatch technologies. This study also aimed to assess whether baseline stress levels may affect the changes seen in heart rate variability at baseline and following stress tasks. A total of 30 student doctor participants and 30 participants from the general population were recruited for the study. The Trier Stress Test was utilized to induce stress, with resting and stress phase ECGs recorded, as well as inter-second heart rate (recorded using a FitBit). Although the present study failed to identify ubiquitous patterns of HRV and HR changes during stress, it did identify novel changes in these parameters between resting and stress states. This study has shown that the utilization of HRV as a measure of stress should be calculated with consideration of resting (baseline) anxiety and stress states in order to ensure an accurate measure of the effects of additive acute stress