A Hierarchical Attention-based Contrastive Learning Method for Micro Video Popularity Prediction

Micro videos popularity prediction (MVPP) has recently attracted widespread research interests given the increasing prevalence of video-based social platforms. However, previous studies have overlooked the unique patterns between popular and unpopular videos and the interactions between asynchronous features different data dimensions. To address this, we propose a novel hierarchical attention contrastive learning method named HACL, which extracts explainable representation features, learns their asynchronous interactions from both temporal and spatial levels, and separates the positive and negative embeddings identities. This reveals video popularity in a contrastive and interrelated view, and thus can be responsible for a better MVPP. Dual neural networks account for separate positive and negative patterns via contrastive learning. To obtain the temporal-wise interaction coefficients, we propose a Hadamard-product based attention approach to optimize the trainable attention-map matrices. Results from our experiments on a TikTok micro video dataset show that HACL outperforms benchmarks and provides insightful managerial implications

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Multimodal sentiment analysis in real-life videos

This thesis extends the emerging field of multimodal sentiment analysis of real-life videos, taking two components into consideration: the emotion and the emotion's target. The emotion component of media is traditionally represented as a segment-based intensity model of emotion classes. This representation is replaced here by a value- and time-continuous view. Adjacent research fields, such as affective computing, have largely neglected the linguistic information available from automatic transcripts of audio-video material. As is demonstrated here, this text modality is well-suited for time- and value-continuous prediction. Moreover, source-specific problems, such as trustworthiness, have been largely unexplored so far. This work examines perceived trustworthiness of the source, and its quantification, in user-generated video data and presents a possible modelling path. Furthermore, the transfer between the continuous and discrete emotion representations is explored in order to summarise the emotional context at a segment level. The other component deals with the target of the emotion, for example, the topic the speaker is addressing. Emotion targets in a video dataset can, as is shown here, be coherently extracted based on automatic transcripts without limiting a priori parameters, such as the expected number of targets. Furthermore, alternatives to purely linguistic investigation in predicting targets, such as knowledge-bases and multimodal systems, are investigated. A new dataset is designed for this investigation, and, in conjunction with proposed novel deep neural networks, extensive experiments are conducted to explore the components described above. The developed systems show robust prediction results and demonstrate strengths of the respective modalities, feature sets, and modelling techniques. Finally, foundations are laid for cross-modal information prediction systems with applications to the correction of corrupted in-the-wild signals from real-life videos