3,482 research outputs found
Interactive Search and Exploration in Online Discussion Forums Using Multimodal Embeddings
In this paper we present a novel interactive multimodal learning system,
which facilitates search and exploration in large networks of social multimedia
users. It allows the analyst to identify and select users of interest, and to
find similar users in an interactive learning setting. Our approach is based on
novel multimodal representations of users, words and concepts, which we
simultaneously learn by deploying a general-purpose neural embedding model. We
show these representations to be useful not only for categorizing users, but
also for automatically generating user and community profiles. Inspired by
traditional summarization approaches, we create the profiles by selecting
diverse and representative content from all available modalities, i.e. the
text, image and user modality. The usefulness of the approach is evaluated
using artificial actors, which simulate user behavior in a relevance feedback
scenario. Multiple experiments were conducted in order to evaluate the quality
of our multimodal representations, to compare different embedding strategies,
and to determine the importance of different modalities. We demonstrate the
capabilities of the proposed approach on two different multimedia collections
originating from the violent online extremism forum Stormfront and the
microblogging platform Twitter, which are particularly interesting due to the
high semantic level of the discussions they feature
Online Healthcare Privacy Disclosure User Group Profile Modeling Based on Multimodal Fusion
With the spread of COVID-19, online healthcare is rapidly evolving to assist the public with health, reduce exposure and avoid the risk of cross-infection. Online healthcare platform requires more information from patients than offline, and insufficient or incorrect information may delay or even mislead treatment. Therefore, it is valuable to predict users’ privacy disclosure behaviors while fully protecting their information, which can provide healthcare services for users accurately and realize a personalized online healthcare environment. Compared with the traditional static online healthcare platform user privacy disclosure behavior influence factor analysis, this paper uses multimodal fusion and group profile technology to build a user privacy disclosure model and lay the foundation for personalized online healthcare services. This paper proposes a cross-modal fusion modeling approach to address the problem that the information of each modality cannot be fully utilized in the current online healthcare privacy disclosure modeling. A multimodal user profile approach is used to construct personal and group profiles, and the privacy disclosure behavioral characteristics reflected by both are integrated to realize accurate personalized services for online healthcare. The case study shows that compared with the static unimodal privacy disclosure model, the accuracy of our method gains significant improvement, which is helpful for precision healthcare services and online healthcare platform development
CD-CNN: A Partially Supervised Cross-Domain Deep Learning Model for Urban Resident Recognition
Driven by the wave of urbanization in recent decades, the research topic
about migrant behavior analysis draws great attention from both academia and
the government. Nevertheless, subject to the cost of data collection and the
lack of modeling methods, most of existing studies use only questionnaire
surveys with sparse samples and non-individual level statistical data to
achieve coarse-grained studies of migrant behaviors. In this paper, a partially
supervised cross-domain deep learning model named CD-CNN is proposed for
migrant/native recognition using mobile phone signaling data as behavioral
features and questionnaire survey data as incomplete labels. Specifically,
CD-CNN features in decomposing the mobile data into location domain and
communication domain, and adopts a joint learning framework that combines two
convolutional neural networks with a feature balancing scheme. Moreover, CD-CNN
employs a three-step algorithm for training, in which the co-training step is
of great value to partially supervised cross-domain learning. Comparative
experiments on the city Wuxi demonstrate the high predictive power of CD-CNN.
Two interesting applications further highlight the ability of CD-CNN for
in-depth migrant behavioral analysis.Comment: 8 pages, 5 figures, conferenc
Formalizing Multimedia Recommendation through Multimodal Deep Learning
Recommender systems (RSs) offer personalized navigation experiences on online
platforms, but recommendation remains a challenging task, particularly in
specific scenarios and domains. Multimodality can help tap into richer
information sources and construct more refined user/item profiles for
recommendations. However, existing literature lacks a shared and universal
schema for modeling and solving the recommendation problem through the lens of
multimodality. This work aims to formalize a general multimodal schema for
multimedia recommendation. It provides a comprehensive literature review of
multimodal approaches for multimedia recommendation from the last eight years,
outlines the theoretical foundations of a multimodal pipeline, and demonstrates
its rationale by applying it to selected state-of-the-art approaches. The work
also conducts a benchmarking analysis of recent algorithms for multimedia
recommendation within Elliot, a rigorous framework for evaluating recommender
systems. The main aim is to provide guidelines for designing and implementing
the next generation of multimodal approaches in multimedia recommendation
- …