5,079 research outputs found
Evaluating Similarity Metrics for Latent Twitter Topics
Topic modelling approaches such as LDA, when applied on a tweet corpus, can often generate a topic model containing redundant topics. To evaluate the quality of a topic model in terms of redundancy, topic similarity metrics can be applied to estimate the similarity among topics in a topic model. There are various topic similarity metrics in the literature, e.g. the Jensen Shannon (JS) divergence-based metric. In this paper, we evaluate the performances of four distance/divergence-based topic similarity metrics and examine how they align with human judgements, including a newly proposed similarity metric that is based on computing word semantic similarity using word embeddings (WE). To obtain human judgements, we conduct a user study through crowdsourcing. Among various insights, our study shows that in general the cosine similarity (CS) and WE-based metrics perform better and appear to be complementary. However, we also find that the human assessors cannot easily distinguish between the distance/divergence-based and the semantic similarity-based metrics when identifying similar latent Twitter topics
Using Word Embedding to Evaluate the Coherence of Topics from Twitter Data
Scholars often seek to understand topics discussed on Twitter using topic modelling approaches. Several coherence
metrics have been proposed for evaluating the coherence
of the topics generated by these approaches, including the
pre-calculated Pointwise Mutual Information (PMI) of word
pairs and the Latent Semantic Analysis (LSA) word representation vectors. As Twitter data contains abbreviations
and a number of peculiarities (e.g. hashtags), it can be challenging to train effective PMI data or LSA word representation. Recently, Word Embedding (WE) has emerged as a
particularly effective approach for capturing the similarity
among words. Hence, in this paper, we propose new Word
Embedding-based topic coherence metrics. To determine
the usefulness of these new metrics, we compare them with
the previous PMI/LSA-based metrics. We also conduct a
large-scale crowdsourced user study to determine whether
the new Word Embedding-based metrics better align with
human preferences. Using two Twitter datasets, our results
show that the WE-based metrics can capture the coherence
of topics in tweets more robustly and efficiently than the
PMI/LSA-based ones
From the User to the Medium: Neural Profiling Across Web Communities
Online communities provide a unique way for individuals to access information
from those in similar circumstances, which can be critical for health
conditions that require daily and personalized management. As these groups and
topics often arise organically, identifying the types of topics discussed is
necessary to understand their needs. As well, these communities and people in
them can be quite diverse, and existing community detection methods have not
been extended towards evaluating these heterogeneities. This has been limited
as community detection methodologies have not focused on community detection
based on semantic relations between textual features of the user-generated
content. Thus here we develop an approach, NeuroCom, that optimally finds dense
groups of users as communities in a latent space inferred by neural
representation of published contents of users. By embedding of words and
messages, we show that NeuroCom demonstrates improved clustering and identifies
more nuanced discussion topics in contrast to other common unsupervised
learning approaches
Semi-Supervised Approach to Monitoring Clinical Depressive Symptoms in Social Media
With the rise of social media, millions of people are routinely expressing
their moods, feelings, and daily struggles with mental health issues on social
media platforms like Twitter. Unlike traditional observational cohort studies
conducted through questionnaires and self-reported surveys, we explore the
reliable detection of clinical depression from tweets obtained unobtrusively.
Based on the analysis of tweets crawled from users with self-reported
depressive symptoms in their Twitter profiles, we demonstrate the potential for
detecting clinical depression symptoms which emulate the PHQ-9 questionnaire
clinicians use today. Our study uses a semi-supervised statistical model to
evaluate how the duration of these symptoms and their expression on Twitter (in
terms of word usage patterns and topical preferences) align with the medical
findings reported via the PHQ-9. Our proactive and automatic screening tool is
able to identify clinical depressive symptoms with an accuracy of 68% and
precision of 72%.Comment: 8 pages, Advances in Social Networks Analysis and Mining (ASONAM),
2017 IEEE/ACM International Conferenc
Data Portraits and Intermediary Topics: Encouraging Exploration of Politically Diverse Profiles
In micro-blogging platforms, people connect and interact with others.
However, due to cognitive biases, they tend to interact with like-minded people
and read agreeable information only. Many efforts to make people connect with
those who think differently have not worked well. In this paper, we
hypothesize, first, that previous approaches have not worked because they have
been direct -- they have tried to explicitly connect people with those having
opposing views on sensitive issues. Second, that neither recommendation or
presentation of information by themselves are enough to encourage behavioral
change. We propose a platform that mixes a recommender algorithm and a
visualization-based user interface to explore recommendations. It recommends
politically diverse profiles in terms of distance of latent topics, and
displays those recommendations in a visual representation of each user's
personal content. We performed an "in the wild" evaluation of this platform,
and found that people explored more recommendations when using a biased
algorithm instead of ours. In line with our hypothesis, we also found that the
mixture of our recommender algorithm and our user interface, allowed
politically interested users to exhibit an unbiased exploration of the
recommended profiles. Finally, our results contribute insights in two aspects:
first, which individual differences are important when designing platforms
aimed at behavioral change; and second, which algorithms and user interfaces
should be mixed to help users avoid cognitive mechanisms that lead to biased
behavior.Comment: 12 pages, 7 figures. To be presented at ACM Intelligent User
Interfaces 201
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Sequential data often possesses a hierarchical structure with complex
dependencies between subsequences, such as found between the utterances in a
dialogue. In an effort to model this kind of generative process, we propose a
neural network-based generative architecture, with latent stochastic variables
that span a variable number of time steps. We apply the proposed model to the
task of dialogue response generation and compare it with recent neural network
architectures. We evaluate the model performance through automatic evaluation
metrics and by carrying out a human evaluation. The experiments demonstrate
that our model improves upon recently proposed models and that the latent
variables facilitate the generation of long outputs and maintain the context.Comment: 15 pages, 5 tables, 4 figure
- …