70 research outputs found
Personalized Ranking for Context-Aware Venue Suggestion
Making personalized and context-aware suggestions of venues to the users is
very crucial in venue recommendation. These suggestions are often based on
matching the venues' features with the users' preferences, which can be
collected from previously visited locations. In this paper we present a novel
user-modeling approach which relies on a set of scoring functions for making
personalized suggestions of venues based on venues content and reviews as well
as users context. Our experiments, conducted on the dataset of the TREC
Contextual Suggestion Track, prove that our methodology outperforms
state-of-the-art approaches by a significant margin.Comment: The 32nd ACM SIGAPP Symposium On Applied Computing (SAC), Marrakech,
Morocco, April 4-6, 201
System Initiative Prediction for Multi-turn Conversational Information Seeking
Identifying the right moment for a system to take the initiative is essential to conversational information seeking (CIS). Existing studies have extensively studied the clarification need prediction task, i.e., predicting when to ask a clarifying question, however, it only covers one specific system-initiative action. We define the system initiative prediction (SIP) task as predicting whether a CIS system should take the initiative at the next turn. Our analysis reveals that for effective modeling of SIP, it is crucial to capture dependencies between adjacent user-system initiative-taking decisions. We propose to model SIP by CRFs. Due to their graphical nature, CRFs are effective in capturing such dependencies and have greater transparency than more complex methods, e.g., LLMs. Applying CRFs to SIP comes with two challenges: (i) CRFs need to be given the unobservable system utterance at the next turn, and (ii) they do not explicitly model multi-turn features. We model SIP as an input-incomplete sequence labeling problem and propose a multiturn system initiative predictor (MuSIc) that has (i) prior-posterior inter-utterance encoders to eliminate the need to be given the unobservable system utterance, and (ii) a multi-turn feature-aware CRF layer to incorporate multi-turn features into the dependencies between adjacent initiative-taking decisions. Experiments show that MuSIc outperforms LLM-based baselines including LLaMA, achieving state-of-the-art results on SIP. We also show the benefits of SIP on clarification need prediction and action prediction.</p
Understanding and Predicting User Satisfaction with Conversational Recommender Systems
User satisfaction depicts the effectiveness of a system from the user’s perspective. Understanding and predicting user satisfaction is vital for the design of user-oriented evaluation methods for conversational recommender systems (CRSs). Current approaches rely on turn-level satisfaction ratings to predict a user’s overall satisfaction with CRS. These methods assume that all users perceive satisfaction similarly, failing to capture the broader dialogue aspects that influence overall user satisfaction. We investigate the effect of several dialogue aspects on user satisfaction when interacting with a CRS. To this end, we annotate dialogues based on six aspects (i.e., relevance, interestingness, understanding, task-completion, interest-arousal, and efficiency) at the turn and dialogue levels. We find that the concept of satisfaction varies per user. At the turn level, a system’s ability to make relevant recommendations is a significant factor in satisfaction. We adopt these aspects as features for predicting response quality and user satisfaction. We achieve an F1-score of 0.80 in classifying dissatisfactory dialogues, and a Pearson’s r of 0.73 for turn-level response quality estimation, demonstrating the effectiveness of the proposed dialogue aspects in predicting user satisfaction and being able to identify dialogues where the system is failing
Performance Prediction for Conversational Search Using Perplexities of Query Rewrites
We consider query performance prediction (QPP) task for conversational search (CS), i.e., to estimate the retrieval quality for queries in multi-turn conversations. We reuse QPP methods from ad-hoc search for CS by feeding them self-contained query rewrites generated by T5. Our experiments on three CS datasets show that (i) lower query rewriting quality may lead to worse QPP performance, and (ii) incorporating query rewriting quality (as measured by perplexity) improves the effectiveness of QPP methods for CS if the query rewriting quality is limited. Our implementation is publicly available at https://github.com/ChuanMeng/QPP4CS.</p
Towards Building Economic Models of Conversational Search
Various conceptual and descriptive models of conversational search have been
proposed in the literature -- while useful, they do not provide insights into
how interaction between the agent and user would change in response to the
costs and benefits of the different interactions. In this paper, we develop two
economic models of conversational search based on patterns previously observed
during conversational search sessions, which we refer to as: Feedback First
where the agent asks clarifying questions then presents results, and Feedback
After where the agent presents results, and then asks follow up questions. Our
models show that the amount of feedback given/requested depends on its
efficiency at improving the initial or subsequent query and the relative cost
of providing said feedback. This theoretical framework for conversational
search provides a number of insights that can be used to guide and inform the
development of conversational search agents. However, empirical work is needed
to estimate the parameters in order to make predictions specific to a given
conversational search setting.Comment: To appear in ECIR 202
Mental disorders on online social media through the lens of language and behaviour:Analysis and visualisation
Due to the worldwide accessibility to the Internet along with the continuous
advances in mobile technologies, physical and digital worlds have become
completely blended, and the proliferation of social media platforms has taken a
leading role over this evolution. In this paper, we undertake a thorough
analysis towards better visualising and understanding the factors that
characterise and differentiate social media users affected by mental disorders.
We perform different experiments studying multiple dimensions of language,
including vocabulary uniqueness, word usage, linguistic style, psychometric
attributes, emotions' co-occurrence patterns, and online behavioural traits,
including social engagement and posting trends. Our findings reveal significant
differences on the use of function words, such as adverbs and verb tense, and
topic-specific vocabulary, such as biological processes. As for emotional
expression, we observe that affected users tend to share emotions more
regularly than control individuals on average. Overall, the monthly posting
variance of the affected groups is higher than the control groups. Moreover, we
found evidence suggesting that language use on micro-blogging platforms is less
distinguishable for users who have a mental disorder than other less
restrictive platforms. In particular, we observe on Twitter less quantifiable
differences between affected and control groups compared to Reddit.Comment: To appear in Elsevier Information Processing & Managemen
- …