What drives the helpfulness of online reviews? A deep learning study of sentiment analysis, pictorial content and reviewer expertise for mature destinations.

User-generated content (UGC) is a growing driver of destination choice. Drawing on dual-process theories on how individuals process information, this study focuses on the role of central and peripheral information processing routes in the formation of consumers’ perceptions of the helpfulness of online reviews. We carried out a two-step process to address the perceived helpfulness of user-generated content, a sentiment analysis using advanced machine-learning techniques (deep learning), and a regression analysis. We used a database of 2,023 comments posted on TripAdvisor about two iconic Venetian cultural attractions, St. Mark’s Square (an open, free attraction) and the Doge’s Palace (a museum which charges an entry fee). Following the application of deep-learning techniques, we first identified which factors influenced whether a review received a “helpful” vote by means of logistic regression. Second, we selected those reviews which received at least one helpful vote to identify, through linear regression, the significant determinants of TripAdvisor users’ voting behaviour. The results showed that reviewer expertise is an influential factor in both free and paid-for attractions, although the impact of central cues (sentiment polarity, subjectivity and pictorial content) is different in both attractions. Our study suggests that managers should look beyond individual ratings and focus on the sentiment analysis of online reviews, which are shown to be based on the nature of the attraction (free vs. paid-for)

Attentive Aspect Modeling for Review-aware Recommendation

In recent years, many studies extract aspects from user reviews and integrate them with ratings for improving the recommendation performance. The common aspects mentioned in a user's reviews and a product's reviews indicate indirect connections between the user and product. However, these aspect-based methods suffer from two problems. First, the common aspects are usually very sparse, which is caused by the sparsity of user-product interactions and the diversity of individual users' vocabularies. Second, a user's interests on aspects could be different with respect to different products, which are usually assumed to be static in existing methods. In this paper, we propose an Attentive Aspect-based Recommendation Model (AARM) to tackle these challenges. For the first problem, to enrich the aspect connections between user and product, besides common aspects, AARM also models the interactions between synonymous and similar aspects. For the second problem, a neural attention network which simultaneously considers user, product and aspect information is constructed to capture a user's attention towards aspects when examining different products. Extensive quantitative and qualitative experiments show that AARM can effectively alleviate the two aforementioned problems and significantly outperforms several state-of-the-art recommendation methods on top-N recommendation task.Comment: Camera-ready manuscript for TOI

Using consumer feedback from location-based services in PoI recommender systems for people with autism

When suggesting Points of Interest (PoIs) to people with autism spectrum disorders, we must take into account that they have idiosyncratic sensory aversions to noise, brightness and other features that influence the way they perceive places. Therefore, recommender systems must deal with these aspects. However, the retrieval of sensory data about PoIs is a real challenge because most geographical information servers fail to provide this data. Moreover, ad-hoc crowdsourcing campaigns do not guarantee to cover large geographical areas and lack sustainability. Thus, we investigate the extraction of sensory data about places from the consumer feedback collected by location-based services, on which people spontaneously post reviews from all over the world. Specifically, we propose a model for the extraction of sensory data from the reviews about PoIs, and its integration in recommender systems to predict item ratings by considering both user preferences and compatibility information. We tested our approach with autistic and neurotypical people by integrating it into diverse recommendation algorithms. For the test, we used a dataset built in a crowdsourcing campaign and another one extracted from TripAdvisor reviews. The results show that the algorithms obtain the highest accuracy and ranking capability when using TripAdvisor data. Moreover, by jointly using these two datasets, the algorithms further improve their performance. These results encourage the use of consumer feedback as a reliable source of information about places in the development of inclusive recommender systems

Personalizing online reviews for better customer decision making

Online consumer reviews have become an important source of information for understanding markets and customer preferences. When making purchase decisions, customers increasingly rely on user-generated online reviews; some even consider the information in online reviews more credible and trustworthy than information provided by vendors. Many studies have revealed that online reviews influence demand and sales. Others have shown the possibility of identifying customer interest in product attributes. However, little work has been done to address customer and review diversity in the process of examining reviews. This research intends to answer the research question: how can we solve the problem of customer and review diversity in the context of online reviews to recommend useful reviews based on customer preferences and improve product recommendation? Our approach to the question is through personalization. Similar to other personalization research, we use an attribute-based model to represent products and customer preferences. Unlike existing personalization research that uses a set of pre-defined product attributes, we explore the possibility of a data-driven approach for identifying more comprehensive product attributes from online reviews to model products and customer preferences. Specifically, we introduce a new topic model for product attribute identification and sentiment analysis. By differentiating word co-occurrences at the sentence level from at the document level, the model better identifies interpretable topics. The use of an inference network with shared structure enables the model to predict product attribute ratings accurately. Based on this topic model, we develop attribute-based representations of products, reviews and customer preferences and use them to construct the personalization of online reviews. We examine personalization from the lens of consumer search theory and human information processing theory and test the hypotheses with an experiment. The personalization of online reviews can 1) recommend products matching customer's preferences; 2) improve custom's intention towards recommended products; 3) best distinguish recommended products from products that do not match customer's preferences; and 4) reduce decision effort

Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations

Explainable AI (XAI) is widely viewed as a sine qua non for ever-expanding AI research. A better understanding of the needs of XAI users, as well as human-centered evaluations of explainable models are both a necessity and a challenge. In this paper, we explore how HCI and AI researchers conduct user studies in XAI applications based on a systematic literature review. After identifying and thoroughly analyzing 97core papers with human-based XAI evaluations over the past five years, we categorize them along the measured characteristics of explanatory methods, namely trust, understanding, usability, and human-AI collaboration performance. Our research shows that XAI is spreading more rapidly in certain application domains, such as recommender systems than in others, but that user evaluations are still rather sparse and incorporate hardly any insights from cognitive or social sciences. Based on a comprehensive discussion of best practices, i.e., common models, design choices, and measures in user studies, we propose practical guidelines on designing and conducting user studies for XAI researchers and practitioners. Lastly, this survey also highlights several open research directions, particularly linking psychological science and human-centered XAI

A framework for leveraging properties of user reviews in recommendation

With the growing volume of information online, it is increasingly harder for users to identify useful information to support their choices when interacting with different items. Review-based recommendation systems, which leverage reviews posted by users on items to estimate the users’ preferences, have been shown to be a credible solution for addressing the problem of identifying their preferred items. However, the actual usefulness of these reviews impact the effectiveness of the resulting recommender systems, especially with the enormous volume of available reviews online. In particular, as argued by the widely cited users’ adoption of information framework, users exhibit distinct preferences for reviews depending on the properties of these reviews (e.g. length, sentiment) when making decisions. Therefore, we argue that not all reviews are equally useful for different users. We aim to effectively modelling the personalised usefulness of reviews through the use of reviews’ properties when developing review-based recommendation techniques. Note that, few studies in the literature investigated the effectiveness of leveraging the properties of reviews to develop effective review-based recommendation approaches. This thesis aims to address this research gap by proposing a review-based recommendation framework. Such a framework models the personalised usefulness of reviews according to various reviews’ properties, including the reviews’ age, length, sentiment, ratings, helpfulness as judged by the users and helpfulness as predicted by a review helpfulness classifier. In particular, the thesis addresses two main challenges: (i) the availability of the attributes of reviews and (ii) the users’ preferences estimation. The first challenge refers to the difficulty of extracting particular review properties from their corresponding attributes. For example, extraction of the age property relies on the availability of the timestamps of the corresponding reviews. We address the availability of the reviews’ attributes to extract their sentiment and helpfulness properties with classification techniques. The sentiment property of reviews is estimated through effective state-of-the-art sentiment classifiers. We first evaluate the estimated reviews’ sentiment in comparison to the users’ ratings in typical recommendation approaches. Then, we introduce a sentiment attention mechanism to encode the estimated reviews’ sentiment. Our experiments show that the sentiment property can effectively replace the users’ ratings when estimating the user preferences. Moreover, by leveraging the estimated sentiment property of reviews, our proposed review-based rating prediction model shows improved performance compared to state-of-the-art rating prediction models. Next, the extraction of the reviews’ helpfulness property leverages the reviews’ helpful votes (i.e. a type of feedback given by other reviewers providing information on whether the corresponding review is helpful to them). However, the number of helpful votes for each review are not commonly available. In particular, we propose a novel weakly supervised review helpfulness classification correction approach (i.e. the Negative Confidence-aware Weakly Supervised (NCWS) approach), which leverages the confidence in a given review being unhelpful with respect to its age. We experimentally show that NCWS-based classifiers significantly outperform existing review helpfulness classifiers on two public review datasets. Moreover, the estimated helpfulness of reviews by NCWS-based classifiers can significantly improve the performance of a review-based rating prediction model. Next, to address our second challenge pertaining to the users’ preferences estimation, we aim to estimate their preferences when using reviews exhibiting different properties to accurately predict their preferred items. In particular, we propose two novel ranking-based recommendation approaches (named RPRM and BanditProp), which models the users’ preferences using different review properties with different techniques. The RPRM model applies the attention mechanism to model the usefulness of reviews according to different review properties. Unlike RPRM, the BanditProp model leverages existing bandit algorithms and introduces a novel contextual bandit algorithm to tackle the users’ preference estimation of using specific reviews’ properties to identify useful reviews. Our experiments show that RPRM can outperform stateof-the-art review-based recommendation models, and BanditProp can significantly outperform RPRM on two publicly available review datasets. These results validate the effectiveness of leveraging the review properties when examining the usefulness of reviews to improve the performances of review-based recommendation techniques. Overall, we contribute an effective review-based recommendation framework that make accurate recommendations by leveraging the reviews’ associated properties. This framework includes models for extracting properties from reviews, and various techniques that are required to integrate the learned properties, which, in turn and according to our conducted experiments, provide good approximations of a given users’ item preferences. These contributions make progress in the development of review-based recommendation techniques and could inspire future directions of research in recommendation systems

A flexible framework for evaluating user and item fairness in recommender systems

This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/s11257-020-09285-1One common characteristic of research works focused on fairness evaluation (in machine learning) is that they call for some form of parity (equality) either in treatment—meaning they ignore the information about users’ memberships in protected classes during training—or in impact—by enforcing proportional beneficial outcomes to users in different protected classes. In the recommender systems community, fairness has been studied with respect to both users’ and items’ memberships in protected classes defined by some sensitive attributes (e.g., gender or race for users, revenue in a multi-stakeholder setting for items). Again here, the concept has been commonly interpreted as some form of equality—i.e., the degree to which the system is meeting the information needs of all its users in an equal sense. In this work, we propose a probabilistic framework based on generalized cross entropy (GCE) to measure fairness of a given recommendation model. The framework comes with a suite of advantages: first, it allows the system designer to define and measure fairness for both users and items and can be applied to any classification task; second, it can incorporate various notions of fairness as it does not rely on specific and predefined probability distributions and they can be defined at design time; finally, in its design it uses a gain factor, which can be flexibly defined to contemplate different accuracy-related metrics to measure fairness upon decision-support metrics (e.g., precision, recall) or rank-based measures (e.g., NDCG, MAP). An experimental evaluation on four real-world datasets shows the nuances captured by our proposed metric regarding fairness on different user and item attributes, where nearest-neighbor recommenders tend to obtain good results under equality constraints. We observed that when the users are clustered based on both their interaction with the system and other sensitive attributes, such as age or gender, algorithms with similar performance values get different behaviors with respect to user fairness due to the different way they process data for each user clusterThe authors thank the reviewers for their thoughtful comments and suggestions. This work was supported in part by the Ministerio de Ciencia, Innovacion y Universidades (Reference: 123496 Y. Deldjoo et al. PID2019-108965GB-I00) and in part by the Center for Intelligent Information Retrieval. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsor