Predicting the helpfulness score of videogames of the STEAM platform

[EN] Online reviews comprise a flood of user-generated content, so to identify the most useful reviews is a vital task. As such, many computational models have been made to automatically analyze the helpfulness of online reviews. In this work, we aim to predict the helpfulness score of videogames reviews using an available online dataset of more than 1M rows. We trained three different machine learning algorithms by implementing two strategies, predicting the helpfulness as a regression problem or as a binary classification problem. Our findings show that binary classification is the best method, and the achieved ROC-AUC of the best model is 0.7 with only a selected set of features. In addition, we found that using the feature vectors from a pretrained NLP model does not improve the performance of the models.The work has been performed under the Project HPC-EUROPA3 (INFRAIA-2016-1-730897), with the support of the EC Research Innovation Action under the H2020 ProgrammeEspinosa-Leal, L.; Olmedilla, M.; Li, Z. (2023). Predicting the helpfulness score of videogames of the STEAM platform. Editorial Universitat Politècnica de València. 337-338. http://hdl.handle.net/10251/20176733733

Exploring Latent Semantic Factors to Find Useful Product Reviews

Online reviews provided by consumers are a valuable asset for e-Commerce platforms, influencing potential consumers in making purchasing decisions. However, these reviews are of varying quality, with the useful ones buried deep within a heap of non-informative reviews. In this work, we attempt to automatically identify review quality in terms of its helpfulness to the end consumers. In contrast to previous works in this domain exploiting a variety of syntactic and community-level features, we delve deep into the semantics of reviews as to what makes them useful, providing interpretable explanation for the same. We identify a set of consistency and semantic factors, all from the text, ratings, and timestamps of user-generated reviews, making our approach generalizable across all communities and domains. We explore review semantics in terms of several latent factors like the expertise of its author, his judgment about the fine-grained facets of the underlying product, and his writing style. These are cast into a Hidden Markov Model -- Latent Dirichlet Allocation (HMM-LDA) based model to jointly infer: (i) reviewer expertise, (ii) item facets, and (iii) review helpfulness. Large-scale experiments on five real-world datasets from Amazon show significant improvement over state-of-the-art baselines in predicting and ranking useful reviews

Helpfulness Guided Review Summarization

User-generated online reviews are an important information resource in people's everyday life. As the review volume grows explosively, the ability to automatically identify and summarize useful information from reviews becomes essential in providing analytic services in many review-based applications. While prior work on review summarization focused on different review perspectives (e.g. topics, opinions, sentiment, etc.), the helpfulness of reviews is an important informativeness indicator that has been less frequently explored. In this thesis, we investigate automatic review helpfulness prediction and exploit review helpfulness for review summarization in distinct review domains. We explore two paths for predicting review helpfulness in a general setting: one is by tailoring existing helpfulness prediction techniques to a new review domain; the other is by using a general representation of review content that reflects review helpfulness across domains. For the first one, we explore educational peer reviews and show how peer-review domain knowledge can be introduced to a helpfulness model developed for product reviews to improve prediction performance. For the second one, we characterize review language usage, content diversity and helpfulness-related topics with respect to different content sources using computational linguistic features. For review summarization, we propose to leverage user-provided helpfulness assessment during content selection in two ways: 1) using the review-level helpfulness ratings directly to filter out unhelpful reviews, 2) developing sentence-level helpfulness features via supervised topic modeling for sentence selection. As a demonstration, we implement our methods based on an extractive multi-document summarization framework and evaluate them in three user studies. Results show that our helpfulness-guided summarizers outperform the baseline in both human and automated evaluation for camera reviews and movie reviews. While for educational peer reviews, the preference for helpfulness depends on student writing performance and prior teaching experience

Using Argument-based Features to Predict and Analyse Review Helpfulness

We study the helpful product reviews identification problem in this paper. We observe that the evidence-conclusion discourse relations, also known as arguments, often appear in product reviews, and we hypothesise that some argument-based features, e.g. the percentage of argumentative sentences, the evidences-conclusions ratios, are good indicators of helpful reviews. To validate this hypothesis, we manually annotate arguments in 110 hotel reviews, and investigate the effectiveness of several combinations of argument-based features. Experiments suggest that, when being used together with the argument-based features, the state-of-the-art baseline features can enjoy a performance boost (in terms of F1) of 11.01\% in average.Comment: 6 pages, EMNLP201

Using Argument-based Features to Predict and Analyse Review Helpfulness

We study the helpful product reviews identification problem in this paper. We observe that the evidence-conclusion discourse relations, also known as arguments, often appear in product reviews, and we hypothesise that some argument-based features, e.g. the percentage of argumentative sentences, the evidences-conclusions ratios, are good indicators of helpful reviews. To validate this hypothesis, we manually annotate arguments in 110 hotel reviews, and investigate the effectiveness of several combinations of argument-based features. Experiments suggest that, when being used together with the argument-based features, the state-of-the-art baseline features can enjoy a performance boost (in terms of F1) of 11.01\% in average.Comment: 6 pages, EMNLP201

A Combinatorial Approach for Predicting Online Review Helpfulness of Indian Online Travel Agencies

Online user reviews are quite popular in social media, e-commerce and review websites. It is commonly referred as word of mouth which provides positive and negative messages from users about products and services. It helps users to get insights through review ratings and subjective feedback. As the volumes of reviews are high, it makes it harder for users to identify the helpfulness upfront. In general helpfulness rating is provided by the users who read the review, but many reviews still stay unrated. In this paper, we propose an approach of predicting helpfulness of such reviews from mouthshut.com using a combinatorial approach of empirical analysis and naïve Bayes machine learning method. The data set is chosen for Indian Online Travel Agencies (OTA) namely Makemytrip, Cleartrip, Yatra, Goibibo, and Expedia India. A detailed experiment is conducted and results are discussed by analyzing review metadata characteristics

Predicting Product Review Helpfulness Using Machine Learning and Specialized Classification Models

In this paper we focus on automatically classifying product reviews as either helpful or unhelpful using machine learning techniques, namely, SVM classifiers. Using LIBSVM and a set of Amazon product reviews from 25 product categories, we train models for each category to determine if a review will be helpful or unhelpful. Previous work has focused on training one classifier for all reviews in the data set, but we hypothesize that a distinct model for each of the 25 product types available in the review dataset will improve the accuracy of classification. ! Furthermore, we develop a framework to inform authors on the fly if their review is predicted to be of great use (helpful) to other readers, with the assumption that authors are more likely to rethink their review post and amend it to be of maximum utility to other readers when given some feedback on whether or not it will be found helpful or unhelpful. ! Using past research as a baseline, we find that specialized SVM classifiers outperform higher level models of review helpfulness prediction