A study on text-score disagreement in online reviews

In this paper, we focus on online reviews and employ artificial intelligence tools, taken from the cognitive computing field, to help understanding the relationships between the textual part of the review and the assigned numerical score. We move from the intuitions that 1) a set of textual reviews expressing different sentiments may feature the same score (and vice-versa); and 2) detecting and analyzing the mismatches between the review content and the actual score may benefit both service providers and consumers, by highlighting specific factors of satisfaction (and dissatisfaction) in texts. To prove the intuitions, we adopt sentiment analysis techniques and we concentrate on hotel reviews, to find polarity mismatches therein. In particular, we first train a text classifier with a set of annotated hotel reviews, taken from the Booking website. Then, we analyze a large dataset, with around 160k hotel reviews collected from Tripadvisor, with the aim of detecting a polarity mismatch, indicating if the textual content of the review is in line, or not, with the associated score. Using well established artificial intelligence techniques and analyzing in depth the reviews featuring a mismatch between the text polarity and the score, we find that -on a scale of five stars- those reviews ranked with middle scores include a mixture of positive and negative aspects. The approach proposed here, beside acting as a polarity detector, provides an effective selection of reviews -on an initial very large dataset- that may allow both consumers and providers to focus directly on the review subset featuring a text/score disagreement, which conveniently convey to the user a summary of positive and negative features of the review target.Comment: This is the accepted version of the paper. The final version will be published in the Journal of Cognitive Computation, available at Springer via http://dx.doi.org/10.1007/s12559-017-9496-

The effectiveness of manual stretching in the treatment of plantar heel pain: a systematic review

Background: Plantar heel pain is a commonly occurring foot complaint. Stretching is frequently utilised as a treatment, yet a systematic review focusing only on its effectiveness has not been published. This review aimed to assess the effectiveness of stretching on pain and function in people with plantar heel pain. Methods: Medline, EMBASE, CINAHL, AMED, and The Cochrane Library were searched from inception to July 2010. Studies fulfilling the inclusion criteria were independently assessed, and their quality evaluated using the modified PEDro scale. Results: Six studies including 365 symptomatic participants were included. Two compared stretching with a control, one study compared stretching to an alternative intervention, one study compared stretching to both alternative and control interventions, and two compared different stretching techniques and durations. Quality rating on the modified Pedro scale varied from two to eight out of a maximum of ten points. The methodologies and interventions varied significantly between studies, making meta-analysis inappropriate. Most participants improved over the course of the studies, but when stretching was compared to alternative or control interventions, the changes only reached statistical significance in one study that used a combination of calf muscle stretches and plantar fascia stretches in their stretching programme. Another study comparing different stretching techniques, showed a statistically significant reduction in some aspects of pain in favour of plantar fascia stretching over calf stretches in the short term. Conclusions: There were too few studies to assess whether stretching is effective compared to control or other interventions, for either pain or function. However, there is some evidence that plantar fascia stretching may be more effective than Achilles tendon stretching alone in the short-term. Appropriately powered randomised controlled trials, utilizing validated outcome measures, blinded assessors and long-term follow up are needed to assess the efficacy of stretching

Privacy and Usability of Image and Text Based Challenge Questions Authentication in Online Examination

In many online examinations, physical invigilation is often replaced with traditional authentication approaches for student identification. Secure and usable authentication approaches are important for high stake online examinations. A Profile Based Authentication Framework (PBAF) was developed and implemented in a real online learning course embedded with summative online examination. Based on users’ experience of using the PBAF in an online course, online questionnaires were used to collect participants' feedback on effectiveness, layout and appearance, user satisfaction, distraction and privacy concerns. Based on overall findings of the quantitative analysis, there was a positive feedback on the use of a hybrid approach utilizing image and text based challenge questions for better usability. However, the number of questions presented during learning and examination processes were reported to be too many and caused distraction. Participants expressed a degree of concern on sharing personal and academic information with little or no privacy concern on using favorite question

Reputation Agent: Prompting Fair Reviews in Gig Markets

Our study presents a new tool, Reputation Agent, to promote fairer reviews from requesters (employers or customers) on gig markets. Unfair reviews, created when requesters consider factors outside of a worker's control, are known to plague gig workers and can result in lost job opportunities and even termination from the marketplace. Our tool leverages machine learning to implement an intelligent interface that: (1) uses deep learning to automatically detect when an individual has included unfair factors into her review (factors outside the worker's control per the policies of the market); and (2) prompts the individual to reconsider her review if she has incorporated unfair factors. To study the effectiveness of Reputation Agent, we conducted a controlled experiment over different gig markets. Our experiment illustrates that across markets, Reputation Agent, in contrast with traditional approaches, motivates requesters to review gig workers' performance more fairly. We discuss how tools that bring more transparency to employers about the policies of a gig market can help build empathy thus resulting in reasoned discussions around potential injustices towards workers generated by these interfaces. Our vision is that with tools that promote truth and transparency we can bring fairer treatment to gig workers.Comment: 12 pages, 5 figures, The Web Conference 2020, ACM WWW 202

Efficacy of interventions that use apps to improve diet, physical activity and sedentary behaviour : a systematic review

Background: Health and fitness applications (apps) have gained popularity in interventions to improve diet, physical activity and sedentary behaviours but their efficacy is unclear. This systematic review examined the efficacy of interventions that use apps to improve diet, physical activity and sedentary behaviour in children and adults. Methods: Systematic literature searches were conducted in five databases to identify papers published between 2006 and 2016. Studies were included if they used a smartphone app in an intervention to improve diet, physical activity and/or sedentary behaviour for prevention. Interventions could be stand-alone interventions using an app only, or multi-component interventions including an app as one of several intervention components. Outcomes measured were changes in the health behaviours and related health outcomes (i.e., fitness, body weight, blood pressure, glucose, cholesterol, quality of life). Study inclusion and methodological quality were independently assessed by two reviewers. Results: Twenty-seven studies were included, most were randomised controlled trials (n = 19; 70%). Twenty-three studies targeted adults (17 showed significant health improvements) and four studies targeted children (two demonstrated significant health improvements). Twenty-one studies targeted physical activity (14 showed significant health improvements), 13 studies targeted diet (seven showed significant health improvements) and five studies targeted sedentary behaviour (two showed significant health improvements). More studies (n = 12; 63%) of those reporting significant effects detected between-group improvements in the health behaviour or related health outcomes, whilst fewer studies (n = 8; 42%) reported significant within-group improvements. A larger proportion of multi-component interventions (8 out of 13; 62%) showed significant between-group improvements compared to stand-alone app interventions (5 out of 14; 36%). Eleven studies reported app usage statistics, and three of them demonstrated that higher app usage was associated with improved health outcomes. Conclusions: This review provided modest evidence that app-based interventions to improve diet, physical activity and sedentary behaviours can be effective. Multi-component interventions appear to be more effective than standalone app interventions, however, this remains to be confirmed in controlled trials. Future research is needed on the optimal number and combination of app features, behaviour change techniques, and level of participant contact needed to maximise user engagement and intervention efficacy

Active learning in annotating micro-blogs dealing with e-reputation

Elections unleash strong political views on Twitter, but what do people really think about politics? Opinion and trend mining on micro blogs dealing with politics has recently attracted researchers in several fields including Information Retrieval and Machine Learning (ML). Since the performance of ML and Natural Language Processing (NLP) approaches are limited by the amount and quality of data available, one promising alternative for some tasks is the automatic propagation of expert annotations. This paper intends to develop a so-called active learning process for automatically annotating French language tweets that deal with the image (i.e., representation, web reputation) of politicians. Our main focus is on the methodology followed to build an original annotated dataset expressing opinion from two French politicians over time. We therefore review state of the art NLP-based ML algorithms to automatically annotate tweets using a manual initiation step as bootstrap. This paper focuses on key issues about active learning while building a large annotated data set from noise. This will be introduced by human annotators, abundance of data and the label distribution across data and entities. In turn, we show that Twitter characteristics such as the author's name or hashtags can be considered as the bearing point to not only improve automatic systems for Opinion Mining (OM) and Topic Classification but also to reduce noise in human annotations. However, a later thorough analysis shows that reducing noise might induce the loss of crucial information.Comment: Journal of Interdisciplinary Methodologies and Issues in Science - Vol 3 - Contextualisation digitale - 201

A Statistical Measure of a Population's Propensity to Engage in Post-Purchase Online Word-of-Mouth

The emergence of online communities has enabled firms to monitor consumer-generated online word-of-mouth (WOM) in real-time by mining publicly available information from the Internet. A prerequisite for harnessing this new ability is the development of appropriate WOM metrics and the identification of relationships between such metrics and consumer behavior. Along these lines this paper introduces a metric of a purchasing population's propensity to rate a product online. Using data from a popular movie website we find that our metric exhibits several relationships that have been previously found to exist between aspects of a product and consumers' propensity to engage in offline WOM about it. Our study, thus, provides positive evidence for the validity of our metric as a proxy of a population's propensity to engage in post-purchase online WOM. Our results also suggest that the antecedents of offline and online WOM exhibit important similarities.Comment: Published at http://dx.doi.org/10.1214/088342306000000169 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Testing Market Response to Auditor Change Filings: a comparison of machine learning classifiers

The use of textual information contained in company filings with the Securities Exchange Commission (SEC), including annual reports on Form 10-K, quarterly reports on Form 10-Q, and current reports on Form 8-K, has gained the increased attention of finance and accounting researchers. In this paper we use a set of machine learning methods to predict the market response to changes in a firm\u27s auditor as reported in public filings. We vectorize the text of 8-K filings to test whether the resulting feature matrix can explain the sign of the market response to the filing. Specifically, using classification algorithms and a sample consisting of the Item 4.01 text of 8-K documents, which provides information on changes in auditors of companies that are registered with the SEC, we predict the sign of the cumulative abnormal return (CAR) around 8-K filing dates. We report the correct classification performance and time efficiency of the classification algorithms. Our results show some improvement over the naïve classification method

SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods

In the last few years thousands of scientific papers have investigated sentiment analysis, several startups that measure opinions on real data have emerged and a number of innovative products related to this theme have been developed. There are multiple methods for measuring sentiments, including lexical-based and supervised machine learning methods. Despite the vast interest on the theme and wide popularity of some methods, it is unclear which one is better for identifying the polarity (i.e., positive or negative) of a message. Accordingly, there is a strong need to conduct a thorough apple-to-apple comparison of sentiment analysis methods, \textit{as they are used in practice}, across multiple datasets originated from different data sources. Such a comparison is key for understanding the potential limitations, advantages, and disadvantages of popular methods. This article aims at filling this gap by presenting a benchmark comparison of twenty-four popular sentiment analysis methods (which we call the state-of-the-practice methods). Our evaluation is based on a benchmark of eighteen labeled datasets, covering messages posted on social networks, movie and product reviews, as well as opinions and comments in news articles. Our results highlight the extent to which the prediction performance of these methods varies considerably across datasets. Aiming at boosting the development of this research area, we open the methods' codes and datasets used in this article, deploying them in a benchmark system, which provides an open API for accessing and comparing sentence-level sentiment analysis methods

Assessment of sexual difficulties associated with multi-modal treatment for cervical or endometrial cancer: A systematic review of measurement instruments

Background: Practitioners and researchers require an outcome measure that accurately identifies the range of common treatment-induced changes in sexual function and well-being experienced by women after cervical or endometrial cancer. This systematic review critically appraised the measurement properties and clinical utility of instruments validated for the measurement of female sexual dysfunction (FSD) in this clinical population. Methods: A bibliographic database search for questionnaire development or validation papers was completed and methodological quality and measurement properties of selected studies rated using the Consensus-based Standards for the selection of health Measurement Instrument (COSMIN) checklist. Results: 738 articles were screened, 13 articles retrieved for full text assessment and 7 studies excluded, resulting in evaluation of 6 papers; 2 QoL and 4 female sexual morbidity measures. Five of the six instruments omitted one or more dimension of female sexual function and only one instrument explicitly measured distress associated with sexual changes as per DSM V (APA 2013) diagnostic criteria. None of the papers reported measurement error, responsiveness data was available for only two instruments, three papers failed to report on criterion validity, and test-retest reliability reporting was inconsistent. Heterosexual penile-vaginal intercourse remains the dominant sexual activity focus for sexual morbidity PROMS terminology and instruments lack explicit reference to solo or non-coital sexual expression or validation in a non-heterosexual sample. Four out of six instruments included mediating treatment or illness items such as vaginal changes, menopause or altered body image. Conclusions: Findings suggest that the Female Sexual Function Index (FSFI) remains the most robust sexual morbidity outcome measure, for research or clinical use, in sexually active women treated for cervical or endometrial cancer