2,279 research outputs found
Automated Crowdturfing Attacks and Defenses in Online Review Systems
Malicious crowdsourcing forums are gaining traction as sources of spreading
misinformation online, but are limited by the costs of hiring and managing
human workers. In this paper, we identify a new class of attacks that leverage
deep learning language models (Recurrent Neural Networks or RNNs) to automate
the generation of fake online reviews for products and services. Not only are
these attacks cheap and therefore more scalable, but they can control rate of
content output to eliminate the signature burstiness that makes crowdsourced
campaigns easy to detect.
Using Yelp reviews as an example platform, we show how a two phased review
generation and customization attack can produce reviews that are
indistinguishable by state-of-the-art statistical detectors. We conduct a
survey-based user study to show these reviews not only evade human detection,
but also score high on "usefulness" metrics by users. Finally, we develop novel
automated defenses against these attacks, by leveraging the lossy
transformation introduced by the RNN training and generation cycle. We consider
countermeasures against our mechanisms, show that they produce unattractive
cost-benefit tradeoffs for attackers, and that they can be further curtailed by
simple constraints imposed by online service providers
Resolving X-Ray Photoelectron Spectra of Ionic Liquids with Difference Spectroscopy
X-ray photoelectron spectroscopy (XPS) is a powerful element-specific technique to determine the composition and chemical state of all elements in an involatile sample. However, for elements such as carbon, the wide variety of chemical states produce complex spectra that are difficult to interpret, consequently concealing important information due to the uncertainty in signal identity. Here we report a process whereby chemical modification of carbon structures with electron withdrawing groups can reveal this information, providing accurate, highly refined fitting models far more complex than previously possible. This method is demonstrated with functionalised ionic liquids bearing chlorine or trifluoromethane groups that shift electron density from targeted locations. By comparing the C 1s spectra of non-functional ionic liquids to their functional analogues, a series of difference spectra can be produced to identify exact binding energies of carbon photoemissions, which can be used to improve the C 1s peak fitting of both samples. Importantly, ionic liquids possess ideal chemical and physical properties, which enhance this methodology to enable significant progress in XPS peak fitting and data interpretation
Evaluating Conversational Recommender Systems via User Simulation
Conversational information access is an emerging research area. Currently,
human evaluation is used for end-to-end system evaluation, which is both very
time and resource intensive at scale, and thus becomes a bottleneck of
progress. As an alternative, we propose automated evaluation by means of
simulating users. Our user simulator aims to generate responses that a real
human would give by considering both individual preferences and the general
flow of interaction with the system. We evaluate our simulation approach on an
item recommendation task by comparing three existing conversational recommender
systems. We show that preference modeling and task-specific interaction models
both contribute to more realistic simulations, and can help achieve high
correlation between automatic evaluation measures and manual human assessments.Comment: Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery
and Data Mining (KDD '20), 202
Estimating Error and Bias in Offline Evaluation Results
Offline evaluations of recommender systems attempt to estimate usersâ satisfaction with recommendations using static data from prior user interactions. These evaluations provide researchers and developers with first approximations of the likely performance of a new system and help weed out bad ideas before presenting them to users. However, offline evaluation cannot accurately assess novel, relevant recommendations, because the most novel items were previously unknown to the user, so they are missing from the historical data and cannot be judged as relevant.
We present a simulation study to estimate the error that such missing data causes in commonly-used evaluation metrics in order to assess its prevalence and impact. We find that missing data in the rating or observation process causes the evaluation protocol to systematically mis-estimate metric values, and in some cases erroneously determine that a popularity-based recommender outperforms even a perfect personalized recommender. Substantial breakthroughs in recommendation quality, therefore, will be difficult to assess with existing offline techniques
Early findings from a large-scale user study of CHESTNUT: Validations and implications
Towards a serendipitous recommender system with user-centred understanding, we have built CHESTNUT , an Information Theory-based Movie Recommender System, which introduced a more comprehensive understanding of the concept. Although off-line evaluations have already demonstrated that CHESTNUT has greatly improved serendip-ity performance, feedback on CHESTNUT from real-world users through online services are still unclear now. In order to evaluate how serendip-itous results could be delivered by CHESTNUT , we consequently designed , organized and conducted large-scale user study, which involved 104 participants from 10 campuses in 3 countries. Our preliminary feedback has shown that, compared with mainstream collaborative filtering techniques, though CHESTNUT limited users' feelings of unex-pectedness to some extent, it showed significant improvement in their feelings about certain metrics being both beneficial and interesting, which substantially increased users' experience of serendipity. Based on them, we have summarized three key takeaways, which would be beneficial for further designs and engineering of serendipitous recommender systems, from our perspective. All details of our large-scale user study could be found at https://github.com/unnc-idl-ucc/Early-Lessons-From-CHESTNU
Nobody cares if you liked Star Wars: KNN graph construction on the cheap
International audienceK-Nearest-Neighbors (KNN) graphs play a key role in a large range of applications. A KNN graph typically connects entities characterized by a set of features so that each entity becomes linked to its k most similar counterparts according to some similarity function. As datasets grow, KNN graphs are unfortunately becoming increasingly costly to construct, and the general approach, which consists in reducing the number of comparisons between entities, seems to have reached its full potential. In this paper we propose to overcome this limit with a simple yet powerful strategy that samples the set of features of each entity and only keeps the least popular features. We show that this strategy outperforms other more straightforward policies on a range of four representative datasets: for instance, keeping the 25 least popular items reduces computational time by up to 63%, while producing a KNN graph close to the ideal one
- âŠ