186 research outputs found
Evaluating Conversational Recommender Systems: A Landscape of Research
Conversational recommender systems aim to interactively support online users
in their information search and decision-making processes in an intuitive way.
With the latest advances in voice-controlled devices, natural language
processing, and AI in general, such systems received increased attention in
recent years. Technically, conversational recommenders are usually complex
multi-component applications and often consist of multiple machine learning
models and a natural language user interface. Evaluating such a complex system
in a holistic way can therefore be challenging, as it requires (i) the
assessment of the quality of the different learning components, and (ii) the
quality perception of the system as a whole by users. Thus, a mixed methods
approach is often required, which may combine objective (computational) and
subjective (perception-oriented) evaluation techniques. In this paper, we
review common evaluation approaches for conversational recommender systems,
identify possible limitations, and outline future directions towards more
holistic evaluation practices
INFACT: An Online Human Evaluation Framework for Conversational Recommendation
Conversational recommender systems (CRS) are interactive agents that support
their users in recommendation-related goals through multi-turn conversations.
Generally, a CRS can be evaluated in various dimensions. Today's CRS mainly
rely on offline(computational) measures to assess the performance of their
algorithms in comparison to different baselines. However, offline measures can
have limitations, for example, when the metrics for comparing a newly generated
response with a ground truth do not correlate with human perceptions, because
various alternative generated responses might be suitable too in a given dialog
situation. Current research on machine learning-based CRS models therefore
acknowledges the importance of humans in the evaluation process, knowing that
pure offline measures may not be sufficient in evaluating a highly interactive
system like a CRS.Comment: 6 pages, 2 figures
Combining Spreadsheet Smells for Improved Fault Prediction
Spreadsheets are commonly used in organizations as a programming tool for
business-related calculations and decision making. Since faults in spreadsheets
can have severe business impacts, a number of approaches from general software
engineering have been applied to spreadsheets in recent years, among them the
concept of code smells. Smells can in particular be used for the task of fault
prediction. An analysis of existing spreadsheet smells, however, revealed that
the predictive power of individual smells can be limited. In this work we
therefore propose a machine learning based approach which combines the
predictions of individual smells by using an AdaBoost ensemble classifier.
Experiments on two public datasets containing real-world spreadsheet faults
show significant improvements in terms of fault prediction accuracy.Comment: 4 pages, 1 figure, to be published in 40th International Conference
on Software Engineering: New Ideas and Emerging Results Trac
INFORMATION QUALITY ASSESSMENT: VALIDATING MEASUREMENT DIMENSIONS AND PROCESSES
Over the last two decades information quality has emerged as a critical concern for most organisations. Foremost research provides several approaches to measure information quality and many case studies constantly illustrate the difficulties in assessing information quality. In this paper, we tackle the problem of assessing information quality and we propose a framework to implement information quality assessment in practice. Our framework incorporates two major components: a set of valid measurement dimensions and a measurement process. We have tested the validity, reliability and usefulness of the dimensions and applied the measurement process to an example dataset. In addition, our study demonstrates typical information quality problems in the example dataset and their potential impact to organisations
A Survey on Point-of-Interest Recommendations Leveraging Heterogeneous Data
Tourism is an important application domain for recommender systems. In this
domain, recommender systems are for example tasked with providing personalized
recommendations for transportation, accommodation, points-of-interest (POIs),
or tourism services. Among these tasks, in particular the problem of
recommending POIs that are of likely interest to individual tourists has gained
growing attention in recent years. Providing POI recommendations to tourists
\emph{during their trip} can however be especially challenging due to the
variability of the users' context. With the rapid development of the Web and
today's multitude of online services, vast amounts of data from various sources
have become available, and these heterogeneous data sources represent a huge
potential to better address the challenges of in-trip POI recommendation
problems. In this work, we provide a comprehensive survey of published research
on POI recommendation between 2017 and 2022 from the perspective of
heterogeneous data sources. Specifically, we investigate which types of data
are used in the literature and which technical approaches and evaluation
methods are predominant. Among other aspects, we find that today's research
works often focus on a narrow range of data sources, leaving great potential
for future works that better utilize heterogeneous data sources and diverse
data types for improved in-trip recommendations.Comment: 35 pages, 19 figure
Semi-supervised Adversarial Learning for Complementary Item Recommendation
Complementary item recommendations are a ubiquitous feature of modern
e-commerce sites. Such recommendations are highly effective when they are based
on collaborative signals like co-purchase statistics. In certain online
marketplaces, however, e.g., on online auction sites, constantly new items are
added to the catalog. In such cases, complementary item recommendations are
often based on item side-information due to a lack of interaction data. In this
work, we propose a novel approach that can leverage both item side-information
and labeled complementary item pairs to generate effective complementary
recommendations for cold items, i.e., for items for which no co-purchase
statistics yet exist. Given that complementary items typically have to be of a
different category than the seed item, we technically maintain a latent space
for each item category. Simultaneously, we learn to project distributed item
representations into these category spaces to determine suitable
recommendations. The main learning process in our architecture utilizes labeled
pairs of complementary items. In addition, we adopt ideas from Cycle Generative
Adversarial Networks (CycleGAN) to leverage available item information even in
case no labeled data exists for a given item and category. Experiments on three
e-commerce datasets show that our method is highly effective.Comment: ACM Web Conference 202
- …