An analysis of popularity biases in recommender system evaluation and algorithms

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de Lectura: 03-10-2019Las tecnologías de recomendación han ido progresivamente extendiendo su presencia en las aplicaciones y servicios de uso diario. Los sistemas de recomendación buscan realizar sugerencias individualizadas de productos u opciones que los usuarios puedan encontrar interesantes o útiles. Implícita en el concepto de recomendación está la idea de que las sugerencias más satisfactorias para cada usuario son aquellas que tienen en cuenta sus gustos particulares, por lo que cabría esperar que los algoritmos de recomendación más eficaces sean los más personalizados. Sin embargo, se ha observado recientemente que recomendar simplemente los productos más populares no resulta una estrategia mucho peor que los mejores y más sofisticados algoritmos personalizados, y más aún, que estos tienden a sesgar sus recomendaciones hacia opciones mayoritarias. Por todo ello, es rele-vante entender en qué medida y bajo qué circunstancias es la popularidad una señal real-mente efectiva a la hora de recomendar, y si su aparente efectividad se debe a la existencia de ciertos sesgos en las metodologías de evaluación offline actuales, como todo parece indicar, o no. En esta tesis abordamos esta cuestión desde un punto de vista plenamente formal, identificando los factores que pueden determinar la respuesta y modelizándolos en térmi-nos de dependencias probabilísticas entre variables aleatorias, tales como la votación, el descubrimiento y la relevancia. De esta forma, caracterizamos situaciones concretas que garantizan que la popularidad sea efectiva o que no lo sea, y establecemos las condiciones bajo las cuales pueden existir contradicciones entre el acierto observado y el real. Las principales conclusiones hacen referencia a escenarios simplificados prototípicos, más allá de los cuales el análisis formal concluye que cualquier resultado es posible. Para profun-dizar en el escenario general sin suposiciones tan simplificadas, estudiamos un caso parti-cular donde el descubrimiento de ítems es consecuencia de la interacción entre usuarios en una red social. Además, en esta tesis proporcionamos una explicación formal del sesgo de populari-dad que presentan los algoritmos de filtrado colaborativo. Para ello, desarrollamos una versión probabilística del algoritmo de vecinos próximos kNN. Dicha versión evidencia además la condición fundamental que hace que kNN produzca recomendaciones perso-nalizadas y se diferencie de la popularidad pura

Conceptualizing the Electronic Word-of-Mouth Process: What We Know and Need to Know About eWOM Creation, Exposure, and Evaluation

Electronic word of mouth (eWOM) is a prevalent consumer practice that has undeniable effects on the company bottom line, yet it remains an over-labeled and under-theorized concept. Thus, marketers could benefit from a practical, science-based roadmap to maximize its business value. Building on the consumer motivation–opportunity–ability framework, this study conceptualizes three distinct stages in the eWOM process: eWOM creation, eWOM exposure, and eWOM evaluation. For each stage, we adopt a dual lens—from the perspective of the consumer (who sends and receives eWOM) and that of the marketer (who amplifies and manages eWOM for business results)—to synthesize key research insights and propose a research agenda based on a multidisciplinary systematic review of 1050 academic publications on eWOM published between 1996 and 2019. We conclude with a discussion of the future of eWOM research and practice

Computational Methods for Medical and Cyber Security

Over the past decade, computational methods, including machine learning (ML) and deep learning (DL), have been exponentially growing in their development of solutions in various domains, especially medicine, cybersecurity, finance, and education. While these applications of machine learning algorithms have been proven beneficial in various fields, many shortcomings have also been highlighted, such as the lack of benchmark datasets, the inability to learn from small datasets, the cost of architecture, adversarial attacks, and imbalanced datasets. On the other hand, new and emerging algorithms, such as deep learning, one-shot learning, continuous learning, and generative adversarial networks, have successfully solved various tasks in these fields. Therefore, applying these new methods to life-critical missions is crucial, as is measuring these less-traditional algorithms' success when used in these fields

Health Misinformation in Search and Social Media

People increasingly rely on the Internet in order to search for and share health-related information. Indeed, searching for and sharing information about medical treatments are among the most frequent uses of online data. While this is a convenient and fast method to collect information, online sources may contain incorrect information that has the potential to cause harm, especially if people believe what they read without further research or professional medical advice. The goal of this thesis is to address the misinformation problem in two of the most commonly used online services: search engines and social media platforms. We examined how people use these platforms to search for and share health information. To achieve this, we designed controlled laboratory user studies and employed large-scale social media data analysis tools. The solutions proposed in this thesis can be used to build systems that better support people's health-related decisions. The techniques described in this thesis addressed online searching and social media sharing in the following manner. First, with respect to search engines, we aimed to determine the extent to which people can be influenced by search engine results when trying to learn about the efficacy of various medical treatments. We conducted a controlled laboratory study wherein we biased the search results towards either correct or incorrect information. We then asked participants to determine the efficacy of different medical treatments. Results showed that people were significantly influenced both positively and negatively by search results bias. More importantly, when the subjects were exposed to incorrect information, they made more incorrect decisions than when they had no interaction with the search results. Following from this work, we extended the study to gain insights into strategies people use during this decision-making process, via the think-aloud method. We found that, even with verbalization, people were strongly influenced by the search results bias. We also noted that people paid attention to what the majority states, authoritativeness, and content quality when evaluating online content. Understanding the effects of cognitive biases that can arise during online search is a complex undertaking because of the presence of unconscious biases (such as the search results ranking) that the think-aloud method fails to show. Moving to social media, we first proposed a solution to detect and track misinformation in social media. Using Zika as a case study, we developed a tool for tracking misinformation on Twitter. We collected 13 million tweets regarding the Zika outbreak and tracked rumors outlined by the World Health Organization and the Snopes fact-checking website. We incorporated health professionals, crowdsourcing, and machine learning to capture health-related rumors as well as clarification communications. In this way, we illustrated insights that the proposed tools provide into potentially harmful information on social media, allowing public health researchers and practitioners to respond with targeted and timely action. From identifying rumor-bearing tweets, we examined individuals on social media who are posting questionable health-related information, in particular those promoting cancer treatments that have been shown to be ineffective. Specifically, we studied 4,212 Twitter users who have posted about one of 139 ineffective ``treatments'' and compared them to a baseline of users generally interested in cancer. Considering features that capture user attributes, writing style, and sentiment, we built a classifier that is able to identify users prone to propagating such misinformation. This classifier achieved an accuracy of over 90%, providing a potential tool for public health officials to identify such individuals for preventive intervention

Transforming Conservation

There are severe problems with the decision-making processes currently widely used, leading to ineffective use of evidence, faulty decisions, wasting of resources and the erosion of public and political support. In this book an international team of experts provide solutions. The transformation suggested includes rethinking how evidence is assessed, combined, communicated and used in decision-making; using effective methods when asking experts to make judgements (i.e. avoiding just asking an expert or a group of experts!); using a structured process for making decisions that incorporate the evidence and having effective processes for learning from actions. In each case, the specific problem with decision making is described with a range of practical solutions. Adopting this approach to decision-making requires societal change so detailed suggestions are made for transforming organisations, governments, businesses, funders and philanthropists. The practical suggestions include twelve downloadable checklists. The vision of the authors is to transform conservation so it is more effective, more cost-efficient, learns from practice and is more attractive to funders. However, the lessons of this important book go well beyond conservation to decision-makers in any field

Designing technologies for exposure to diverse opinions

Exposure to diverse opinions can help individuals develop accurate beliefs, make better decisions, and become more understanding and tolerant persons. It is also necessary for the governance of a stable and democratic society. However, the exposure is often limited by people's natural tendency towards selective exposure—preferential seeking of confirmatory over challenging information. This has prompted many scholars to warn about the rise of "echo chambers" and "filter bubbles" online, with individuals' easy control over information exposure enabled by digital technologies. Such concern has motivated HCI researchers to study a class of diversity-enhancing technologies—information and social technologies that host diverse viewpoints and take increasing users' exposure to information that challenges their existing beliefs as a design goal. In this dissertation, I seek to answer the following question: What kind of design features can nudge users to be exposed to more attitude-challenging information? To complement the current technical-HCI approaches, I focus on bridging social science theories on selective exposure and design guidelines for diversity-enhancing technologies. Specifically, the central objective of this dissertation is to understand the key factors that moderate individuals' propensity to engage in selective exposure in interacting with information and social technologies and apply the knowledge in four aspects of diversity-enhancing technology design: 1) design by enabling the moderators that reduce, and eliminating ones that increase, selective exposure; 2) design for personalization by identifying user groups and use contexts that have varied selective exposure tendencies; 3) design for personalization by tailoring diversity-enhancing designs based on the underlying individual differences; and 4) design beyond individuals by considering the opinion group differences in selective exposure tendencies and the implication for user behaviors and social network structure. This dissertation provides empirical evidence that user behaviors in seeking attitude-relevant information are subject to the influence of various individual and contextual factors and recommends a more personalized approach that carefully controls and leverages these factors to nudge users into more desirable information consumption. It contributes several new lessons for designing technologies that present diverse viewpoints, including a theory-driven guideline for personalizing diversity-enhancing designs, insights on the selective exposure bias in consumer health information seeking, and an exploration of group selective exposure and its implication for social technology design. Perhaps most importantly, the dissertation pinpoints several directions in which selective exposure theories can be applied to the design of diversity-enhancing technologies, which opens up opportunities for developing a unified knowledge framework to push this research field forward

Strategies That Improve UX (User Experience) Design Through Product Innovation

User Experience (UX) design improvement can alter business results. Information technology (IT) company leaders are concerned with UX design improvement, as it is the number one indication of product innovation success and user satisfaction. Grounded in Christensen’s disruptive innovation theory, the purpose of the qualitative single case study was to explore strategies IT company leaders and UX designers used to identify critical UX design elements that lead to improved product innovations. The participants were five IT company leaders and a focus group of four UX designers employed by a sizeable telecom organization in Beijing, China. The data were collected from five semistructured interviews and the focus group discussion. Through thematic analysis, five themes emerged: cultivate a user-centered company culture, improve UX design basic factors, focus on the users, measure UX design key performance indicators, and optimize the UX design process. The primary recommendation is for business leaders to cultivate a user-centered culture through building a user-centered belief across the organization and reaching support from the senior management. The implications for positive social change include the potential to develop useful products that may help users solve real-life problems and enhance their quality of life

CHARACTERIZING ENABLING INNOVATIONS AND ENABLING THINKING

The pursuit of innovation is engrained throughout society whether in business via the introduction of offerings, non-profits in their mission-driven initiatives, universities and agencies in their drive for discoveries and inventions, or governments in their desire to improve the quality of life of their citizens. Yet, despite these pursuits, innovations with long-lasting, significant impact represent an infrequent outcome in most domains. The seemingly random nature of these results stems, in part, from the definitions of innovation and the models based on such definitions. Although there is debate on this topic, a comprehensive and pragmatic perspective developed in this work defines innovation as the introduction of a novel or different idea into practice that has a positive impact on society. To date, models of innovation have focused on, for example, new technological advances, new approaches to connectivity in systems, new conceptual frameworks, or even new dimensions of performance - all effectively building on the first half of the definition of innovation and encouraging its pursuit based on the novelty of ideas. However, as explored herein, achieving profound results by innovating on demand might require a perspective that focuses on the impact of an innovation. In this view, innovation does not only entail doing new things, but consciously driving them towards achieving impact through proactive design behaviors. Explicit consideration of the impact dimension in innovation models has been missing, even though it may arguably be the most important since it represents the outcome of innovation

Transforming Conservation

There are severe problems with the decision-making processes currently widely used, leading to ineffective use of evidence, faulty decisions, wasting of resources and the erosion of public and political support. In this book an international team of experts provide solutions. The transformation suggested includes rethinking how evidence is assessed, combined, communicated and used in decision-making; using effective methods when asking experts to make judgements (i.e. avoiding just asking an expert or a group of experts!); using a structured process for making decisions that incorporate the evidence and having effective processes for learning from actions. In each case, the specific problem with decision making is described with a range of practical solutions. Adopting this approach to decision-making requires societal change so detailed suggestions are made for transforming organisations, governments, businesses, funders and philanthropists. The practical suggestions include twelve downloadable checklists. The vision of the authors is to transform conservation so it is more effective, more cost-efficient, learns from practice and is more attractive to funders. However, the lessons of this important book go well beyond conservation to decision-makers in any field

Proposal for an Organic Web, The missing link between the Web and the Semantic Web, Part 1

A huge amount of information is produced in digital form. The Semantic Web stems from the realisation that dealing efficiently with this production requires getting better at interlinking digital informational resources together. Its focus is on linking data. Linking data isn't enough. We need to provide infrastructural support for linking all sorts of informational resources including resources whose understanding and fine interlinking requires domain-specific human expertise. At times when many problems scale to planetary dimensions, it is essential to scale coordination of information processing and information production, without giving up on expertise and depth of analysis, nor forcing languages and formalisms onto thinkers, decision-makers and innovators that are only suitable to some forms of intelligence. This article makes a proposal in this direction and in line with the idea of interlinking championed by the Semantic Web.Comment: Supplementary material by Guillaume Bouzige and Mathilde Noua