Where the earth is flat and 9/11 is an inside job: A comparative algorithm audit of conspiratorial information in web search results

Web search engines are important online information intermediaries that are frequently used and highly trusted by the public despite multiple evidence of their outputs being subjected to inaccuracies and biases. One form of such inaccuracy, which so far received little scholarly attention, is the presence of conspiratorial information, namely pages promoting conspiracy theories. We address this gap by conducting a comparative algorithm audit to examine the distribution of conspiratorial information in search results across five search engines: Google, Bing, DuckDuckGo, Yahoo and Yandex. Using a virtual agent-based infrastructure, we systematically collect search outputs for six conspiracy theory-related queries (“flat earth”, “new world order”, “qanon”, “9/11”, “illuminati”, “george soros”) across three locations (two in the US and one in the UK) and two waves (March and May 2021). We find that all search engines except Google consistently displayed conspiracy-promoting results and returned links to conspiracy-dedicated websites, with variations across queries. Most conspiracy-promoting results came from social media and conspiracy-dedicated websites while conspiracy-debunking information was shared by scientific websites and legacy media. These observations are consistent across different locations and time periods highlighting the possibility that some engines systematically prioritize conspiracy-promoting content

Data privacy by design: digital infrastructures for clinical collaborations

The clinical sciences have arguably the most stringent security demands on the adoption and roll-out of collaborative e-Infrastructure solutions such as those based upon Grid-based middleware. Experiences from the Medical Research Council (MRC) funded Virtual Organisations for Trials and Epidemiological Studies (VOTES) project and numerous other real world security driven projects at the UK e-Science National e-Science Centre (NeSC – www.nesc.ac.uk) have shown that whilst advanced Grid security and middleware solutions now offer capabilities to address many of the distributed data and security challenges in the clinical domain, the real clinical world as typified by organizations such as the National Health Service (NHS) in the UK are extremely wary of adoption of such technologies: firewalls; ethics; information governance, software validation, and the actual realities of existing infrastructures need to be considered from the outset. Based on these experiences we present a novel data linkage and anonymisation infrastructure that has been developed with close co-operation of the various stakeholders in the clinical domain (including the NHS) that addresses their concerns and satisfies the needs of the academic clinical research community. We demonstrate the implementation of this infrastructure through a representative clinical study on chronic diseases in Scotland

Entry and Patenting in the Software Industry

To what extent are firms kept out of a market by patents covering related technologies? Do patents held by potential entrants make it easier to enter markets? We estimate the empirical relationship between market entry and patents for 27 narrowly defined categories of software products during the period 1990-2004. Controlling for demand, market structure, average patent quality, and other factors, we find that a 10% increase in the number of patents relevant to market reduces the rate of entry by 3-8%, and this relationship intensified following expansions in the patentability of software in the mid-1990s. However, potential entrants with patent applications relevant to a market are more likely to enter it. Finally, patents appear to substitute for complementary assets in the entry process, as patents have both greater entry-deterring and entry-promoting effects for firms without prior experience in other markets.

Scaling up search engine audits: Practical insights for algorithm auditing

Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter and rank the large and dynamic amount of information available on the Internet. Among several methodologies to perform such audits, virtual agents stand out because they offer the ability to perform systematic experiments, simulating human behaviour without the associated costs of recruiting participants. Motivated by the importance of research transparency and replicability of results, this article focuses on the challenges of such an approach. It provides methodological details, recommendations, lessons learned and limitations based on our experience of setting up experiments for eight search engines (including main, news, image and video sections) with hundreds of virtual agents placed in different regions. We demonstrate the successful performance of our research infrastructure across multiple data collections, with diverse experimental designs, and point to different changes and strategies that improve the quality of the method. We conclude that virtual agents are a promising venue for monitoring the performance of algorithms across long periods of time, and we hope that this article can serve as a basis for further research in this area

A bayesian approach for on-line max and min auditing

In this paper we consider the on-line max and min query auditing problem: given a private association between fields in a data set, a sequence of max and min queries that have already been posed about the data, their corresponding answers and a new query, deny the answer if a private information is inferred or give the true answer otherwise. We give a probabilistic definition of privacy and demonstrate that max and min queries, without “no duplicates”assumption, can be audited by means of a Bayesian network. Moreover, we show how our auditing approach is able to manage user prior-knowledge

XRay: Enhancing the Web's Transparency with Differential Correlation

Today's Web services - such as Google, Amazon, and Facebook - leverage user data for varied purposes, including personalizing recommendations, targeting advertisements, and adjusting prices. At present, users have little insight into how their data is being used. Hence, they cannot make informed choices about the services they choose. To increase transparency, we developed XRay, the first fine-grained, robust, and scalable personal data tracking system for the Web. XRay predicts which data in an arbitrary Web account (such as emails, searches, or viewed products) is being used to target which outputs (such as ads, recommended products, or prices). XRay's core functions are service agnostic and easy to instantiate for new services, and they can track data within and across services. To make predictions independent of the audited service, XRay relies on the following insight: by comparing outputs from different accounts with similar, but not identical, subsets of data, one can pinpoint targeting through correlation. We show both theoretically, and through experiments on Gmail, Amazon, and YouTube, that XRay achieves high precision and recall by correlating data from a surprisingly small number of extra accounts.Comment: Extended version of a paper presented at the 23rd USENIX Security Symposium (USENIX Security 14

Process Performance Analysis in Large-Scale Systems Integrating Different Sources of Information

Process auditing using historical data can identify causes for poor performance and reveal opportunities to improve process operation. To date, the data used has been limited to process measurements; however other sources hold complementary information about the process behavior. This paper proposes a new approach to root-cause diagnosis, which also takes advantage of the information in utility, mechanical and electrical data, alarms and diagrams. Its benefit is demonstrated in an industrial case study, by tackling an important challenge in root-cause analysis: large-scale systems. This paper also defines specifications for a semi-automated tool to implement the proposed approach. © 2012 IFAC