Automated Discovery of Internet Censorship by Web Crawling

Censorship of the Internet is widespread around the world. As access to the web becomes increasingly ubiquitous, filtering of this resource becomes more pervasive. Transparency about specific content that citizens are denied access to is atypical. To counter this, numerous techniques for maintaining URL filter lists have been proposed by various individuals and organisations that aim to empirical data on censorship for benefit of the public and wider censorship research community. We present a new approach for discovering filtered domains in different countries. This method is fully automated and requires no human interaction. The system uses web crawling techniques to traverse between filtered sites and implements a robust method for determining if a domain is filtered. We demonstrate the effectiveness of the approach by running experiments to search for filtered content in four different censorship regimes. Our results show that we perform better than the current state of the art and have built domain filter lists an order of magnitude larger than the most widely available public lists as of Jan 2018. Further, we build a dataset mapping the interlinking nature of blocked content between domains and exhibit the tightly networked nature of censored web resources

Internet Filters: A Public Policy Report (Second edition; fully revised and updated)

No sooner was the Internet upon us than anxiety arose over the ease of accessing pornography and other controversial content. In response, entrepreneurs soon developed filtering products. By the end of the decade, a new industry had emerged to create and market Internet filters....Yet filters were highly imprecise from the beginning. The sheer size of the Internet meant that identifying potentially offensive content had to be done mechanically, by matching "key" words and phrases; hence, the blocking of Web sites for "Middlesex County," or words such as "magna cum laude". Internet filters are crude and error-prone because they categorize expression without regard to its context, meaning, and value. Yet these sweeping censorship tools are now widely used in companies, homes, schools, and libraries. Internet filters remain a pressing public policy issue to all those concerned about free expression, education, culture, and democracy. This fully revised and updated report surveys tests and studies of Internet filtering products from the mid-1990s through 2006. It provides an essential resource for the ongoing debate

FilteredWeb: A Framework for the Automated Search-Based Discovery of Blocked URLs

Various methods have been proposed for creating and maintaining lists of potentially filtered URLs to allow for measurement of ongoing internet censorship around the world. Whilst testing a known resource for evidence of filtering can be relatively simple, given appropriate vantage points, discovering previously unknown filtered web resources remains an open challenge. We present a new framework for automating the process of discovering filtered resources through the use of adaptive queries to well-known search engines. Our system applies information retrieval algorithms to isolate characteristic linguistic patterns in known filtered web pages; these are then used as the basis for web search queries. The results of these queries are then checked for evidence of filtering, and newly discovered filtered resources are fed back into the system to detect further filtered content. Our implementation of this framework, applied to China as a case study, shows that this approach is demonstrably effective at detecting significant numbers of previously unknown filtered web pages, making a significant contribution to the ongoing detection of internet filtering as it develops. Our tool is currently deployed and has been used to discover 1355 domains that are poisoned within China as of Feb 2017 - 30 times more than are contained in the most widely-used public filter list. Of these, 759 are outside of the Alexa Top 1000 domains list, demonstrating the capability of this framework to find more obscure filtered content. Further, our initial analysis of filtered URLs, and the search terms that were used to discover them, gives further insight into the nature of the content currently being blocked in China.Comment: To appear in "Network Traffic Measurement and Analysis Conference 2017" (TMA2017

Institutionalized Algorithmic Enforcement—The Pros and Cons of the EU Approach to UGC Platform Liability

Algorithmic copyright enforcement – the use of automated filtering tools to detect infringing content before it appears on the internet – has a deep impact on the freedom of users to upload and share information. Instead of presuming that user-generated content ("UGC") does not amount to infringement unless copyright owners take action and provide proof, the default position of automated filtering systems is that every upload is suspicious and that copyright owners are entitled to ex ante control over the sharing of information online. If platform providers voluntarily introduce algorithmic enforcement measures, this may be seen as a private decision following from the freedom of companies to run their business as they wish. If, however, copyright legislation institutionalizes algorithmic enforcement and imposes a legal obligation on platform providers to employ automated filtering tools, the law itself transforms copyright into a censorship and filtering instrument. Nonetheless, the new EU Directive on Copyright in the Digital Single Market (“DSM Directive”) follows this path and requires the employment of automated filtering tools to ensure that unauthorized protected content does not populate UGC platforms. The new EU rules on UGC licensing and screening will inevitably lead to the adoption of algorithmic enforcement measures in practice. Without automated content control, UGC platforms will be unable to escape liability for infringing user uploads. To provide a complete picture, however, it is important to also shed light on counterbalances which may distinguish this new, institutionalized form of algorithmic enforcement from known content filtering tools that have evolved as voluntary measures in the private sector. The DSM Directive underlines the necessity to safeguard user freedoms that support transformative, creative remixes and mash-ups of pre-existing content. This feature of the new legislation may offer important incentives to develop algorithmic tools that go beyond the mere identification of unauthorized takings from protected works. It has the potential to encourage content assessment mechanisms that factor the degree of transformative effort and user creativity into the equation. As a result, more balanced content filtering tools may emerge in the EU. Against this background, the analysis shows that the new EU legislation not only escalates the use of algorithmic enforcement measures that already commenced in the private sector years ago. If rightly implemented, it may also add an important nuance to existing content identification tools and alleviate the problems arising from reliance on automated filtering mechanisms

Access Controlled

Reports on a new generation of Internet controls that establish a new normative terrain in which surveillance and censorship are routine. Internet filtering, censorship of Web content, and online surveillance are increasing in scale, scope, and sophistication around the world, in democratic countries as well as in authoritarian states. The first generation of Internet controls consisted largely of building firewalls at key Internet gateways; China's famous “Great Firewall of China” is one of the first national Internet filtering systems. Today the new tools for Internet controls that are emerging go beyond mere denial of information. These new techniques, which aim to normalize (or even legalize) Internet control, include targeted viruses and the strategically timed deployment of distributed denial-of-service (DDoS) attacks, surveillance at key points of the Internet's infrastructure, take-down notices, stringent terms of usage policies, and national information shaping strategies. Access Controlled reports on this new normative terrain. The book, a project from the OpenNet Initiative (ONI), a collaboration of the Citizen Lab at the University of Toronto's Munk Centre for International Studies, Harvard's Berkman Center for Internet and Society, and the SecDev Group, offers six substantial chapters that analyze Internet control in both Western and Eastern Europe and a section of shorter regional reports and country profiles drawn from material gathered by the ONI around the world through a combination of technical interrogation and field research methods

Systemization of Pluggable Transports for Censorship Resistance

An increasing number of countries implement Internet censorship at different scales and for a variety of reasons. In particular, the link between the censored client and entry point to the uncensored network is a frequent target of censorship due to the ease with which a nation-state censor can control it. A number of censorship resistance systems have been developed thus far to help circumvent blocking on this link, which we refer to as link circumvention systems (LCs). The variety and profusion of attack vectors available to a censor has led to an arms race, leading to a dramatic speed of evolution of LCs. Despite their inherent complexity and the breadth of work in this area, there is no systematic way to evaluate link circumvention systems and compare them against each other. In this paper, we (i) sketch an attack model to comprehensively explore a censor's capabilities, (ii) present an abstract model of a LC, a system that helps a censored client communicate with a server over the Internet while resisting censorship, (iii) describe an evaluation stack that underscores a layered approach to evaluate LCs, and (iv) systemize and evaluate existing censorship resistance systems that provide link circumvention. We highlight open challenges in the evaluation and development of LCs and discuss possible mitigations.Comment: Content from this paper was published in Proceedings on Privacy Enhancing Technologies (PoPETS), Volume 2016, Issue 4 (July 2016) as "SoK: Making Sense of Censorship Resistance Systems" by Sheharbano Khattak, Tariq Elahi, Laurent Simon, Colleen M. Swanson, Steven J. Murdoch and Ian Goldberg (DOI 10.1515/popets-2016-0028

The Impact of Information Security Technologies Upon Society

This paper's aims are concerned with the effects of information security technologies upon society in general and civil society organisations in particular. Information security mechanisms have the potential to act as enablers or disablers for the work of civil society groups. Recent increased emphasis on national security issues by state actors, particularly 'anti-terrorism' initiatives, have resulted in legislative instruments that impinge upon the civil liberties of many citizens and have the potential to restrict the free flow of information vital for civil society actors. The nascent area of cyberactivism, or hactivism, is at risk of being labelled cyberterrorism, with the accompanying change of perception from a legitimate form of electronic civil disobedience to an abhorrent crime. Biometric technology can be an invasive intrusion into citizens' privacy. Internet censorship and surveillance is widespread and increasing. These implementations of information security technology are becoming more widely deployed with profound implications for the type of societies that will result