FilteredWeb: A Framework for the Automated Search-Based Discovery of Blocked URLs

Various methods have been proposed for creating and maintaining lists of potentially filtered URLs to allow for measurement of ongoing internet censorship around the world. Whilst testing a known resource for evidence of filtering can be relatively simple, given appropriate vantage points, discovering previously unknown filtered web resources remains an open challenge. We present a new framework for automating the process of discovering filtered resources through the use of adaptive queries to well-known search engines. Our system applies information retrieval algorithms to isolate characteristic linguistic patterns in known filtered web pages; these are then used as the basis for web search queries. The results of these queries are then checked for evidence of filtering, and newly discovered filtered resources are fed back into the system to detect further filtered content. Our implementation of this framework, applied to China as a case study, shows that this approach is demonstrably effective at detecting significant numbers of previously unknown filtered web pages, making a significant contribution to the ongoing detection of internet filtering as it develops. Our tool is currently deployed and has been used to discover 1355 domains that are poisoned within China as of Feb 2017 - 30 times more than are contained in the most widely-used public filter list. Of these, 759 are outside of the Alexa Top 1000 domains list, demonstrating the capability of this framework to find more obscure filtered content. Further, our initial analysis of filtered URLs, and the search terms that were used to discover them, gives further insight into the nature of the content currently being blocked in China.Comment: To appear in "Network Traffic Measurement and Analysis Conference 2017" (TMA2017

Few Throats to Choke: On the Current Structure of the Internet

The original design of the Internet was as a resilient, distributed system, able to route around (and therefore recover from) massive disruption - up to and including nuclear war. However, network effects and business decisions (e.g. the pur- chase of GlobalCrossing by Level-3) have led to centralization of routing power. This is not merely an academic issue; it has practical implications, such as whether the citizens of a country may be subject to censorship by an “upstream” ISP in some other country, that controls its entire access to the Internet. In this paper, we examine the extent of routing centralization in the Internet; identify the major players who control the “Internet backbone”; and point out how many these are, in fact, under the jurisdiction of censorious countries. We also measure the collateral damage caused by censorship, particularly by the two largest Internet-using nations, China and India

Mending Wall: On the Implementation of Censorship in India

This paper presents a study of the Internet infrastructure in India from the point of view of censorship. First, we show that the current state of affairs — where each ISP implements its own content filters (nominally as per a governmental blacklist) — results in dramatic differences in the censorship experienced by customers. In practice, a well-informed Indian citizen can escape censorship through a judicious choice of service provider. We then consider the question of whether India might potentially follow the Chinese model and institute a single, government-controlled filter. This would not be difficult, as the Indian Internet is quite centralized already. A few “key” ASes (≈ 1% of Indian ASes) collectively intercept ≈ 95% of paths to the censored sites we sample in our study, and also to all publicly-visible DNS servers. 5, 000 routers spanning these key ASes would suffice to carry out IP or DNS filtering for the entire country; ≈ 70% of these routers belong to only two private ISPs. If the government is willing to employ more powerful measures, such as an IP Prefix Hijacking attack, any one of several key ASes can censor traffic for nearly all Indian users. Finally, we demonstrate that such federated censorship by India would cause substantial collateral damage to non-Indian ASes whose traffic passes through Indian cyberspace (which do not legally come under Indian jurisdiction at all)

Systematizing Decentralization and Privacy: Lessons from 15 Years of Research and Deployments

Decentralized systems are a subset of distributed systems where multiple authorities control different components and no authority is fully trusted by all. This implies that any component in a decentralized system is potentially adversarial. We revise fifteen years of research on decentralization and privacy, and provide an overview of key systems, as well as key insights for designers of future systems. We show that decentralized designs can enhance privacy, integrity, and availability but also require careful trade-offs in terms of system complexity, properties provided, and degree of decentralization. These trade-offs need to be understood and navigated by designers. We argue that a combination of insights from cryptography, distributed systems, and mechanism design, aligned with the development of adequate incentives, are necessary to build scalable and successful privacy-preserving decentralized systems