Measuring internet activity: a (selective) review of methods and metrics

Two Decades after the birth of the World Wide Web, more than two billion people around the world are Internet users. The digital landscape is littered with hints that the affordances of digital communications are being leveraged to transform life in profound and important ways. The reach and influence of digitally mediated activity grow by the day and touch upon all aspects of life, from health, education, and commerce to religion and governance. This trend demands that we seek answers to the biggest questions about how digitally mediated communication changes society and the role of different policies in helping or hindering the beneficial aspects of these changes. Yet despite the profusion of data the digital age has brought upon us—we now have access to a flood of information about the movements, relationships, purchasing decisions, interests, and intimate thoughts of people around the world—the distance between the great questions of the digital age and our understanding of the impact of digital communications on society remains large. A number of ongoing policy questions have emerged that beg for better empirical data and analyses upon which to base wider and more insightful perspectives on the mechanics of social, economic, and political life online. This paper seeks to describe the conceptual and practical impediments to measuring and understanding digital activity and highlights a sample of the many efforts to fill the gap between our incomplete understanding of digital life and the formidable policy questions related to developing a vibrant and healthy Internet that serves the public interest and contributes to human wellbeing. Our primary focus is on efforts to measure Internet activity, as we believe obtaining robust, accurate data is a necessary and valuable first step that will lead us closer to answering the vitally important questions of the digital realm. Even this step is challenging: the Internet is difficult to measure and monitor, and there is no simple aggregate measure of Internet activity—no GDP, no HDI. In the following section we present a framework for assessing efforts to document digital activity. The next three sections offer a summary and description of many of the ongoing projects that document digital activity, with two final sections devoted to discussion and conclusions

Ranking Spaces for Predicting Human Movement in an Urban Environment

A city can be topologically represented as a connectivity graph, consisting of nodes representing individual spaces and links if the corresponding spaces are intersected. It turns out in the space syntax literature that some defined topological metrics can capture human movement rates in individual spaces. In other words, the topological metrics are significantly correlated to human movement rates, and individual spaces can be ranked by the metrics for predicting human movement. However, this correlation has never been well justified. In this paper, we study the same issue by applying the weighted PageRank algorithm to the connectivity graph or space-space topology for ranking the individual spaces, and find surprisingly that (1) the PageRank scores are better correlated to human movement rates than the space syntax metrics, and (2) the underlying space-space topology demonstrates small world and scale free properties. The findings provide a novel justification as to why space syntax, or topological analysis in general, can be used to predict human movement. We further conjecture that this kind of analysis is no more than predicting a drunkard's walking on a small world and scale free network. Keywords: Space syntax, topological analysis of networks, small world, scale free, human movement, and PageRankComment: 11 pages, 5 figures, and 2 tables, English corrections from version 1 to version 2, major changes in the section of introduction from version 2 to

The Cognitive Virtues of Dynamic Networks

For the most part, studies in the network science literature tend to focus on networks whose functional connectivity is largely invariant with respect to some episode of collective information processing. In the real world, however, networks with highly dynamic functional topologies tend to be the norm. In order to improve our understanding of the effect of dynamic networks on collective cognitive processing, we explored the problem-solving abilities of synthetic agents in dynamic networks, where the links between agents were progressively added throughout the problem-solving process. The results support the conclusion that (at least in some task contexts) dynamic networks contribute to a better profile of problem-solving performance compared to static networks (whose topologies are fixed throughout the course of information processing). Furthermore, the results suggest that constructive networks (like those used in the present study) strike a productive balance between autonomy and social influence. When agents are allowed to operate independently at the beginning of a problem-solving process, and then later allowed to communicate, the result is often a better profile of collective performance than if extensive communication had been permitted from the very outset of the problem-solving process. These results are relevant, we suggest, to a range of phenomena, such as groupthink, the common knowledge effect and production blocking, all of which have been observed in group problem-solving contexts

Defining and Evaluating Network Communities based on Ground-truth

Nodes in real-world networks organize into densely linked communities where edges appear with high concentration among the members of the community. Identifying such communities of nodes has proven to be a challenging task mainly due to a plethora of definitions of a community, intractability of algorithms, issues with evaluation and the lack of a reliable gold-standard ground-truth. In this paper we study a set of 230 large real-world social, collaboration and information networks where nodes explicitly state their group memberships. For example, in social networks nodes explicitly join various interest based social groups. We use such groups to define a reliable and robust notion of ground-truth communities. We then propose a methodology which allows us to compare and quantitatively evaluate how different structural definitions of network communities correspond to ground-truth communities. We choose 13 commonly used structural definitions of network communities and examine their sensitivity, robustness and performance in identifying the ground-truth. We show that the 13 structural definitions are heavily correlated and naturally group into four classes. We find that two of these definitions, Conductance and Triad-participation-ratio, consistently give the best performance in identifying ground-truth communities. We also investigate a task of detecting communities given a single seed node. We extend the local spectral clustering algorithm into a heuristic parameter-free community detection method that easily scales to networks with more than hundred million nodes. The proposed method achieves 30% relative improvement over current local clustering methods.Comment: Proceedings of 2012 IEEE International Conference on Data Mining (ICDM), 201