Incremental closeness centrality in distributed memory

Networks are commonly used to model traffic patterns, social interactions, or web pages. The vertices in a network do not possess the same characteristics: some vertices are naturally more connected and some vertices can be more important. Closeness centrality (CC) is a global metric that quantifies how important is a given vertex in the network. When the network is dynamic and keeps changing, the relative importance of the vertices also changes. The best known algorithm to compute the CC scores makes it impractical to recompute them from scratch after each modification. In this paper, we propose Streamer, a distributed memory framework for incrementally maintaining the closeness centrality scores of a network upon changes. It leverages pipelined, replicated parallelism, and SpMM-based BFSs, and it takes NUMA effects into account. It makes maintaining the Closeness Centrality values of real-life networks with millions of interactions significantly faster and obtains almost linear speedups on a 64 nodes 8 threads/node cluster

Fast Shortest Path Distance Estimation in Large Networks

We study the problem of preprocessing a large graph so that point-to-point shortest-path queries can be answered very fast. Computing shortest paths is a well studied problem, but exact algorithms do not scale to huge graphs encountered on the web, social networks, and other applications. In this paper we focus on approximate methods for distance estimation, in particular using landmark-based distance indexing. This approach involves selecting a subset of nodes as landmarks and computing (offline) the distances from each node in the graph to those landmarks. At runtime, when the distance between a pair of nodes is needed, we can estimate it quickly by combining the precomputed distances of the two nodes to the landmarks. We prove that selecting the optimal set of landmarks is an NP-hard problem, and thus heuristic solutions need to be employed. Given a budget of memory for the index, which translates directly into a budget of landmarks, different landmark selection strategies can yield dramatically different results in terms of accuracy. A number of simple methods that scale well to large graphs are therefore developed and experimentally compared. The simplest methods choose central nodes of the graph, while the more elaborate ones select central nodes that are also far away from one another. The efficiency of the suggested techniques is tested experimentally using five different real world graphs with millions of edges; for a given accuracy, they require as much as 250 times less space than the current approach in the literature which considers selecting landmarks at random. Finally, we study applications of our method in two problems arising naturally in large-scale networks, namely, social search and community detection.Yahoo! Research (internship

Modeling phase changes of road networks

Adopting an agent-based approach, this paper explores the topological evolution of road networks from a microscopic perspective. We assume a decentralized decision-making mechanism where roads are built by self-interested land parcel owners. By building roads, parcel owners hope to increase their parcelsÕ accessibility and economic value. The simulation model is performed on a grid-like land use layer with a downtown in the center, whose structure is similar to the early form of many Midwestern and Western (US) cities. The topological attributes for the networks are evaluated by multiple centrality measures such as degree centrality, closeness centrality, and betweenness centrality. Our findings disclose that the growth of road network experiences an evolutionary process where tree-like structure first emerges around the centered parcel before the network pushes outward to the periphery. In addition, road network topology undergoes obvious phase changes as the economic values of parcels vary. The results demonstrate that even without a centralized authority, road networks have the property of self-organization and evolution; furthermore, the rise-and-fall of places in terms of their economic/social values may considerably impact road network topology.road network, land parcel, network evolution, network growth, phase change

Generating realistic scaled complex networks

Research on generative models is a central project in the emerging field of network science, and it studies how statistical patterns found in real networks could be generated by formal rules. Output from these generative models is then the basis for designing and evaluating computational methods on networks, and for verification and simulation studies. During the last two decades, a variety of models has been proposed with an ultimate goal of achieving comprehensive realism for the generated networks. In this study, we (a) introduce a new generator, termed ReCoN; (b) explore how ReCoN and some existing models can be fitted to an original network to produce a structurally similar replica, (c) use ReCoN to produce networks much larger than the original exemplar, and finally (d) discuss open problems and promising research directions. In a comparative experimental study, we find that ReCoN is often superior to many other state-of-the-art network generation methods. We argue that ReCoN is a scalable and effective tool for modeling a given network while preserving important properties at both micro- and macroscopic scales, and for scaling the exemplar data by orders of magnitude in size.Comment: 26 pages, 13 figures, extended version, a preliminary version of the paper was presented at the 5th International Workshop on Complex Networks and their Application

A Positive Theory of Network Connectivity

This paper develops a positive theory of network connectivity, seeking to explain the micro-foundations of alternative network topologies as the result of self-interested actors. By building roads, landowners hope to increase their parcelsÕ accessibility and economic value. A simulation model is performed on a grid-like land use layer with a downtown in the center, whose structure resembles the early form of many Midwest- ern and Western (US) cities. The topological attributes for the networks are evaluated. This research posits that road networks experience an evolutionary process where a tree-like structure first emerges around the centered parcel before the network pushes outward to the periphery. In addition, road network topology undergoes clear phase changes as the economic values of parcels vary. The results demonstrate that even without a centralized authority, road networks have the property of self-organization and evolution, and, that in the absence of intervention, the tree-like or web-like nature of networks is a result of the underlying economics.road network, land parcel, network evolution, network growth, phase change, centrality measures, degree centrality, closeness centrality, betweenness centrality, network structure, treeness, circuitness, topology

NOESIS: A Framework for Complex Network Data Analysis

Network data mining has attracted a lot of attention since a large number of real-world problems have to deal with complex network data. In this paper, we present NOESIS, an open-source framework for network-based data mining. NOESIS features a large number of techniques and methods for the analysis of structural network properties, network visualization, community detection, link scoring, and link prediction. ­e proposed framework has been designed following solid design principles and exploits parallel computing using structured parallel programming. NOESIS also provides a stand-alone graphical user interface allowing the use of advanced software analysis techniques to users without prior programming experience. ­is framework is available under a BSD open-source software license.The NOESIS project was partially supported by the Spanish Ministry of Economy and the European Regional Development Fund (FEDER), under grant TIN2012–36951, and the Spanish Ministry of Education under the program “Ayudas para contratos predoctorales para la formación de doctores 2013” (predoctoral grant BES–2013–064699)