Hierarchical community structure in networks

Modular and hierarchical structures are pervasive in real-world complex systems. A great deal of effort has gone into trying to detect and study these structures. Important theoretical advances in the detection of modular, or "community", structures have included identifying fundamental limits of detectability by formally defining community structure using probabilistic generative models. Detecting hierarchical community structure introduces additional challenges alongside those inherited from community detection. Here we present a theoretical study on hierarchical community structure in networks, which has thus far not received the same rigorous attention. We address the following questions: 1)~How should we define a valid hierarchy of communities? 2)~How should we determine if a hierarchical structure exists in a network? and 3)~how can we detect hierarchical structure efficiently? We approach these questions by introducing a definition of hierarchy based on the concept of stochastic externally equitable partitions and their relation to probabilistic models, such as the popular stochastic block model. We enumerate the challenges involved in detecting hierarchies and, by studying the spectral properties of hierarchical structure, present an efficient and principled method for detecting them.Comment: 22 pages, 12 figure

Finding community structure in very large networks

The discovery and analysis of community structure in networks is a topic of considerable recent interest within the physics community, but most methods proposed so far are unsuitable for very large networks because of their computational cost. Here we present a hierarchical agglomeration algorithm for detecting community structure which is faster than many competing algorithms: its running time on a network with n vertices and m edges is O(m d log n) where d is the depth of the dendrogram describing the community structure. Many real-world networks are sparse and hierarchical, with m ~ n and d ~ log n, in which case our algorithm runs in essentially linear time, O(n log^2 n). As an example of the application of this algorithm we use it to analyze a network of items for sale on the web-site of a large online retailer, items in the network being linked if they are frequently purchased by the same buyer. The network has more than 400,000 vertices and 2 million edges. We show that our algorithm can extract meaningful communities from this network, revealing large-scale patterns present in the purchasing habits of customers

A Method to Find Community Structures Based on Information Centrality

Community structures are an important feature of many social, biological and technological networks. Here we study a variation on the method for detecting such communities proposed by Girvan and Newman and based on the idea of using centrality measures to define the community boundaries (M. Girvan and M. E. J. Newman, Community structure in social and biological networks Proc. Natl. Acad. Sci. USA 99, 7821-7826 (2002)). We develop an algorithm of hierarchical clustering that consists in finding and removing iteratively the edge with the highest information centrality. We test the algorithm on computer generated and real-world networks whose community structure is already known or has been studied by means of other methods. We show that our algorithm, although it runs to completion in a time O(n^4), is very effective especially when the communities are very mixed and hardly detectable by the other methods.Comment: 13 pages, 13 figures. Final version accepted for publication in Physical Review

Social Network Analysis Using a Multi-agent System: A School System Case

The quality of k-12 education has been a major concern in the nation for years. School systems, just like many other social networks, appear to have a hierarchical structure. Understanding this structure could be the key to better evaluate student performance and improve school quality. Many researches have been focusing on detecting hierarchical structure by using hierarchical clustering algorithms. Compared to existing methods, we design an interaction-based similarity measure to accomplish hierarchical clustering in order to detect hierarchical structures in social networks (e.g. school district networks). This method uses a Multi-agent System for it is based on agent interactions. With the network structure detected, we also build a model, which is inspired by the MAXQ algorithm, to decompose funding policy task into subtask and then evaluate these subtasks by using funding distribution policies from past years and looking for possible relationships between student performances and funding policies. For experiment, we use real school data from Bexar county’s 15 school districts. The first result shows that our interaction based method is able to generate meaningful clustering and dendrogram for social networks. And our policy evaluation model is able to evaluate funding policies from past three years in Bexar County and conclude that increasing funding does not necessarily have a positive impact on student performance and it is generally not the case that the more spend the better

Community Detection in Quantum Complex Networks

Determining community structure is a central topic in the study of complex networks, be it technological, social, biological or chemical, in static or interacting systems. In this paper, we extend the concept of community detection from classical to quantum systems---a crucial missing component of a theory of complex networks based on quantum mechanics. We demonstrate that certain quantum mechanical effects cannot be captured using current classical complex network tools and provide new methods that overcome these problems. Our approaches are based on defining closeness measures between nodes, and then maximizing modularity with hierarchical clustering. Our closeness functions are based on quantum transport probability and state fidelity, two important quantities in quantum information theory. To illustrate the effectiveness of our approach in detecting community structure in quantum systems, we provide several examples, including a naturally occurring light-harvesting complex, LHCII. The prediction of our simplest algorithm, semiclassical in nature, mostly agrees with a proposed partitioning for the LHCII found in quantum chemistry literature, whereas our fully quantum treatment of the problem uncovers a new, consistent, and appropriately quantum community structure.Comment: 16 pages, 4 figures, 1 tabl

Detection and localization of change points in temporal networks with the aid of stochastic block models

A framework based on generalized hierarchical random graphs (GHRGs) for the detection of change points in the structure of temporal networks has recently been developed by Peel and Clauset [1]. We build on this methodology and extend it to also include the versatile stochastic block models (SBMs) as a parametric family for reconstructing the empirical networks. We use five different techniques for change point detection on prototypical temporal networks, including empirical and synthetic ones. We find that none of the considered methods can consistently outperform the others when it comes to detecting and locating the expected change points in empirical temporal networks. With respect to the precision and the recall of the results of the change points, we find that the method based on a degree-corrected SBM has better recall properties than other dedicated methods, especially for sparse networks and smaller sliding time window widths.Comment: This is an author-created, un-copyedited version of an article accepted for publication/published in Journal of Statistical Mechanics: Theory and Experiment. IOP Publishing Ltd is not responsible for any errors or omissions in this version of the manuscript or any version derived from it. The Version of Record is available online at http://dx.doi.org/10.1088/1742-5468/2016/11/11330