Clustering and Community Detection in Directed Networks: A Survey

Networks (or graphs) appear as dominant structures in diverse domains, including sociology, biology, neuroscience and computer science. In most of the aforementioned cases graphs are directed - in the sense that there is directionality on the edges, making the semantics of the edges non symmetric. An interesting feature that real networks present is the clustering or community structure property, under which the graph topology is organized into modules commonly called communities or clusters. The essence here is that nodes of the same community are highly similar while on the contrary, nodes across communities present low similarity. Revealing the underlying community structure of directed complex networks has become a crucial and interdisciplinary topic with a plethora of applications. Therefore, naturally there is a recent wealth of research production in the area of mining directed graphs - with clustering being the primary method and tool for community detection and evaluation. The goal of this paper is to offer an in-depth review of the methods presented so far for clustering directed networks along with the relevant necessary methodological background and also related applications. The survey commences by offering a concise review of the fundamental concepts and methodological base on which graph clustering algorithms capitalize on. Then we present the relevant work along two orthogonal classifications. The first one is mostly concerned with the methodological principles of the clustering algorithms, while the second one approaches the methods from the viewpoint regarding the properties of a good cluster in a directed network. Further, we present methods and metrics for evaluating graph clustering results, demonstrate interesting application domains and provide promising future research directions.Comment: 86 pages, 17 figures. Physics Reports Journal (To Appear

Overlapping Community Discovery Methods: A Survey

The detection of overlapping communities is a challenging problem which is gaining increasing interest in recent years because of the natural attitude of individuals, observed in real-world networks, to participate in multiple groups at the same time. This review gives a description of the main proposals in the field. Besides the methods designed for static networks, some new approaches that deal with the detection of overlapping communities in networks that change over time, are described. Methods are classified with respect to the underlying principles guiding them to obtain a network division in groups sharing part of their nodes. For each of them we also report, when available, computational complexity and web site address from which it is possible to download the software implementing the method.Comment: 20 pages, Book Chapter, appears as Social networks: Analysis and Case Studies, A. Gunduz-Oguducu and A. S. Etaner-Uyar eds, Lecture Notes in Social Networks, pp. 105-125, Springer,201

Link communities reveal multiscale complexity in networks

Networks have become a key approach to understanding systems of interacting objects, unifying the study of diverse phenomena including biological organisms and human society. One crucial step when studying the structure and dynamics of networks is to identify communities: groups of related nodes that correspond to functional subunits such as protein complexes or social spheres. Communities in networks often overlap such that nodes simultaneously belong to several groups. Meanwhile, many networks are known to possess hierarchical organization, where communities are recursively grouped into a hierarchical structure. However, the fact that many real networks have communities with pervasive overlap, where each and every node belongs to more than one group, has the consequence that a global hierarchy of nodes cannot capture the relationships between overlapping groups. Here we reinvent communities as groups of links rather than nodes and show that this unorthodox approach successfully reconciles the antagonistic organizing principles of overlapping communities and hierarchy. In contrast to the existing literature, which has entirely focused on grouping nodes, link communities naturally incorporate overlap while revealing hierarchical organization. We find relevant link communities in many networks, including major biological networks such as protein-protein interaction and metabolic networks, and show that a large social network contains hierarchically organized community structures spanning inner-city to regional scales while maintaining pervasive overlap. Our results imply that link communities are fundamental building blocks that reveal overlap and hierarchical organization in networks to be two aspects of the same phenomenon.Comment: Main text and supplementary informatio

Overlapping Community Detection Extended from Disjoint Community Structure

Community detection is a hot issue in the study of complex networks. Many community detection algorithms have been put forward in different fields. But most of the existing community detection algorithms are used to find disjoint community structure. In order to make full use of the disjoint community detection algorithms to adapt to the new demand of overlapping community detection, this paper proposes an overlapping community detection algorithm extended from disjoint community structure by selecting overlapping nodes (ONS-OCD). In the algorithm, disjoint community structure with high qualities is firstly taken as input, then, potential members of each community are identified. Overlapping nodes are determined according to the node contribution to the community. Finally, adding overlapping nodes to all communities they belong to and get the final overlapping community structure. ONS-OCD algorithm reduces the computation of judging overlapping nodes by narrowing the scope of the potential member nodes of each community. Experimental results both on synthetic and real networks show that the community detection quality of ONS-OCD algorithm is better than several other representative overlapping community detection algorithms

Fundamental structures of dynamic social networks

Social systems are in a constant state of flux with dynamics spanning from minute-by-minute changes to patterns present on the timescale of years. Accurate models of social dynamics are important for understanding spreading of influence or diseases, formation of friendships, and the productivity of teams. While there has been much progress on understanding complex networks over the past decade, little is known about the regularities governing the micro-dynamics of social networks. Here we explore the dynamic social network of a densely-connected population of approximately 1000 individuals and their interactions in the network of real-world person-to-person proximity measured via Bluetooth, as well as their telecommunication networks, online social media contacts, geo-location, and demographic data. These high-resolution data allow us to observe social groups directly, rendering community detection unnecessary. Starting from 5-minute time slices we uncover dynamic social structures expressed on multiple timescales. On the hourly timescale, we find that gatherings are fluid, with members coming and going, but organized via a stable core of individuals. Each core represents a social context. Cores exhibit a pattern of recurring meetings across weeks and months, each with varying degrees of regularity. Taken together, these findings provide a powerful simplification of the social network, where cores represent fundamental structures expressed with strong temporal and spatial regularity. Using this framework, we explore the complex interplay between social and geospatial behavior, documenting how the formation of cores are preceded by coordination behavior in the communication networks, and demonstrating that social behavior can be predicted with high precision.Comment: Main Manuscript: 16 pages, 4 figures. Supplementary Information: 39 pages, 34 figure

Skeleton coupling: a novel interlayer mapping of community evolution in temporal networks

Dynamic community detection (DCD) in temporal networks is a complicated task that involves the selection of an algorithm and its associated parameters. How to choose the most appropriate algorithm generally depends on the type of network being analyzed and the specific properties of the data that define the network. In functional temporal networks derived from neuronal spike train data, communities are expected to be transient, and it is common for the network to contain multiple singleton communities. Here, we compare the performance of different DCD algorithms on functional temporal networks built from synthetic neuronal time series data with known community structure. We find that, for these networks, DCD algorithms that utilize interlayer links to perform community carryover between layers outperform other methods. However, we also observe that algorithm performance is highly dependent on the topology of interlayer links, especially in the presence of singleton and transient communities. We therefore define a novel method for defining interlayer links in temporal networks called skeleton coupling that is specifically designed to enhance the linkage of communities in the network throughout time based on the topological properties of the community history. We show that integrating skeleton coupling with current DCD methods improves algorithm performance in synthetic data with planted singleton and transient communities. The use of skeleton coupling to perform DCD will therefore allow for more accurate and interpretable results of community evolution in real-world neuronal data or in other systems with transient structure and singleton communities.Comment: 19 pages, 8 figure

Local Intrinsic Dimensionality Measures for Graphs, with Applications to Graph Embeddings

The notion of local intrinsic dimensionality (LID) is an important advancement in data dimensionality analysis, with applications in data mining, machine learning and similarity search problems. Existing distance-based LID estimators were designed for tabular datasets encompassing data points represented as vectors in a Euclidean space. After discussing their limitations for graph-structured data considering graph embeddings and graph distances, we propose NC-LID, a novel LID-related measure for quantifying the discriminatory power of the shortest-path distance with respect to natural communities of nodes as their intrinsic localities. It is shown how this measure can be used to design LID-aware graph embedding algorithms by formulating two LID-elastic variants of node2vec with personalized hyperparameters that are adjusted according to NC-LID values. Our empirical analysis of NC-LID on a large number of real-world graphs shows that this measure is able to point to nodes with high link reconstruction errors in node2vec embeddings better than node centrality metrics. The experimental evaluation also shows that the proposed LID-elastic node2vec extensions improve node2vec by better preserving graph structure in generated embeddings