Finding local community structure in networks

Although the inference of global community structure in networks has recently become a topic of great interest in the physics community, all such algorithms require that the graph be completely known. Here, we define both a measure of local community structure and an algorithm that infers the hierarchy of communities that enclose a given vertex by exploring the graph one vertex at a time. This algorithm runs in time O(d*k^2) for general graphs when $d$ is the mean degree and k is the number of vertices to be explored. For graphs where exploring a new vertex is time-consuming, the running time is linear, O(k). We show that on computer-generated graphs this technique compares favorably to algorithms that require global knowledge. We also use this algorithm to extract meaningful local clustering information in the large recommender network of an online retailer and show the existence of mesoscopic structure.Comment: 7 pages, 6 figure

Finding community structure in very large networks

The discovery and analysis of community structure in networks is a topic of considerable recent interest within the physics community, but most methods proposed so far are unsuitable for very large networks because of their computational cost. Here we present a hierarchical agglomeration algorithm for detecting community structure which is faster than many competing algorithms: its running time on a network with n vertices and m edges is O(m d log n) where d is the depth of the dendrogram describing the community structure. Many real-world networks are sparse and hierarchical, with m ~ n and d ~ log n, in which case our algorithm runs in essentially linear time, O(n log^2 n). As an example of the application of this algorithm we use it to analyze a network of items for sale on the web-site of a large online retailer, items in the network being linked if they are frequently purchased by the same buyer. The network has more than 400,000 vertices and 2 million edges. We show that our algorithm can extract meaningful communities from this network, revealing large-scale patterns present in the purchasing habits of customers

Mallard Use of Elevated Nesting Structures: Fifteen Years of Management in Eastern South Dakota

Studies of mallard use and nesting success on elevated structures have generally been of a short-term nature. In this report we evaluated 15 consecutive years of mallard nesting on elevated structures (baskets, cylinders, and culverts) located over standing water in wetland basins in eastern South Dakota. Our objectives were to: 1) determine long-term trends in mallard occupancy and nest success in nesting baskets, cylinders, and culverts, 2) evaluate the effects of various nest structure adaptations for enhancing mallard use and maintaining high nest success rates, and 3) provide managers with various enhancement techniques for mallard nesting structures that have proven productive in this long-term management effort

Upper and lower treeline biogeographic patterns in semi-arid pinyon-juniper woodlands

none7siAim: Upper and lower treelines are particularly exposed to a changing climate. It has been hypothesized that upper treelines are constrained by growing season temperature, whereas lower tree lines are water limited. We expect different causal mechanisms of upper versus lower tree line formation to generate distinct patterns of spatial heterogeneity. Here, we compare dynamics, spatial patterns and shape complexity of upper and lower tree lines of semi‐arid pinyon‐juniper woodlands. Location: Toiyabe Range of the Nevada Great Basin (western US). Taxon: Pinus monophylla Torr. & Frém. and Juniperus osteosperma (Torr.). Methods: Within 20 sample plots (10 along the upper and 10 along the lower tree line), we mapped tree canopies through photointerpretation of high‐resolution imagery. We performed point pattern analyses to compare the spatial arrangement of trees and used LANDSAT 30‐year time series and NDVI to understand the vegetation dynamics of these ecotones. We adopted the surface roughness method to measure tree line shape complexity. Results: Lower tree lines were denser and showed a stronger trend of increasing NDVI change over the 1984–2015 period. Trees at the lower tree line were more strongly aggregated than at the upper tree line at spatial scales ranging from 15 to 65 meters. Shape complexity was higher at upper tree lines, expressed by a higher mean surface roughness; however, the spatial structures of upper and lower tree lines were similar. Main conclusions: Upper tree line expansion of pinyon‐juniper woodlands in the study area has been limited and highly variable, but lower tree line downslope expansion into adjacent shrub steppe vegetation was evident. The expected difference between energy‐ and water‐limited tree lines did not manifest in the observed spatial structures. Differences in tree line shape complexity were not significant, although lower tree lines exhibited less complex shapes, likely because they have been more strongly influenced by anthropogenic factors.The datasets generated and analysed during the current study are available in the Figshare repository, https://doi.org/10.6084/m9.figshare.11836284mixedGarbarino, Matteo; Malandra, Francesco; Dilts, Thomas; Flake, Sam; Montalto, Luigi; Spinsante, Susanna; Weisberg, Peter J.Garbarino, Matteo; Malandra, Francesco; Dilts, Thomas; Flake, Sam; Montalto, Luigi; Spinsante, Susanna; Weisberg, Peter J

Identifying network communities with a high resolution

Community structure is an important property of complex networks. An automatic discovery of such structure is a fundamental task in many disciplines, including sociology, biology, engineering, and computer science. Recently, several community discovery algorithms have been proposed based on the optimization of a quantity called modularity (Q). However, the problem of modularity optimization is NP-hard, and the existing approaches often suffer from prohibitively long running time or poor quality. Furthermore, it has been recently pointed out that algorithms based on optimizing Q will have a resolution limit, i.e., communities below a certain scale may not be detected. In this research, we first propose an efficient heuristic algorithm, Qcut, which combines spectral graph partitioning and local search to optimize Q. Using both synthetic and real networks, we show that Qcut can find higher modularities and is more scalable than the existing algorithms. Furthermore, using Qcut as an essential component, we propose a recursive algorithm, HQcut, to solve the resolution limit problem. We show that HQcut can successfully detect communities at a much finer scale and with a higher accuracy than the existing algorithms. Finally, we apply Qcut and HQcut to study a protein-protein interaction network, and show that the combination of the two algorithms can reveal interesting biological results that may be otherwise undetectable.Comment: 14 pages, 5 figures. 1 supplemental file at http://cic.cs.wustl.edu/qcut/supplemental.pd

Community Structure of the Physical Review Citation Network

We investigate the community structure of physics subfields in the citation network of all Physical Review publications between 1893 and August 2007. We focus on well-cited publications (those receiving more than 100 citations), and apply modularity maximization to uncover major communities that correspond to clearly-identifiable subfields of physics. While most of the links between communities connect those with obvious intellectual overlap, there sometimes exist unexpected connections between disparate fields due to the development of a widely-applicable theoretical technique or by cross fertilization between theory and experiment. We also examine communities decade by decade and also uncover a small number of significant links between communities that are widely separated in time.Comment: 14 pages, 7 figures, 8 tables. Version 2: various small additions in response to referee comment

Parallel Gaussian Process Optimization with Upper Confidence Bound and Pure Exploration

In this paper, we consider the challenge of maximizing an unknown function f for which evaluations are noisy and are acquired with high cost. An iterative procedure uses the previous measures to actively select the next estimation of f which is predicted to be the most useful. We focus on the case where the function can be evaluated in parallel with batches of fixed size and analyze the benefit compared to the purely sequential procedure in terms of cumulative regret. We introduce the Gaussian Process Upper Confidence Bound and Pure Exploration algorithm (GP-UCB-PE) which combines the UCB strategy and Pure Exploration in the same batch of evaluations along the parallel iterations. We prove theoretical upper bounds on the regret with batches of size K for this procedure which show the improvement of the order of sqrt{K} for fixed iteration cost over purely sequential versions. Moreover, the multiplicative constants involved have the property of being dimension-free. We also confirm empirically the efficiency of GP-UCB-PE on real and synthetic problems compared to state-of-the-art competitors

Benchmark graphs for testing community detection algorithms

Community structure is one of the most important features of real networks and reveals the internal organization of the nodes. Many algorithms have been proposed but the crucial issue of testing, i.e. the question of how good an algorithm is, with respect to others, is still open. Standard tests include the analysis of simple artificial graphs with a built-in community structure, that the algorithm has to recover. However, the special graphs adopted in actual tests have a structure that does not reflect the real properties of nodes and communities found in real networks. Here we introduce a new class of benchmark graphs, that account for the heterogeneity in the distributions of node degrees and of community sizes. We use this new benchmark to test two popular methods of community detection, modularity optimization and Potts model clustering. The results show that the new benchmark poses a much more severe test to algorithms than standard benchmarks, revealing limits that may not be apparent at a first analysis.Comment: 6 pages, 8 figures. Extended version published on Physical Review E. The code to build the new benchmark graphs can be downloaded from http://santo.fortunato.googlepages.com/inthepress

Finding community structure in networks using the eigenvectors of matrices

We consider the problem of detecting communities or modules in networks, groups of vertices with a higher-than-average density of edges connecting them. Previous work indicates that a robust approach to this problem is the maximization of the benefit function known as "modularity" over possible divisions of a network. Here we show that this maximization process can be written in terms of the eigenspectrum of a matrix we call the modularity matrix, which plays a role in community detection similar to that played by the graph Laplacian in graph partitioning calculations. This result leads us to a number of possible algorithms for detecting community structure, as well as several other results, including a spectral measure of bipartite structure in networks and a new centrality measure that identifies those vertices that occupy central positions within the communities to which they belong. The algorithms and measures proposed are illustrated with applications to a variety of real-world complex networks.Comment: 22 pages, 8 figures, minor corrections in this versio