Using geodesic space density gradients for network community detection

Many real world complex systems naturally map to network data structures instead of geometric spaces because the only available information is the presence or absence of a link between two entities in the system. To enable data mining techniques to solve problems in the network domain, the nodes need to be mapped to a geometric space. We propose this mapping by representing each network node with its geodesic distances from all other nodes. The space spanned by the geodesic distance vectors is the geodesic space of that network. Position of different nodes in the geodesic space encode the network structure. In this space, considering a continuous density field induced by each node, density at a specific point is the summation of density fields induced by all nodes. We drift each node in the direction of positive density gradient using an iterative algorithm till each node reaches a local maximum. Due to the network structure captured by this space, the nodes that drift to the same region of space belong to the same communities in the original network. We use the direction of movement and final position of each node as important clues for community membership assignment. The proposed algorithm is compared with more than ten state of the art community detection techniques on two benchmark networks with known communities using Normalized Mutual Information criterion. The proposed algorithm outperformed these methods by a significant margin. Moreover, the proposed algorithm has also shown excellent performance on many real-world networks

Spatial Optimization Methods And System For Redistricting Problems

Redistricting is the process of dividing space into districts or zones while optimizing a set of spatial criteria under certain constraints. Example applications of redistricting include political redistricting, school redistricting, business service planning, and city management, among many others. Redistricting is a mission-critical component in operating governments and businesses alike. In research fields, redistricting (or region building) are also widely used, such as climate zoning, traffic zone analysis, and complex network analysis. However, as a combinatorial optimization problem, redistricting optimization remains one of the most difficult research challenges. There are currently few automated redistricting methods that have the optimization capability to produce solutions that meet practical needs. The absence of effective and efficient computational approaches for redistricting makes it extremely time-consuming and difficult for an individual person to consider multiple criteria/constraints and manually create solutions using a trial-and-error approach. To address both the scientific and practical challenges in solving real-world redistricting problems, this research advances the methodology and application of redistricting by developing a new computational spatial optimization method and a system platform that can address a wide range of redistricting problems, in an automated and computation-assisted manner. The research has three main contributions. First, an efficient and effective spatial optimization method is developed for redistricting. The new method is based on a spatially constrained and Tabu-based heuristics, which can optimize multiple criteria under multiple constraints to construct high-quality optimization solutions. The new approach is evaluated with real-world redistricting applications and compared with existing methods. Evaluation results show that the new optimization algorithm is more efficient (being able to allow real-time user interaction), more flexible (considering multiple user-expressed criteria and constraints), and more powerful (in terms of optimization quality) than existing methods. As such, it has the potential to enable general users to perform complex redistricting tasks. Second, a redistricting system, iRedistrict, is developed based on the newly developed spatial optimization method to provide user-friendly visual interface for defining redistricting problems, incorporating domain knowledge, configuring optimization criteria and methodology parameters, and ultimately meeting the needs of real-world applications for tackling complex redistricting tasks. It is particularly useful for users of different skill levels, including researchers, practitioners, and the general public, and thus enables public participation in challenging redistricting tasks that are of immense public interest. Performance evaluations with real-world case studies are carried out. Further computational strategies are developed and implemented to handle large datasets. Third, the newly developed spatial optimization method is extended to solve a different spatial optimization problem, i.e., spatial community structure detection in complex networks, which is to partition networks to discover spatial communities by optimizing an objective function. Moreover, a series of new evaluations are carried out with synthetic datasets. This set of evaluations is different from the previous evaluations with case studies in that, the optimal solution is known with synthetic data and therefore it is possible to evaluate (1) whether the optimization method can discover the true pattern (global optima), and (2) how different data characteristics may affect the performance of the method. Evaluation results reveal that existing non-spatial methods are not robust in detecting spatial community structure, which may produce dramatically different outcomes for the same data with different characteristics, such as different spatial aggregations, sampling rates, or noise levels. The new optimization method with spatial constraints is significantly more stable and consistent. In addition to evaluations with synthetic datasets, a case study is also carried out to detect urban community structure with human movements, to demonstrate the application and effectiveness of the approach