Scale-freeness and small-world phenomenon in information-flow graphs of geometrical neural networks

In this dissertation we set out to study a simplified model of activation flow in artificial neural networks with geometrical embedding. The model provides a mathematical description of abstract neural activation transfer in terms, which bear resemblances to multi-value Boltzmann-like evolution. The activation-preserving constraint mimics a critical regime of the dynamics and, along with accounting for geometrical location of the neurons, makes the system more feasible for modelling of real-world networks. We focus on scale invariance or scale-freeness and small-world phenomena in the said networks. Our results clearly confirm presence of both features at the functional level of the activity-flow graph. We show that the degree distribution preserves a power-law shape with the exponent value approximately equal to -2. In addition, we present our results concerning characteristic path length in the said graphs, which grows roughly logarithmically with the size of the network, while the clustering coefficient turns out to be relatively high. Taken together, the clustering and path length ratios are surprisingly high, and thus confirm large both local and global efficiency of the network. Finally, we compare the properties of activation-flow model to those reported in neurobiological analyses of brain networks recorded with functional magnetic resonance imagining (fMRI). There is a strong agreement between the shape and exponent value of degree distribution also the clustering and characteristic path lengths are comparable in both the model and medical data.Celem niniejszej rozprawy jest analiza uproszczonego modelu przepływu aktywności w sztucznych sieciach neuronowych zanurzonych w przestrzeni geometrycznej. Przedstawiony model dostarcza matematycznego opisu transferu aktywności w terminach zbliżonych do wielowartościowych maszyn Boltzmanna. Wymóg zachowania stałej sumarycznej aktywności odzwierciedla krytyczność dynamiki i wraz z uwzględnieniem wpływu lokalizacji geometrycznej neuronów sprawia, że system jest bardziej adekwatny do modelowania rzeczywistych sieci. Badania koncentrują się na bezskalowości oraz fenomenie małego świata w wyżej wymienionych sieciach. Uzyskane rezultaty potwierdzają obecność obu własności w omawianych grafach. Pokażemy, że rozkład stopni wejściowych wierzchołków zachowuje się jak funkcja potęgowa z wykładnikiem równym -2. Ponadto prezentujemy wyniki dotyczące charakterystycznej długości ścieżki, który rośnie logarytmicznie wraz z wielkością systemu, podczas gdy współczynnik klasteryzacji okazuje się dość duży. W konsekwencji stosunek klasteryzacji do długości ścieżek jest zaskakująco wysoki, co jest dystynktywną własnością sieci małego świata. Wreszcie, dokonujemy porównania cech omawianego modelu przepływu aktywności z neuro-biologicznymi rezultatami, przedstawionymi w badaniach grafów mózgowych z danych uzyskanych z funkcjonalnego obrazowania z wykorzystaniem rezonansu magnetycznego (fMRI). Wskazujemy silną odpowiedniość pomiędzy kształtem i wartością wykładnika rozkładu stopni, zaś klasteryzacja i charakterystyczna długość ścieżki są porównywalne w modelu i danych medycznych

Parallel Programming with Global Asynchronous Memory: Models, C++ APIs and Implementations

In the realm of High Performance Computing (HPC), message passing has been the programming paradigm of choice for over twenty years. The durable MPI (Message Passing Interface) standard, with send/receive communication, broadcast, gather/scatter, and reduction collectives is still used to construct parallel programs where each communication is orchestrated by the developer-based precise knowledge of data distribution and overheads; collective communications simplify the orchestration but might induce excessive synchronization. Early attempts to bring shared-memory programming model—with its programming advantages—to distributed computing, referred as the Distributed Shared Memory (DSM) model, faded away; one of the main issue was to combine performance and programmability with the memory consistency model. The recently proposed Partitioned Global Address Space (PGAS) model is a modern revamp of DSM that exposes data placement to enable optimizations based on locality, but it still addresses (simple) data- parallelism only and it relies on expensive sharing protocols. We advocate an alternative programming model for distributed computing based on a Global Asynchronous Memory (GAM), aiming to avoid coherency and consistency problems rather than solving them. We materialize GAM by designing and implementing a distributed smart pointers library, inspired by C++ smart pointers. In this model, public and pri- vate pointers (resembling C++ shared and unique pointers, respectively) are moved around instead of messages (i.e., data), thus alleviating the user from the burden of minimizing transfers. On top of smart pointers, we propose a high-level C++ template library for writing applications in terms of dataflow-like networks, namely GAM nets, consisting of stateful processors exchanging pointers in fully asynchronous fashion. We demonstrate the validity of the proposed approach, from the expressiveness perspective, by showing how GAM nets can be exploited to implement both standalone applications and higher-level parallel program- ming models, such as data and task parallelism. As for the performance perspective, preliminary experiments show both close-to-ideal scalability and negligible overhead with respect to state-of-the-art benchmark implementations. For instance, the GAM implementation of a high-quality video restoration filter sustains a 100 fps throughput over 70%-noisy high-quality video streams on a 4-node cluster of Graphics Processing Units (GPUs), with minimal programming effort

Evaluating Network Models: A Likelihood Analysis

Many models are put forward to mimic the evolution of real networked systems. A well-accepted way to judge the validity is to compare the modeling results with real networks subject to several structural features. Even for a specific real network, we cannot fairly evaluate the goodness of different models since there are too many structural features while there is no criterion to select and assign weights on them. Motivated by the studies on link prediction algorithms, we propose a unified method to evaluate the network models via the comparison of the likelihoods of the currently observed network driven by different models, with an assumption that the higher the likelihood is, the better the model is. We test our method on the real Internet at the Autonomous System (AS) level, and the results suggest that the Generalized Linear Preferential (GLP) model outperforms the Tel Aviv Network Generator (Tang), while both two models are better than the Barab\'asi-Albert (BA) and Erd\"os-R\'enyi (ER) models. Our method can be further applied in determining the optimal values of parameters that correspond to the maximal likelihood. Experiment indicates that the parameters obtained by our method can better capture the characters of newly-added nodes and links in the AS-level Internet than the original methods in the literature.Comment: 6 pages, 2 figures, 3 table

BioCode: A Data-Driven Procedure to Learn the Growth of Biological Networks

Probabilistic biological network growth models have been utilized for many tasks including but not limited to capturing mechanism and dynamics of biological growth activities, null model representation, capturing anomalies, etc. Well-known examples of these probabilistic models are Kronecker model, preferential attachment model, and duplication-based model. However, we should frequently keep developing new models to better fit and explain the observed network features while new networks are being observed. Additionally, it is difficult to develop a growth model each time we study a new network. In this paper, we propose BioCode, a framework to automatically discover novel biological growth models matching user-specified graph attributes in directed and undirected biological graphs. BioCode designs a basic set of instructions which are common enough to model a number of well-known biological graph growth models. We combine such instruction-wise representation with a genetic algorithm based optimization procedure to encode models for various biological networks. We mainly evaluate the performance of BioCode in discovering models for biological collaboration networks, gene regulatory networks, metabolic networks, and protein interaction networks which features such as assortativity, clustering coefficient, degree distribution closely match with the true ones in the corresponding real biological networks. As shown by the tests on the simulated graphs, the variance of the distributions of biological networks generated by BioCode is similar to the known models' variance for these biological network types

Identifying the Presence of Communities in Complex Networks Through Topological Decomposition and Component Densities

International audienceThe exponential growth of data in various fields such as Social Networks and Internet has stimulated lots of activity in the field of network analysis and data mining. Identifying Communities remains a fundamental technique to explore and organize these networks. Few metrics are widely used to discover the presence of communities in a network. We argue that these metrics do not truly reflect the presence of communities by presenting counter examples. This is because these metrics concentrate on local cohesiveness among nodes where the goal is to judge whether two nodes belong to the same community or vise versa. Thus loosing the overall perspective of the presence of communities in the entire network. In this paper, we propose a new metric to identify the presence of communities in real world networks. This metric is based on the topological decomposition of networks taking into account two important ingredients of real world networks, the degree distribution and the density of nodes. We show the effectiveness of the proposed metric by testing it on various real world data sets

A nonuniform popularity-similarity optimization (nPSO) model to efficiently generate realistic complex networks with communities

The hidden metric space behind complex network topologies is a fervid topic in current network science and the hyperbolic space is one of the most studied, because it seems associated to the structural organization of many real complex systems. The Popularity-Similarity-Optimization (PSO) model simulates how random geometric graphs grow in the hyperbolic space, reproducing strong clustering and scale-free degree distribution, however it misses to reproduce an important feature of real complex networks, which is the community organization. The Geometrical-Preferential-Attachment (GPA) model was recently developed to confer to the PSO also a community structure, which is obtained by forcing different angular regions of the hyperbolic disk to have variable level of attractiveness. However, the number and size of the communities cannot be explicitly controlled in the GPA, which is a clear limitation for real applications. Here, we introduce the nonuniform PSO (nPSO) model that, differently from GPA, forces heterogeneous angular node attractiveness by sampling the angular coordinates from a tailored nonuniform probability distribution, for instance a mixture of Gaussians. The nPSO differs from GPA in other three aspects: it allows to explicitly fix the number and size of communities; it allows to tune their mixing property through the network temperature; it is efficient to generate networks with high clustering. After several tests we propose the nPSO as a valid and efficient model to generate networks with communities in the hyperbolic space, which can be adopted as a realistic benchmark for different tasks such as community detection and link prediction