The Thermodynamics of Network Coding, and an Algorithmic Refinement of the Principle of Maximum Entropy

The principle of maximum entropy (Maxent) is often used to obtain prior probability distributions as a method to obtain a Gibbs measure under some restriction giving the probability that a system will be in a certain state compared to the rest of the elements in the distribution. Because classical entropy-based Maxent collapses cases confounding all distinct degrees of randomness and pseudo-randomness, here we take into consideration the generative mechanism of the systems considered in the ensemble to separate objects that may comply with the principle under some restriction and whose entropy is maximal but may be generated recursively from those that are actually algorithmically random offering a refinement to classical Maxent. We take advantage of a causal algorithmic calculus to derive a thermodynamic-like result based on how difficult it is to reprogram a computer code. Using the distinction between computable and algorithmic randomness we quantify the cost in information loss associated with reprogramming. To illustrate this we apply the algorithmic refinement to Maxent on graphs and introduce a Maximal Algorithmic Randomness Preferential Attachment (MARPA) Algorithm, a generalisation over previous approaches. We discuss practical implications of evaluation of network randomness. Our analysis provides insight in that the reprogrammability asymmetry appears to originate from a non-monotonic relationship to algorithmic probability. Our analysis motivates further analysis of the origin and consequences of the aforementioned asymmetries, reprogrammability, and computation.Comment: 30 page

Effective and Efficient Similarity Index for Link Prediction of Complex Networks

Predictions of missing links of incomplete networks like protein-protein interaction networks or very likely but not yet existent links in evolutionary networks like friendship networks in web society can be considered as a guideline for further experiments or valuable information for web users. In this paper, we introduce a local path index to estimate the likelihood of the existence of a link between two nodes. We propose a network model with controllable density and noise strength in generating links, as well as collect data of six real networks. Extensive numerical simulations on both modeled networks and real networks demonstrated the high effectiveness and efficiency of the local path index compared with two well-known and widely used indices, the common neighbors and the Katz index. Indeed, the local path index provides competitively accurate predictions as the Katz index while requires much less CPU time and memory space, which is therefore a strong candidate for potential practical applications in data mining of huge-size networks.Comment: 8 pages, 5 figures, 3 table

An architecturally constrained model of random number generation and its application to modeling the effect of generation rate

Random number generation (RNG) is a complex cognitive task for human subjects, requiring deliberative control to avoid production of habitual, stereotyped sequences. Under various manipulations (e.g., speeded responding, transcranial magnetic stimulation, or neurological damage) the performance of human subjects deteriorates, as reflected in a number of qualitatively distinct, dissociable biases. For example, the intrusion of stereotyped behavior (e.g., counting) increases at faster rates of generation. Theoretical accounts of the task postulate that it requires the integrated operation of multiple, computationally heterogeneous cognitive control (“executive”) processes. We present a computational model of RNG, within the framework of a novel, neuropsychologically-inspired cognitive architecture, ESPro. Manipulating the rate of sequence generation in the model reproduced a number of key effects observed in empirical studies, including increasing sequence stereotypy at faster rates. Within the model, this was due to time limitations on the interaction of supervisory control processes, namely, task setting, proposal of responses, monitoring, and response inhibition. The model thus supports the fractionation of executive function into multiple, computationally heterogeneous processes

Analysis and evaluation of the entropy indices of a static network structure

Although degree distribution entropy (DDE), SD structure entropy (SDSE), Wu structure entropy (WSE) and FB structure entropy (FBSE) are four static network structure entropy indices widely used to quantify the heterogeneity of a complex network, previous studies have paid little attention to their differing abilities to describe network structure. We calculate these four structure entropies for four benchmark networks and compare the results by measuring the ability of each index to characterize network heterogeneity. We find that SDSE and FBSE more accurately characterize network heterogeneity than WSE and DDE. We also find that existing benchmark networks fail to distinguish SDSE and FBSE because they cannot discriminate local and global network heterogeneity. We solve this problem by proposing an evolving caveman network that reveals the differences between structure entropy indices by comparing the sensitivities during the network evolutionary process. Mathematical analysis and computational simulation both indicate that FBSE describes the global topology variation in the evolutionary process of a caveman network, and that the other three structure entropy indices reflect only local network heterogeneity. Our study offers an expansive view of the structural complexity of networks and expands our understanding of complex network behavior.The authors would like to thank the financial support of the National Natural Science Foundation of China (71501153), Natural Science Foundation of Shaanxi Province of China (2016JQ6072), and the Foundation of China Scholarship Council (201506965039, 201606965057). (71501153 - National Natural Science Foundation of China; 2016JQ6072 - Natural Science Foundation of Shaanxi Province of China; 201506965039 - Foundation of China Scholarship Council; 201606965057 - Foundation of China Scholarship Council)Published versio

Algorithmic information and incompressibility of families of multidimensional networks

This article presents a theoretical investigation of string-based generalized representations of families of finite networks in a multidimensional space. First, we study the recursive labeling of networks with (finite) arbitrary node dimensions (or aspects), such as time instants or layers. In particular, we study these networks that are formalized in the form of multiaspect graphs. We show that, unlike classical graphs, the algorithmic information of a multidimensional network is not in general dominated by the algorithmic information of the binary sequence that determines the presence or absence of edges. This universal algorithmic approach sets limitations and conditions for irreducible information content analysis in comparing networks with a large number of dimensions, such as multilayer networks. Nevertheless, we show that there are particular cases of infinite nesting families of finite multidimensional networks with a unified recursive labeling such that each member of these families is incompressible. From these results, we study network topological properties and equivalences in irreducible information content of multidimensional networks in comparison to their isomorphic classical graph.Comment: Extended preprint version of the pape

Perturbation Detection Through Modeling of Gene Expression on a Latent Biological Pathway Network: A Bayesian hierarchical approach

Cellular response to a perturbation is the result of a dynamic system of biological variables linked in a complex network. A major challenge in drug and disease studies is identifying the key factors of a biological network that are essential in determining the cell's fate. Here our goal is the identification of perturbed pathways from high-throughput gene expression data. We develop a three-level hierarchical model, where (i) the first level captures the relationship between gene expression and biological pathways using confirmatory factor analysis, (ii) the second level models the behavior within an underlying network of pathways induced by an unknown perturbation using a conditional autoregressive model, and (iii) the third level is a spike-and-slab prior on the perturbations. We then identify perturbations through posterior-based variable selection. We illustrate our approach using gene transcription drug perturbation profiles from the DREAM7 drug sensitivity predication challenge data set. Our proposed method identified regulatory pathways that are known to play a causative role and that were not readily resolved using gene set enrichment analysis or exploratory factor models. Simulation results are presented assessing the performance of this model relative to a network-free variant and its robustness to inaccuracies in biological databases