Estimating mutual information and multi--information in large networks

We address the practical problems of estimating the information relations that characterize large networks. Building on methods developed for analysis of the neural code, we show that reliable estimates of mutual information can be obtained with manageable computational effort. The same methods allow estimation of higher order, multi--information terms. These ideas are illustrated by analyses of gene expression, financial markets, and consumer preferences. In each case, information theoretic measures correlate with independent, intuitive measures of the underlying structures in the system

From modes to movement in the behavior of C. elegans

Organisms move through the world by changing their shape, and here we explore the mapping from shape space to movements in the nematode C. elegans as it crawls on a planar agar surface. We characterize the statistics of the trajectories through the correlation functions of the orientation angular velocity, orientation angle and the mean-squared displacement, and we find that the loss of orientational memory has significant contributions from both abrupt, large amplitude turning events and the continuous dynamics between these events. Further, we demonstrate long-time persistence of orientational memory in the intervals between abrupt turns. Building on recent work demonstrating that C. elegans movements are restricted to a low-dimensional shape space, we construct a map from the dynamics in this shape space to the trajectory of the worm along the agar. We use this connection to illustrate that changes in the continuous dynamics reveal subtle differences in movement strategy that occur among mutants defective in two classes of dopamine receptors

Information based clustering

In an age of increasingly large data sets, investigators in many different disciplines have turned to clustering as a tool for data analysis and exploration. Existing clustering methods, however, typically depend on several nontrivial assumptions about the structure of data. Here we reformulate the clustering problem from an information theoretic perspective which avoids many of these assumptions. In particular, our formulation obviates the need for defining a cluster "prototype", does not require an a priori similarity metric, is invariant to changes in the representation of the data, and naturally captures non-linear relations. We apply this approach to different domains and find that it consistently produces clusters that are more coherent than those extracted by existing algorithms. Finally, our approach provides a way of clustering based on collective notions of similarity rather than the traditional pairwise measures.Comment: To appear in Proceedings of the National Academy of Sciences USA, 11 pages, 9 figure

The role of input noise in transcriptional regulation

Even under constant external conditions, the expression levels of genes fluctuate. Much emphasis has been placed on the components of this noise that are due to randomness in transcription and translation; here we analyze the role of noise associated with the inputs to transcriptional regulation, the random arrival and binding of transcription factors to their target sites along the genome. This noise sets a fundamental physical limit to the reliability of genetic control, and has clear signatures, but we show that these are easily obscured by experimental limitations and even by conventional methods for plotting the variance vs. mean expression level. We argue that simple, global models of noise dominated by transcription and translation are inconsistent with the embedding of gene expression in a network of regulatory interactions. Analysis of recent experiments on transcriptional control in the early Drosophila embryo shows that these results are quantitatively consistent with the predicted signatures of input noise, and we discuss the experiments needed to test the importance of input noise more generally.Comment: 11 pages, 5 figures minor correction