Fitness-based growth of directed networks with hierarchy

Growing attention has been brought to the fact that many real directed networks exhibit hierarchy and directionality as measured through techniques like trophic analysis and non-normality. We propose a simple growing network model where the probability of connecting to a node is defined by a preferential attachment mechanism based on degree and the difference in fitness between nodes. In particular, we show how mechanisms such as degree-based preferential attachment and node fitness interactions can lead to the emergence of the spectrum of hierarchy and directionality observed in real networks. In this work, we study various features of this model relating to network hierarchy, as measured by trophic analysis. This includes (I) how preferential attachment can lead to network hierarchy, (II) how scale-free degree distributions and network hierarchy can coexist, (III) the correlation between node fitness and trophic level, (IV) how the fitness parameters can predict trophic incoherence and how the trophic level difference distribution compares to the fitness difference distribution, (V) the relationship between trophic level and degree imbalance and the unique role of nodes at the ends of the fitness hierarchy and (VI) how fitness interactions and degree-based preferential attachment can interplay to generate networks of varying coherence and degree distribution. We also provide an example of the intuition this work enables in the analysis of a real historical network. This work provides insight into simple mechanisms which can give rise to hierarchy in directed networks and quantifies the usefulness and limitations of using trophic analysis as an analysis tool for real networks

Simple Cycle Reservoirs are Universal

Reservoir computation models form a subclass of recurrent neural networks with fixed non-trainable input and dynamic coupling weights. Only the static readout from the state space (reservoir) is trainable, thus avoiding the known problems with propagation of gradient information backwards through time. Reservoir models have been successfully applied in a variety of tasks and were shown to be universal approximators of time-invariant fading memory dynamic filters under various settings. Simple cycle reservoirs (SCR) have been suggested as severely restricted reservoir architecture, with equal weight ring connectivity of the reservoir units and input-to-reservoir weights of binary nature with the same absolute value. Such architectures are well suited for hardware implementations without performance degradation in many practical tasks. In this contribution, we rigorously study the expressive power of SCR in the complex domain and show that they are capable of universal approximation of any unrestricted linear reservoir system (with continuous readout) and hence any time-invariant fading memory filter over uniformly bounded input streams.Comment: 21 page

A Survey on Neural Network Interpretability

Along with the great success of deep neural networks, there is also growing concern about their black-box nature. The interpretability issue affects people's trust on deep learning systems. It is also related to many ethical problems, e.g., algorithmic discrimination. Moreover, interpretability is a desired property for deep networks to become powerful tools in other research fields, e.g., drug discovery and genomics. In this survey, we conduct a comprehensive review of the neural network interpretability research. We first clarify the definition of interpretability as it has been used in many different contexts. Then we elaborate on the importance of interpretability and propose a novel taxonomy organized along three dimensions: type of engagement (passive vs. active interpretation approaches), the type of explanation, and the focus (from local to global interpretability). This taxonomy provides a meaningful 3D view of distribution of papers from the relevant literature as two of the dimensions are not simply categorical but allow ordinal subcategories. Finally, we summarize the existing interpretability evaluation methods and suggest possible research directions inspired by our new taxonomy.Comment: This work has been accepted by IEEE-TETC

Kernel regression estimates of time delays between gravitationally lensed fluxes

Strongly lensed variable quasars can serve as precise cosmological probes, provided that time delays between the image fluxes can be accurately measured. A number of methods have been proposed to address this problem. In this paper, we explore in detail a new approach based on kernel regression estimates, which is able to estimate a single time delay given several datasets for the same quasar. We develop realistic artificial data sets in order to carry out controlled experiments to test of performance of this new approach. We also test our method on real data from strongly lensed quasar Q0957+561 and compare our estimates against existing results.Comment: Updated to match published versio

Dynamics and topographic organization of recursive self-organizing maps

Recently there has been an outburst of interest in extending topographic maps of vectorial data to more general data structures, such as sequences or trees. However, there is no general consensus as to how best to process sequences using topographicmaps, and this topic remains an active focus of neurocomputational research. The representational capabilities and internal representations of the models are not well understood. Here, we rigorously analyze a generalization of the self-organizingmap (SOM) for processing sequential data, recursive SOM (RecSOM) (Voegtlin, 2002), as a nonautonomous dynamical system consisting of a set of fixed input maps. We argue that contractive fixed-input maps are likely to produce Markovian organizations of receptive fields on the RecSOM map. We derive bounds on parameter β (weighting the importance of importing past information when processing sequences) under which contractiveness of the fixed-input maps is guaranteed. Some generalizations of SOM contain a dynamic module responsible for processing temporal contexts as an integral part of the model. We show that Markovian topographic maps of sequential data can be produced using a simple fixed (nonadaptable) dynamic module externally feeding a standard topographic model designed to process static vectorial data of fixed dimensionality (e.g., SOM). However, by allowing trainable feedback connections, one can obtain Markovian maps with superior memory depth and topography preservation. We elaborate on the importance of non-Markovian organizations in topographic maps of sequential data. © 2006 Massachusetts Institute of Technology

Spatial-temporal modelling of fMRI data through spatially regularized mixture of hidden process models.

Previous work investigated a range of spatio-temporal constraints for fMRI data analysis to provide robust detection of neural activation. We present a mixture-based method for the spatio-temporal modelling of fMRI data. This approach assumes that fMRI time series are generated by a probabilistic superposition of a small set of spatio-temporal prototypes (mixture components). Each prototype comprises a temporal model that explains fMRI signals on a single voxel and the model's "region of influence" through a spatial prior over the voxel space. As the key ingredient of our temporal model, the Hidden Process Model (HPM) framework proposed in Hutchinson et al. (2009) is adopted to infer the overlapping cognitive processes triggered by stimuli. Unlike the original HPM framework, we use a parametric model of Haemodynamic Response Function (HRF) so that biological constraints are naturally incorporated in the HRF estimation. The spatial priors are defined in terms of a parameterised distribution. Thus, the total number of parameters in the model does not depend on the number of voxels. The resulting model provides a conceptually principled and computationally efficient approach to identify spatio-temporal patterns of neural activation from fMRI data, in contrast to most conventional approaches in the literature focusing on the detection of spatial patterns. We first verify the proposed model in a controlled experimental setting using synthetic data. The model is further validated on real fMRI data obtained from a rapid event-related visual recognition experiment (Mayhew et al., 2012). Our model enables us to evaluate in a principled manner the variability of neural activations within individual regions of interest (ROIs). The results strongly suggest that, compared with occipitotemporal regions, the frontal ones are less homogeneous, requiring two HPM prototypes per region. Despite the rapid event-related experimental design, the model is capable of disentangling the perceptual judgement and motor response processes that are both activated in the frontal ROIs. Spatio-temporal heterogeneity in the frontal regions seems to be associated with diverse dynamic localizations of the two hidden processes in different subregions of frontal ROIs

A new framework for analysis of coevolutionary systems:Directed graph representation and random walks

Studying coevolutionary systems in the context of simplified models (i.e. games with pairwise interactions between coevolving solutions modelled as self plays) remains an open challenge since the rich underlying structures associated with pairwise comparison-based fitness measures are often not taken fully into account. Although cyclic dynamics have been demonstrated in several contexts (such as intransitivity in coevolutionary problems), there is no complete characterization of cycle structures and their effects on coevolutionary search. We develop a new framework to address this issue. At the core of our approach is the directed graph (digraph) representation of coevolutionary problem that fully captures structures in the relations between candidate solutions. Coevolutionary processes are modelled as a specific type of Markov chains ? random walks on digraphs. Using this framework, we show that coevolutionary problems admit a qualitative characterization: a coevolutionary problem is either solvable (there is a subset of solutions that dominates the remaining candidate solutions) or not. This has an implication on coevolutionary search. We further develop our framework that provide the means to construct quantitative tools for analysis of coevolutionary processes and demonstrate their applications through case studies. We show that coevolution of solvable problems corresponds to an absorbing Markov chain for which we can compute the expected hitting time of the absorbing class. Otherwise, coevolution will cycle indefinitely and the quantity of interest will be the limiting invariant distribution of the Markov chain. We also provide an index for characterizing complexity in coevolutionary problems and show how they can be generated in a controlled mannerauthorsversionPeer reviewe

Unsupervised multimodal modeling of cognitive and brain health trajectories for early dementia prediction

Predicting the course of neurodegenerative disorders early has potential to greatly improve clinical management and patient outcomes. A key challenge for early prediction in real-world clinical settings is the lack of labeled data (i.e., clinical diagnosis). In contrast to supervised classification approaches that require labeled data, we propose an unsupervised multimodal trajectory modeling (MTM) approach based on a mixture of state space models that captures changes in longitudinal data (i.e., trajectories) and stratifies individuals without using clinical diagnosis for model training. MTM learns the relationship between states comprising expensive, invasive biomarkers (β-amyloid, grey matter density) and readily obtainable cognitive observations. MTM training on trajectories stratifies individuals into clinically meaningful clusters more reliably than MTM training on baseline data alone and is robust to missing data (i.e., cognitive data alone or single assessments). Extracting an individualized cognitive health index (i.e., MTM-derived cluster membership index) allows us to predict progression to AD more precisely than standard clinical assessments (i.e., cognitive tests or MRI scans alone). Importantly, MTM generalizes successfully from research cohort to real-world clinical data from memory clinic patients with missing data, enhancing the clinical utility of our approach. Thus, our multimodal trajectory modeling approach provides a cost-effective and non-invasive tool for early dementia prediction without labeled data (i.e., clinical diagnosis) with strong potential for translation to clinical practice