Training Matters: Unlocking Potentials of Deeper Graph Convolutional Neural Networks

The performance limit of Graph Convolutional Networks (GCNs) and the fact that we cannot stack more of them to increase the performance, which we usually do for other deep learning paradigms, are pervasively thought to be caused by the limitations of the GCN layers, including insufficient expressive power, etc. However, if so, for a fixed architecture, it would be unlikely to lower the training difficulty and to improve performance by changing only the training procedure, which we show in this paper not only possible but possible in several ways. This paper first identify the training difficulty of GCNs from the perspective of graph signal energy loss. More specifically, we find that the loss of energy in the backward pass during training nullifies the learning of the layers closer to the input. Then, we propose several methodologies to mitigate the training problem by slightly modifying the GCN operator, from the energy perspective. After empirical validation, we confirm that these changes of operator lead to significant decrease in the training difficulties and notable performance boost, without changing the composition of parameters. With these, we conclude that the root cause of the problem is more likely the training difficulty than the others

Incorporation of causality structures to complex network analysis of time-varying behaviour of multivariate time series

This paper presents a new methodology for characterising the evolving behaviour of the time-varying causality between multivariate time series, from the perspective of change in the structure of the causality pattern. We propose that such evolutionary behaviour should be tracked by means of a complex network whose nodes are causality patterns and edges are transitions between those patterns of causality. In our new methodology each edge has a weight that includes the frequency of the given transition and two metrics relating to the gross and net structural change in causality pattern, which we call [Formula: see text] and [Formula: see text]. To characterise aspects of the behaviour within this network, five approaches are presented and motivated. To act as a demonstration of this methodology an application of sample data from the international oil market is presented. This example illustrates how our new methodology is able to extract information about evolving causality behaviour. For example, it reveals non-random time-varying behaviour that favours transitions resulting in predominantly similar causality patterns, and it discovers clustering of similar causality patterns and some transitional behaviour between these clusters. The example illustrates how our new methodology supports the inference that the evolution of causality in the system is related to the addition or removal of a few causality links, primarily keeping a similar causality pattern, and that the evolution is not related to some other measure such as the overall number of causality links

Community Detection in Multimodal Networks

Community detection on networks is a basic, yet powerful and ever-expanding set of methodologies that is useful in a variety of settings. This dissertation discusses a range of different community detection on networks with multiple and non-standard modalities. A major focus of analysis is on the study of networks spanning several layers, which represent relationships such as interactions over time, different facets of high-dimensional data. These networks may be represented by several different ways; namely the few-layer (i.e. longitudinal) case as well as the many-layer (time-series cases). In the first case, we develop a novel application of variational expectation maximization as an example of the top-down mode of simultaneous community detection and parameter estimation. In the second case, we use a bottom-up strategy of iterative nodal discovery for these longer time-series, abetted with the assumption of their structural properties. In addition, we explore significantly self-looping networks, whose features are inseparable from the inherent construction of spatial networks whose weights are reflective of distance information. These types of networks are used to model and demarcate geographical regions. We also describe some theoretical properties and applications of a method for finding communities in bipartite networks that are weighted by correlations between samples. We discuss different strategies for community detection in each of these different types of networks, as well as their implications for the broader contributions to the literature. In addition to the methodologies, we also highlight the types of data wherein these ``non-standard" network structures arise and how they are fitting for the applications of the proposed methodologies: particularly spatial networks and multilayer networks. We apply the top-down and bottom-up community detection algorithms to data in the domains of demography, human mobility, genomics, climate science, psychiatry, politics, and neuroimaging. The expansiveness and diversity of these data speak to the flexibility and ubiquity of our proposed methods to all forms of relational data.Doctor of Philosoph