Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Networks

The core operation of current Graph Neural Networks (GNNs) is the aggregation enabled by the graph Laplacian or message passing, which filters the neighborhood node information. Though effective for various tasks, in this paper, we show that they are potentially a problematic factor underlying all GNN methods for learning on certain datasets, as they force the node representations similar, making the nodes gradually lose their identity and become indistinguishable. Hence, we augment the aggregation operations with their dual, i.e. diversification operators that make the node more distinct and preserve the identity. Such augmentation replaces the aggregation with a two-channel filtering process that, in theory, is beneficial for enriching the node representations. In practice, the proposed two-channel filters can be easily patched on existing GNN methods with diverse training strategies, including spectral and spatial (message passing) methods. In the experiments, we observe desired characteristics of the models and significant performance boost upon the baselines on 9 node classification tasks

Differentiable Genetic Programming for High-dimensional Symbolic Regression

Symbolic regression (SR) is the process of discovering hidden relationships from data with mathematical expressions, which is considered an effective way to reach interpretable machine learning (ML). Genetic programming (GP) has been the dominator in solving SR problems. However, as the scale of SR problems increases, GP often poorly demonstrates and cannot effectively address the real-world high-dimensional problems. This limitation is mainly caused by the stochastic evolutionary nature of traditional GP in constructing the trees. In this paper, we propose a differentiable approach named DGP to construct GP trees towards high-dimensional SR for the first time. Specifically, a new data structure called differentiable symbolic tree is proposed to relax the discrete structure to be continuous, thus a gradient-based optimizer can be presented for the efficient optimization. In addition, a sampling method is proposed to eliminate the discrepancy caused by the above relaxation for valid symbolic expressions. Furthermore, a diversification mechanism is introduced to promote the optimizer escaping from local optima for globally better solutions. With these designs, the proposed DGP method can efficiently search for the GP trees with higher performance, thus being capable of dealing with high-dimensional SR. To demonstrate the effectiveness of DGP, we conducted various experiments against the state of the arts based on both GP and deep neural networks. The experiment results reveal that DGP can outperform these chosen peer competitors on high-dimensional regression benchmarks with dimensions varying from tens to thousands. In addition, on the synthetic SR problems, the proposed DGP method can also achieve the best recovery rate even with different noisy levels. It is believed this work can facilitate SR being a powerful alternative to interpretable ML for a broader range of real-world problems

Using Q methodology to investigate pre-service EFL teachers’ mindsets about teaching competences

This paper reports on a study investigating the mindsets of 51 pre-service teachers at an Austrian university using Q methodology. Despite the recent growth in interest in the concept of mindsets, little research has addressed the mindsets of teachers – most of it focusing on the mindsets of learners – and the research that does investigate teachers tends to focus on beliefs about learning or intelligence. This study offers a new perspective by focusing on teachers’ beliefs about their own teaching competences. A further aim of the study is to expand the methodological repertoire in language education researchers. This study considers the potential of Q methodology, a research approach used widely in social sciences and education, but, as yet, rare in this field. The data indicate that the most common mindset among the pre-service teachers is one based around a strong belief in the learnability of the more technical aspects of teaching, while interpersonal skills tend to be regarded as more of a natural talent fixed within the individual. One practical implication of this finding is that teacher education programmes may need to pay more attention to explicitly developing the interpersonal side of teaching. A further finding was that teacher mindsets are constructed through individuals’ management of various sets of implicit theories and tend not to conform to the established dichotomous model of mindsets.

Robustifying learnability

In recent years, the learnability of rational expectations equilibria (REE) and determinacy of economic structures have rightfully joined the usual performance criteria among the sought-after goals of policy design. Some contributions to the literature, including Bullard and Mitra (2001) and Evans and Honkapohja (2002), have made significant headway in establishing certain features of monetary policy rules that facilitate learning. However a treatment of policy design for learnability in worlds where agents have potentially misspecified their learning models has yet to surface. This paper provides such a treatment. We begin with the notion that because the profession has yet to settle on a consensus model of the economy, it is unreasonable to expect private agents to have collective rational expectations. We assume that agents have only an approximate understanding of the workings of the economy and that their learning the reduced forms of the economy is subject to potentially destabilizing perturbations. The issue is then whether a central bank can design policy to account for perturbations and still assure the learnability of the model. Our test case is the standard New Keynesian business cycle model. For different parameterizations of a given policy rule, we use structured singular value analysis (from robust control theory) to find the largest ranges of misspecifications that can be tolerated in a learning model without compromising convergence to an REE. In addition, we study the cost, in terms of performance in the steady state of a central bank that acts to robustify learnability on the transition path to REE. (Note: This paper contains full-color graphics) JEL Classification: C6, E5E-stability, learnability, Learning, monetary policy, robust control