Practical Gauss-Newton Optimisation for Deep Learning

We present an efficient block-diagonal ap- proximation to the Gauss-Newton matrix for feedforward neural networks. Our result- ing algorithm is competitive against state- of-the-art first order optimisation methods, with sometimes significant improvement in optimisation performance. Unlike first-order methods, for which hyperparameter tuning of the optimisation parameters is often a labo- rious process, our approach can provide good performance even when used with default set- tings. A side result of our work is that for piecewise linear transfer functions, the net- work objective function can have no differ- entiable local maxima, which may partially explain why such transfer functions facilitate effective optimisation.Comment: ICML 201

Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting

We introduce the Kronecker factored online Laplace approximation for overcoming catastrophic forgetting in neural networks. The method is grounded in a Bayesian online learning framework, where we recursively approximate the posterior after every task with a Gaussian, leading to a quadratic penalty on changes to the weights. The Laplace approximation requires calculating the Hessian around a mode, which is typically intractable for modern architectures. In order to make our method scalable, we leverage recent block-diagonal Kronecker factored approximations to the curvature. Our algorithm achieves over 90% test accuracy across a sequence of 50 instantiations of the permuted MNIST dataset, substantially outperforming related methods for overcoming catastrophic forgetting.Comment: 13 pages, 6 figure

Determinants of Senior High School Students' Performance in Social Studies in the Central Region of Ghana

The study is an exposition on the determinants of student' performance in social studies in the Central Region of Ghana. Literature was reviewed on the teacher, school and home as determinants of students' performance in Senior High Schools. Past social studies results covering 2006-2009 and 2011 from WAEC on students' performance in Ghana were analysed and used as the basis for the review of literature. It was concluded that many variables contribute to students' performance in social studies across schools in Ghana. Based on the conclusions drawn, it has been recommended for the consideration of relevant stakeholders and researchers in Ghana, to carry out further research into the real and perceived factors that could contribute to students' performance in social studies

Scalable approximate inference methods for Bayesian deep learning

This thesis proposes multiple methods for approximate inference in deep Bayesian neural networks split across three parts. The first part develops a scalable Laplace approximation based on a block- diagonal Kronecker factored approximation of the Hessian. This approximation accounts for parameter correlations – overcoming the overly restrictive independence assumption of diagonal methods – while avoiding the quadratic scaling in the num- ber of parameters of the full Laplace approximation. The chapter further extends the method to online learning where datasets are observed one at a time. As the experiments demonstrate, modelling correlations between the parameters leads to improved performance over the diagonal approximation in uncertainty estimation and continual learning, in particular in the latter setting the improvements can be substantial. The second part explores two parameter-efficient approaches for variational inference in neural networks, one based on factorised binary distributions over the weights, one extending ideas from sparse Gaussian processes to neural network weight matrices. The former encounters similar underfitting issues as mean-field Gaussian approaches, which can be alleviated by a MAP-style method in a hierarchi- cal model. The latter, based on an extension of Matheron’s rule to matrix normal distributions, achieves comparable uncertainty estimation performance to ensembles with the accuracy of a deterministic network while using only 25% of the number of parameters of a single ResNet-50. The third part introduces TyXe, a probabilistic programming library built on top of Pyro to facilitate turning PyTorch neural networks into Bayesian ones. In contrast to existing frameworks, TyXe avoids introducing a layer abstraction, allowing it to support arbitrary architectures. This is demonstrated in a range of applications, from image classification with torchvision ResNets over node labelling with DGL graph neural networks to incorporating uncertainty into neural radiance fields with PyTorch3d

Do Social Studies Teachers’ Variables Influence Students’ Performance in Senior High School Social Studies in Ghana? Evidence from Students’ Performance Test in Social Studies

The study concerned itself with the role of Social Studies teachers’ variables on the performance of Senior High School students’ in social studies in Ghana. The purpose of the study was to find out if social studies teachers’ variables had any influence on the performance of Senior High School (SHS) students’ performance in social studies. A descriptive survey design was used for the study which involved 635 social studies teachers and 74,249 Senior High School 2 students across the 36 Metropolitan, Municipal and District Assemblies (MMDA’S) Ghana. A total of 2,253 SHS 2 students and 75 social studies teachers (30 female and 45 male) from 25 SHS were sampled through multi-stage sampling procedures. Two researcher-designed instruments tagged: Teacher Demographic Variables Questionnaire (TDVQ) and Students’ Performance Test in Social Studies (SPTISS) were used in data collection. Data was analysed with the aid of frequencies and percentage counts. The results showed that, teacher variables exerted some influence on students’ performance in social studies. It was therefore concluded that social studies teachers’ variables exerted significant influence on the performance of Senior High School students in social studies. Based on the results and conclusion, it is suggested for consideration of relevant stakeholders in education, that social studies teacher variables should be considered in the recruitment of teachers for the teaching of social studies at the SHS level. Keywords: Teacher, Variables, Students’, Performance, Social Studie

Black-box Coreset Variational Inference

Recent advances in coreset methods have shown that a selection of representative datapoints can replace massive volumes of data for Bayesian inference, preserving the relevant statistical information and significantly accelerating subsequent downstream tasks. Existing variational coreset constructions rely on either selecting subsets of the observed datapoints, or jointly performing approximate inference and optimizing pseudodata in the observed space akin to inducing points methods in Gaussian Processes. So far, both approaches are limited by complexities in evaluating their objectives for general purpose models, and require generating samples from a typically intractable posterior over the coreset throughout inference and testing. In this work, we present a black-box variational inference framework for coresets that overcomes these constraints and enables principled application of variational coresets to intractable models, such as Bayesian neural networks. We apply our techniques to supervised learning problems, and compare them with existing approaches in the literature for data summarization and inference.Comment: NeurIPS 202

Addressing Catastrophic Forgetting in Few-Shot Problems

Neural networks are known to suffer from catastrophic forgetting when trained on sequential datasets. While there have been numerous attempts to solve this problem in large-scale supervised classification, little has been done to overcome catastrophic forgetting in few-shot classification problems. We demonstrate that the popular gradient-based model-agnostic meta-learning algorithm (MAML) indeed suffers from catastrophic forgetting and introduce a Bayesian online meta-learning framework that tackles this problem. Our framework utilises Bayesian online learning and meta-learning along with Laplace approximation and variational inference to overcome catastrophic forgetting in few-shot classification problems. The experimental evaluations demonstrate that our framework can effectively achieve this goal in comparison with various baselines. As an additional utility, we also demonstrate empirically that our framework is capable of meta-learning on sequentially arriving few-shot tasks from a stationary task distribution.Comment: ICML 202

Measuring competition in the Olympic Winter Games 1992–2014 using economic indices

Since the early 1990s, competition in the Olympic Winter Games has changed notably in terms of events contested and nations taking part. Despite, these changes, which are overseen by the International Olympic Committee (IOC), the number of medal-winning nations has remained relatively stable. As a first attempt to illustrate this issue on a discipline by discipline basis, economic techniques are used to examine the outcome of competition between 1992 and 2014. The purpose of this paper is to measure: market size; the number of competing nations; and the balance between competitive nations in six disciplines. Focusing on competitive balance, the Herfindahl–Hirschman Index is applied to measure the concentration of domination; while the Przeworski Index is used to quantify instability over time. Important changes are identified in biathlon (2010) and short track (2014). While the change in the former is consistent with the IOC’s substantial increase in biathlon events, the latter can be attributed to athletes changing their nationality. IOC policy-makers can benefit from this research as it provides a method by which to monitor competition in a discipline. This method provides the potential for evaluating the likely effects of governing the Olympic Games by increasing the number of events

Threads and Stitches of Peace- Understanding What Makes Ghana an Oasis of Peace?

Ghana is considered an oasis of peace despite having the same mix of ethno-political competitions for state power and resources; north-south horizontal inequalities; ethno-regional concentrations of Christians and Muslims; highly ethnicised elections; a natural resource dependent economy; and a politically polarized public sphere, among others, that have plunged other countries in Africa into violent and often protracted national conflicts. Use of the conflict paradigm to explain Africa\u27s conflicts glosses over positive deviance cases such as Ghana. This study used the peace paradigm in a mixed method, grounded theory research to examine Ghana\u27s apparent exceptionalism in staving off violent national conflicts. From the survey of 1429 respondents and 31 Key Informants, findings indicate Ghanaians are divided on whether their country is peaceful or not. They are equally divided on classifying the state of peace in Ghana as negative or positive. Instead, they have identified sets of centrifugal and centripetal forces that somehow self-neutralize to keep Ghana in a steady state of unstable peace. Among the lift forces are strongly shared cultural and Indigenous African Religious values; symbiotic interethnic economic relationships; identity dissolution and cultural miscegenation due to open interethnic systems of accommodation and incorporation; and the persistence of historical multi-lateral political, sociocultural, and economic relationships. On the drag side are the youth bulge; emergent religious intolerance; elite exit from the state in using private solutions for public problems; and highly politicized and partisan national discourses that leave the country with no national agenda. In sum, Ghana is no exception to the rule. The four interconnected meso theories that this study identifies provide pointers to what factors Ghana needs to strengthen to avert descent into violence