Devising New Models for School Improvement in Developing Nations: Sierra Leone, a case study

Abstract Background and Introduction: This research describes the planning, design, implementation and evaluation of a (Continuous Professional Development & Learning) CPDL-based programme for schools in Sierra Leone, a low-income country with low educational standards. Aims: The research aimed to: (i) assess evidence of EducAid schools’ effectiveness; (ii) identify features of EducAid practice that government schools might adopt; (iii) design a CPDL programme for Sierra Leonean teachers; (iv) report the programme’s impact on students’ progress; (v) explore the possibility of programme delivery by local and largely untrained teachers; (vi) throw light on aspects of the programme that participants saw as strengths and weaknesses. Design and Methods: Within an innovative quasi-experimental design, an impact evaluation drew on data from five intervention and ten comparison schools, and a process evaluation drew on data on information from programme participants and the trainers. The impact evaluation was based on public exam results, literacy test scores and attendance data collected pre- and post-CPDL. Process data included information from lesson observations, semi-structured interviews, focus groups and a post-intervention evaluation workshop. Results: The impact evaluation showed larger improvements in student attendance and literacy test results in the intervention schools than in the comparison schools. The process evaluation identified challenges in embedding changes in pedagogic practice, and in data collection. However, it also identified consistent evidence of improvements in student behaviour. These were supported by head teachers and community groups and were seen as a necessary but not sufficient condition for the literacy score improvements. Discussion and Conclusions: Discussion focuses on how far the six aims were met and on how the research adds to understanding of CPDL and school improvement in a low-income country. The impossibility of randomisation in sample selection prevents any strong causal claims for the CPDL’s impact. The possibility of a larger scale roll-out is considered, subject to changes in the programme suggested by the process evaluation

Personalised exercise recognition towards improved self-management of musculoskeletal disorders.

Musculoskeletal Disorders (MSD) have been the primary contributor to the global disease burden, with increased years lived with disability. Such chronic conditions require self-management, typically in the form of maintaining an active lifestyle while adhering to prescribed exercises. Today, exercise monitoring in fitness applications wholly relies on user input. Effective digital intervention for self-managing MSD should be capable of monitoring, recognising and assessing performance quality of exercises in real-time. Exercise Recognition (ExRec) is the machine learning problem that investigates the automation of exercise monitoring. Multiple challenges arise when implementing high performing ExRec algorithms for a wide range of exercises performed by people from different demographics. In this thesis, we explore three personalisation challenges. Different sensor combinations can be used to capture exercises, to improve usability and deployability in restricted settings. Accordingly, a recognition algorithm should be adaptable to different sensor combinations. To address this challenge, we investigate the best feature learners for individual sensors, and effective fusion methods that minimise the need for data and very deep architectures. We implement a modular hybrid attention fusion architecture that emphasises significant features and understates noisy features from multiple sensors for each exercise. Persons perform exercises differently when not supervised; they incorporate personal rhythms and nuances. Accordingly, a recognition algorithm should be able to adapt to different persons. To address the personalised recognition challenge, we investigate how to adapt learned models to new, unseen persons. Key to achieving effective personalisation is the ability to personalise with few data instances. Accordingly, we bring together personalisation methods and advances in meta-learning to introduce personalised meta-learning methodology. The resulting personalised meta-learners are learning to adapt to new end-users with only few data instances. It is infeasible to design algorithms to recognise all expected exercises a physiotherapist would prescribe. Accordingly, the ability to integrate new exercises after deployment is another challenge in ExRec. The challenge of adapting to unseen exercises is known as open-ended recognition. We extend the personalised meta-learning methodology to the open-ended domain, such that an end-user can introduce a new exercise to the model with only a few data instances. Finally, we address the lack of publicly available data and collaborate with health science researchers to curate a heterogeneous multi-modal physiotherapy exercise dataset, MEx. We conduct comprehensive evaluations of the proposed methods using MEx to demonstrate that our methods successfully address the three ExRec challenges. We also show that our contributions are not restricted to the domain of ExRec, but are applicable in a wide range of activity recognition tasks by extending the evaluation to other human activity recognition domains

DeepVATS : Deep Visual Analytics for time series

The field of Deep Visual Analytics (DVA) has recently arisen from the idea of developing Visual Interactive Systems supported by deep learning, in order to provide them with large-scale data processing capabilities and to unify their implementation across different data and domains. In this paper we present DeepVATS, an open-source tool that brings the field of DVA into time series data. DeepVATS trains, in a self-supervised way, a masked time series autoencoder that reconstructs patches of a time series, and projects the knowledge contained in the embeddings of that model in an interactive plot, from which time series patterns and anomalies emerge and can be easily spotted. The tool includes a back-end for data processing pipeline and model training, as well as a front-end with an interactive user interface. We report on results that validate the utility of DeepVATS, running experiments on both synthetic and real datasets. The code is publicly available on https://github.com/vrodriguezf/deepvats

Group optimization to maximize peer assessment accuracy using item response theory and integer programming

With the wide spread of large-scale e-learning environments such as MOOCs, peer assessment has been popularly used to measure learner ability. When the number of learners increases, peer assessment is often conducted by dividing learners into multiple groups to reduce the learner\u27s assessment workload. However, in such cases, the peer assessment accuracy depends on the method of forming groups. To resolve that difficulty, this study proposes a group formation method to maximize peer assessment accuracy using item response theory and integer programming. Experimental results, however, have demonstrated that the proposed method does not present sufficiently higher accuracy than a random group formation method does. Therefore, this study further proposes an external rater assignment method that assigns a few outside-group raters to each learner after groups are formed using the proposed group formation method. Through results of simulation and actual data experiments, this study demonstrates that the proposed external rater assignment can substantially improve peer assessment accuracy

A study of university law students’ self-perceived digital competences.

The concept of digital competences incorporates the effective use of constantly-changing digital tools and media for learning and performing digital tasks, digital behaviours (such as online communication, teamwork, ethical sharing of information), as well as digital mindsets that value lifelong digital learning and development. The current pandemic crisis has accelerated the need to diagnose and understand more systematically Higher Education students’ digital competences and the way in which they shape academic performance and outcomes. This empirical study explores the digital competences of students, studying in Law related courses, by means of a self-assessment survey tool, which has been previously tested with information and library science students, and was developed to study students’ technology mastery (i.e. the abilities, competences, capabilities and skills required for using digital technology, media and tools) and their digital citizenship mindsets (consisting of attitudes and behaviours necessary to develop as a critical, reflective and lifelong learners). The study found age demographic differences, which presented significant correlations pointing to the presence of diverse levels of competences in the student group. Correlation statistics of the survey data demonstrated that students’ prior everyday participation as a digital citizen was connected to a number of important academic skills, such as the ability to identify information in different contexts, students’ digital learning and development, their digital abilities to complete academic work, their information literacy skills and their skills around managing their digital wellbeing and identity. Focus groups data with academics revealed that they valued the development of students’ digital competences for the purposes of learning, while studying at university and placed less emphasis on digital citizenship skills. These academics also considered the value of digital platforms and tools (the focus on ‘ICT Proficiency’) to be more relevant for academic study than digital citizenship mindsets

Designing dissemination and validation of a framework for teaching cloud fundamentals

Three previous Working Groups (WGs) met at ITiCSE conferences to explore ways to help educators incorporate cloud computing into their courses and curricula by mapping industry job skills to knowledge areas (KAs). These WGs identified, organized, and grouped together student learning objectives (LOs) and developed these KAs and LOs in a repository of learning materials and course exemplars. This WG focused on the sustainability of the work of its predecessors through dissemination, community building and validation of the framework of KAs and LOs and its contribution to curriculum development. Firstly, a case study is presented which analyzed the implementation of a new Masters program which was based on the KAs and LOs. It was found that these provide a useful basis for program development and approval and demonstrate that successful program development of this nature can provide a valuable opportunity to communicate the work of the previous WGs. Thereafter, a plan was formulated for dissemination of the work done in order to drive adoption and to encourage instructors with an interest in teaching cloud computing to participate and grow the community. While the strategy included a range of dissemination methods, the importance of interaction with users was a guiding principle. Initial pilots of webinar and workshop activities have been implemented. Approaches to validating that a cloud computing course designed around the KAs and LOs can meet the needs of industry have been outlined with further iterations being considered. A research plan has been designed for a study to be implemented over the coming year in order to perform this validation

Learning to de-anonymize social networks

Releasing anonymized social network data for analysis has been a popular idea among data providers. Despite evidence to the contrary the belief that anonymization will solve the privacy problem in practice refuses to die. This dissertation contributes to the field of social graph de-anonymization by demonstrating that even automated models can be quite successful in breaching the privacy of such datasets. We propose novel machine-learning based techniques to learn the identities of nodes in social graphs, thereby automating manual, heuristic-based attacks. Our work extends the vast literature of social graph de-anonymization attacks by systematizing them. We present a random-forests based classifier which uses structural node features based on neighborhood degree distribution to predict their similarity. Using these simple and efficient features we design versatile and expressive learning models which can learn the de-anonymization task just from a few examples. Our evaluation establishes their efficacy in transforming de-anonymization to a learning problem. The learning is transferable in that the model can be trained to attack one graph when trained on another. Moving on, we demonstrate the versatility and greater applicability of the proposed model by using it to solve the long-standing problem of benchmarking social graph anonymization schemes. Our framework bridges a fundamental research gap by making cheap, quick and automated analysis of anonymization schemes possible, without even requiring their full description. The benchmark is based on comparison of structural information leakage vs. utility preservation. We study the trade-off of anonymity vs. utility for six popular anonymization schemes including those promising k-anonymity. Our analysis shows that none of the schemes are fit for the purpose. Finally, we present an end-to-end social graph de-anonymization attack which uses the proposed machine learning techniques to recover node mappings across intersecting graphs. Our attack enhances the state of art in graph de-anonymization by demonstrating better performance than all the other attacks including those that use seed knowledge. The attack is seedless and heuristic free, which demonstrates the superiority of machine learning techniques as compared to hand-selected parametric attacks

The contribution of Randomised Control Trials (RCTs) to improving education evaluations for policy: evidence from developing countries and South African case studies

A research report submitted to the Wits School of Education, University of Witwatersrand, in partial fulfilment of the requirements for the degree Master of Education Submission 17 October 2016As access to formal schooling has expanded all over the world, there is acknowledgement that the quality of learning in many schooling systems, including South Africa, is extremely weak. Nationally representative samples of South African children participated in the PIRLS 2006 and pre-PIRLS 2011 studies, along with 48 other countries as a benchmarking exercise to measure the literacy levels of primary schools according to international standards. The PIRLS 2006 study indicated that more than 80% of South African children had not yet learned to read with meaning by grade 5. The pre-PIRLS results provided a new baseline of reading literacy levels for Grade 4 learners in South Africa, 29% of Grade 4 learners that participated did not have the rudimentary reading skills required at a Grade 2 level. Learners tested in African languages, particularly Sepedi and Tshivenda, achieved the lowest performance overall and were considered to be educationally at risk (University of Pretoria, 2012). The context in which schooling takes place is key in understanding learner performance in South Africa. After decades of differential provision of education on the basis of race, the education system has been overhauled since the early 1990s. The South African government has introduced several initiatives and policies to address these systemic imbalances. All things considered, South Africa’s learner performance has remained poor, even relative to several poorer countries in the region. There is a wealth of research describing weaknesses in the education system. However, going a step further and identifying resources and practices that actually improve learner performance is central to improving education planning, policy and ultimately classroom practice. Rigorous evidence on classroom-based practice and resources that will have a measurable effect on learner performance in a developing country like South Africa is limited. The most significant shortfall of non-experimental evaluation methods (including qualitative and many quantitative approaches) is the absence of a valid estimate of the counterfactual – what outcomes would have been obtained amongst programme beneficiaries had they not received the programme. This often leads to the reporting of large positive effects of programmes being evaluated. By using a lottery to allocate participants to an intervention and a control group, the Randomised Control Trial (RCT) methodology constructs a credible ‘counterfactual’ scenario – what might have happened to those who received an intervention had they not received it. This study provides a systematic literature-based argument on why RCTs should be part of the methodological options education researchers and policy makers consider in developing countries such as South Africa. Both the strengths and limitations of RCTs are discussed in light of the debate on RCTs and evaluation methods in education, as well as the technical critique of the methodology. The main critique of external validity is also elaborated on with efforts that may be taken to diminish the limitations discussed. In addition, the study illustrates the value of RCTs using data from two South Africa RCTs on early grade reading interventions through a secondary analysis of the RCT data. The first case study in Chapter 4, is the Reading Catch-Up Programme (RCUP) conducted in Pinetown, KwaZulu-Natal. The main findings of the RCUP evaluation were that although learners in intervention schools improved their test scores between the baseline and the endline assessment, the learners in comparison schools improved by a similar margin. The results should contribute to a sobering realisation that the effects of the various interventions introduced by education stakeholders including NGOs and government are not obviously positive or more importantly, different from normal schooling. This points to the need to evaluate programmes before they are rolled out provincially or nationally, using RCTs and other rigorous methods. The new analysis of data in this study explores the so-called “Matthew Effect” - the notion that initially better-performing children typically gain more from additional interventions and from schooling itself. The data from the RCUP RCT indicates that children with higher baseline test scores benefited from the intervention, whereas children with very low English proficiency at the outset did not benefit from the programme. Although females significantly outperform males in the reading tests used, there was no clear evidence of a differential effect of the intervention by gender. The Matthew Effect therefore seems to be driven by prior knowledge and not gender or any other characteristic that was measured in the data. The second case study in Chapter 5, is the Early Grade Reading Study (EGRS) conducted in the North West province. The EGRS may be seen as a more extensive follow-up to the RCUP to answer some of the unanswered questions. For example, will an early grade reading intervention that is implemented over a longer duration (two years) have an impact? Can intervening right at the start of school be a strategic point to intervene? Can a Home Language literacy intervention have lasting educational benefits? In conclusion, although the policy formulation and evaluation process should draw on research using a variety of methods, the policy process will certainly be impoverished if there is a lack of research meeting two core criteria: interventions and findings that are relevant to the larger schooling population; and the precise measurement of the causal impact of interventions and/or policies. This study makes a clear literature-based argument on the contribution of internally valid methods, specifically RCTs in fulfilling these criteria and illustrates this with two case studies of RCTS. The study also provides a demonstration of the insights that are possible through secondary analysis founded on the richness of RCT data.MT201