Learning Objective-Specific Active Learning Strategies with Attentive Neural Processes

Pool-based active learning (AL) is a promising technology for increasing data-efficiency of machine learning models. However, surveys show that performance of recent AL methods is very sensitive to the choice of dataset and training setting, making them unsuitable for general application. In order to tackle this problem, the field Learning Active Learning (LAL) suggests to learn the active learning strategy itself, allowing it to adapt to the given setting. In this work, we propose a novel LAL method for classification that exploits symmetry and independence properties of the active learning problem with an Attentive Conditional Neural Process model. Our approach is based on learning from a myopic oracle, which gives our model the ability to adapt to non-standard objectives, such as those that do not equally weight the error on all data points. We experimentally verify that our Neural Process model outperforms a variety of baselines in these settings. Finally, our experiments show that our model exhibits a tendency towards improved stability to changing datasets. However, performance is sensitive to choice of classifier and more work is necessary to reduce the performance the gap with the myopic oracle and to improve scalability. We present our work as a proof-of-concept for LAL on nonstandard objectives and hope our analysis and modelling considerations inspire future LAL work.Comment: Accepted at ECML 202

Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement

The well-known Gumbel-Max trick for sampling from a categorical distribution can be extended to sample $k$ elements without replacement. We show how to implicitly apply this 'Gumbel-Top-$k$' trick on a factorized distribution over sequences, allowing to draw exact samples without replacement using a Stochastic Beam Search. Even for exponentially large domains, the number of model evaluations grows only linear in $k$ and the maximum sampled sequence length. The algorithm creates a theoretical connection between sampling and (deterministic) beam search and can be used as a principled intermediate alternative. In a translation task, the proposed method compares favourably against alternatives to obtain diverse yet good quality translations. We show that sequences sampled without replacement can be used to construct low-variance estimators for expected sentence-level BLEU score and model entropy.Comment: ICML 2019 ; 13 pages, 4 figure

Attention, Learn to Solve Routing Problems!

The recently presented idea to learn heuristics for combinatorial optimization problems is promising as it can save costly development. However, to push this idea towards practical implementation, we need better models and better ways of training. We contribute in both directions: we propose a model based on attention layers with benefits over the Pointer Network and we show how to train this model using REINFORCE with a simple baseline based on a deterministic greedy rollout, which we find is more efficient than using a value function. We significantly improve over recent learned heuristics for the Travelling Salesman Problem (TSP), getting close to optimal results for problems up to 100 nodes. With the same hyperparameters, we learn strong heuristics for two variants of the Vehicle Routing Problem (VRP), the Orienteering Problem (OP) and (a stochastic variant of) the Prize Collecting TSP (PCTSP), outperforming a wide range of baselines and getting results close to highly optimized and specialized algorithms.Comment: Accepted at ICLR 2019. 25 pages, 7 figure

Class-Level School Performance and Life Satisfaction: Differential Sensitivity for Low- and High-Performing School-Aged Children

Rathmann K, Herke M, Bilz L, Rimpelä A, Hurrelmann K. Class-Level School Performance and Life Satisfaction: Differential Sensitivity for Low- and High-Performing School-Aged Children. International Journal of Environmental Research and Public Health. 2018;15(12): 2750

MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning

This paper introduces MDP homomorphic networks for deep reinforcement learning. MDP homomorphic networks are neural networks that are equivariant under symmetries in the joint state-action space of an MDP. Current approaches to deep reinforcement learning do not usually exploit knowledge about such structure. By building this prior knowledge into policy and value networks using an equivariance constraint, we can reduce the size of the solution space. We specifically focus on group-structured symmetries (invertible transformations). Additionally, we introduce an easy method for constructing equivariant network layers numerically, so the system designer need not solve the constraints by hand, as is typically done. We construct MDP homomorphic MLPs and CNNs that are equivariant under either a group of reflections or rotations. We show that such networks converge faster than unstructured baselines on CartPole, a grid world and Pong

Perceived class climate and school-aged children’s life satisfaction: The role of the learning environment in classrooms

The aim of this study is to examine the impact of class-level class climate on school-aged children’s life satisfaction. Data was derived from the German National Educational Panel Study (NEPS) using sixth grade school-aged children (n = 4,764, 483 classes). Class climate includes indicators of teachers' care and monitoring, demands, interaction, autonomy, as well as school-aged children's attitudes towards schoolwork at the class- and individual-level. Results showed that individual perceived class climate in terms of teachers' care and monitoring and autonomy was positively related to life satisfaction, whereas school-related demands were related to lower life satisfaction. Besides teachers' care and monitoring at class-level, indicators of class climate were not associated with school-aged children’s life satisfaction, while the individual perceived class climate is more important for life satisfaction

Lebenszufriedenheit von Schülerinnen und Schülern mit sonderpädagogischem Förderbedarf: Gibt es Unterschiede zwischen der Beschulungsart?

Dieser Beitrag untersucht Unterschiede in der Lebenszufriedenheit von SchülerInnen mit und ohne sonderpädagogischem Förderbedarf zwischen der Beschulungsart in Förder- oder (integrativen) Regelschulen. Datenbasis bildet das Nationale Bildungspanel (NEPS) mit 5.388 SchülerInnen der 7. Klassen (n=375 mit Förderbedarf, davon n=91 in integrativen Regelschulen; n=5.013 ohne Förderbedarf). In hierarchischen Regressionsanalysen wurde als abhängige Variable die Lebenszufriedenheit unter Kontrolle des Alters und Geschlechts sowie der Anzahl der Bücher im Haushalt und des Migrationshintergrundes analysiert. Die Ergebnisse zeigen, dass der Besuch einer integrativen Regelschule für SchülerInnen mit Förderbedarf mit einer niedrigeren Lebenszufriedenheit einhergeht im Vergleich zu jenen, die eine Förderschule besuchen. Die Ergebnisse heben damit die Bedeutung der Förderschule für die Lebenszufriedenheit von SchülerInnen mit Förderbedarf hervor.This study examines differences in life satisfaction among students with and without special educational needs (SEN) in regular or special educational needs schools. Data was obtained from the German National Educational Panel Study (NEPS), with n=5,388 seventh grade students (n=375 with SEN, among them n=91 students with SEN in regular schools; n=5,013 without SEN). Hierarchical linear modelling was applied, using life satisfaction as dependent variable, controlling for age and gender as well as migration background and number of books at home. Students with SEN, attending special educational needs schools, report higher life satisfaction compared to students with SEN in regular schools. The results highlight the importance of special educational needs schools for students’ life satisfaction

Role of contextual and compositional characteristics of schools for health inequalities in childhood and adolescence: protocol for a scoping review

Introduction: Childhood and adolescence are crucial life stages for health trajectories and the development of health inequalities in later life. The relevance of schools for health and well-being of children and adolescents has long been recognised, and there is some research regarding the association of contextual and compositional characteristics of schools and classes with health, health behaviour and well-being in this population. Little is known about the role of meso-level characteristics in relation to health inequalities. The aim of this scoping review is to retrieve and synthesise evidence about the mediating or moderating role of compositional or contextual characteristics of schools for the association between students' socioeconomic position and health in primary and secondary education. Methods and analysis We will conduct a systematic search of electronic databases in PubMed/Medline, Web of Science and Education Resources Information Center. Studies must meet the following inclusion criteria: (1) The population must be students attending primary or secondary schools in developed economies. (2) The outcomes must include at least one indicator for individual health, health behaviour or well-being. (3) The study must include at least one contextual or compositional characteristic of the school context and one individual determinant of socioeconomic position. (4) The study must also examine the mediating or moderating role of the contextual or compositional characteristic of the school context for the associations between socioeconomic position and health, health behaviour or well-being. (5) The study must be published since 1 January 2000 in English or German language. We will provide a narrative synthesis of findings. Ethics and dissemination We will not collect primary data and only include secondary data derived from previously published studies. Therefore, ethical approval is not required. We intend to publish our findings in an international peer-reviewed journal and to present them at national and international conferences