Latent Space Representations of Neural Algorithmic Reasoners

Neural Algorithmic Reasoning (NAR) is a research area focused on designing neural architectures that can reliably capture classical computation, usually by learning to execute algorithms. A typical approach is to rely on Graph Neural Network (GNN) architectures, which encode inputs in high-dimensional latent spaces that are repeatedly transformed during the execution of the algorithm. In this work we perform a detailed analysis of the structure of the latent space induced by the GNN when executing algorithms. We identify two possible failure modes: (i) loss of resolution, making it hard to distinguish similar values; (ii) inability to deal with values outside the range observed during training. We propose to solve the first issue by relying on a softmax aggregator, and propose to decay the latent space in order to deal with out-of-range values. We show that these changes lead to improvements on the majority of algorithms in the standard CLRS-30 benchmark when using the state-of-the-art Triplet-GMPNN processor. Our code is available at \href{https://github.com/mirjanic/nar-latent-spaces}{https://github.com/mirjanic/nar-latent-spaces}.Comment: 18 pages, 17 figures, accepted at KLR Workshop at ICML 202

Exploiting Cognitive Structure for Adaptive Learning

Adaptive learning, also known as adaptive teaching, relies on learning path recommendation, which sequentially recommends personalized learning items (e.g., lectures, exercises) to satisfy the unique needs of each learner. Although it is well known that modeling the cognitive structure including knowledge level of learners and knowledge structure (e.g., the prerequisite relations) of learning items is important for learning path recommendation, existing methods for adaptive learning often separately focus on either knowledge levels of learners or knowledge structure of learning items. To fully exploit the multifaceted cognitive structure for learning path recommendation, we propose a Cognitive Structure Enhanced framework for Adaptive Learning, named CSEAL. By viewing path recommendation as a Markov Decision Process and applying an actor-critic algorithm, CSEAL can sequentially identify the right learning items to different learners. Specifically, we first utilize a recurrent neural network to trace the evolving knowledge levels of learners at each learning step. Then, we design a navigation algorithm on the knowledge structure to ensure the logicality of learning paths, which reduces the search space in the decision process. Finally, the actor-critic algorithm is used to determine what to learn next and whose parameters are dynamically updated along the learning path. Extensive experiments on real-world data demonstrate the effectiveness and robustness of CSEAL.Comment: Accepted by KDD 2019 Research Track. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD'19

Dynamic Key-Value Memory Networks for Knowledge Tracing

Knowledge Tracing (KT) is a task of tracing evolving knowledge state of students with respect to one or more concepts as they engage in a sequence of learning activities. One important purpose of KT is to personalize the practice sequence to help students learn knowledge concepts efficiently. However, existing methods such as Bayesian Knowledge Tracing and Deep Knowledge Tracing either model knowledge state for each predefined concept separately or fail to pinpoint exactly which concepts a student is good at or unfamiliar with. To solve these problems, this work introduces a new model called Dynamic Key-Value Memory Networks (DKVMN) that can exploit the relationships between underlying concepts and directly output a student's mastery level of each concept. Unlike standard memory-augmented neural networks that facilitate a single memory matrix or two static memory matrices, our model has one static matrix called key, which stores the knowledge concepts and the other dynamic matrix called value, which stores and updates the mastery levels of corresponding concepts. Experiments show that our model consistently outperforms the state-of-the-art model in a range of KT datasets. Moreover, the DKVMN model can automatically discover underlying concepts of exercises typically performed by human annotations and depict the changing knowledge state of a student.Comment: To appear in 26th International Conference on World Wide Web (WWW), 201

Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks

We propose a computational framework to learn stylisation patterns from example drawings or writings, and then generate new trajectories that possess similar stylistic qualities. We particularly focus on the generation and stylisation of trajectories that are similar to the ones that can be seen in calligraphy and graffiti art. Our system is able to extract and learn dynamic and visual qualities from a small number of user defined examples which can be recorded with a digitiser device, such as a tablet, mouse or motion capture sensors. Our system is then able to transform new user drawn traces to be kinematically and stylistically similar to the training examples. We implement the system using a Recurrent Mixture Density Network (RMDN) combined with a representation given by the parameters of the Sigma Lognormal model, a physiologically plausible model of movement that has been shown to closely reproduce the velocity and trace of human handwriting gestures

Diminishing benefits of urban living for children and adolescents’ growth and development

Optimal growth and development in childhood and adolescence is crucial for lifelong health and well-being1–6. Here we used data from 2,325 population-based studies, with measurements of height and weight from 71 million participants, to report the height and body-mass index (BMI) of children and adolescents aged 5–19 years on the basis of rural and urban place of residence in 200 countries and territories from 1990 to 2020. In 1990, children and adolescents residing in cities were taller than their rural counterparts in all but a few high-income countries. By 2020, the urban height advantage became smaller in most countries, and in many high-income western countries it reversed into a small urban-based disadvantage. The exception was for boys in most countries in sub-Saharan Africa and in some countries in Oceania, south Asia and the region of central Asia, Middle East and north Africa. In these countries, successive cohorts of boys from rural places either did not gain height or possibly became shorter, and hence fell further behind their urban peers. The difference between the age-standardized mean BMI of children in urban and rural areas was <1.1 kg m–2 in the vast majority of countries. Within this small range, BMI increased slightly more in cities than in rural areas, except in south Asia, sub-Saharan Africa and some countries in central and eastern Europe. Our results show that in much of the world, the growth and developmental advantages of living in cities have diminished in the twenty-first century, whereas in much of sub-Saharan Africa they have amplified

Worldwide trends in body-mass index, underweight, overweight, and obesity from 1975 to 2016: a pooled analysis of 2416 population-based measurement studies in 128·9 million children, adolescents, and adults.

BACKGROUND: Underweight, overweight, and obesity in childhood and adolescence are associated with adverse health consequences throughout the life-course. Our aim was to estimate worldwide trends in mean body-mass index (BMI) and a comprehensive set of BMI categories that cover underweight to obesity in children and adolescents, and to compare trends with those of adults. METHODS: We pooled 2416 population-based studies with measurements of height and weight on 128·9 million participants aged 5 years and older, including 31·5 million aged 5-19 years. We used a Bayesian hierarchical model to estimate trends from 1975 to 2016 in 200 countries for mean BMI and for prevalence of BMI in the following categories for children and adolescents aged 5-19 years: more than 2 SD below the median of the WHO growth reference for children and adolescents (referred to as moderate and severe underweight hereafter), 2 SD to more than 1 SD below the median (mild underweight), 1 SD below the median to 1 SD above the median (healthy weight), more than 1 SD to 2 SD above the median (overweight but not obese), and more than 2 SD above the median (obesity). FINDINGS: Regional change in age-standardised mean BMI in girls from 1975 to 2016 ranged from virtually no change (-0·01 kg/m2 per decade; 95% credible interval -0·42 to 0·39, posterior probability [PP] of the observed decrease being a true decrease=0·5098) in eastern Europe to an increase of 1·00 kg/m2 per decade (0·69-1·35, PP>0·9999) in central Latin America and an increase of 0·95 kg/m2 per decade (0·64-1·25, PP>0·9999) in Polynesia and Micronesia. The range for boys was from a non-significant increase of 0·09 kg/m2 per decade (-0·33 to 0·49, PP=0·6926) in eastern Europe to an increase of 0·77 kg/m2 per decade (0·50-1·06, PP>0·9999) in Polynesia and Micronesia. Trends in mean BMI have recently flattened in northwestern Europe and the high-income English-speaking and Asia-Pacific regions for both sexes, southwestern Europe for boys, and central and Andean Latin America for girls. By contrast, the rise in BMI has accelerated in east and south Asia for both sexes, and southeast Asia for boys. Global age-standardised prevalence of obesity increased from 0·7% (0·4-1·2) in 1975 to 5·6% (4·8-6·5) in 2016 in girls, and from 0·9% (0·5-1·3) in 1975 to 7·8% (6·7-9·1) in 2016 in boys; the prevalence of moderate and severe underweight decreased from 9·2% (6·0-12·9) in 1975 to 8·4% (6·8-10·1) in 2016 in girls and from 14·8% (10·4-19·5) in 1975 to 12·4% (10·3-14·5) in 2016 in boys. Prevalence of moderate and severe underweight was highest in India, at 22·7% (16·7-29·6) among girls and 30·7% (23·5-38·0) among boys. Prevalence of obesity was more than 30% in girls in Nauru, the Cook Islands, and Palau; and boys in the Cook Islands, Nauru, Palau, Niue, and American Samoa in 2016. Prevalence of obesity was about 20% or more in several countries in Polynesia and Micronesia, the Middle East and north Africa, the Caribbean, and the USA. In 2016, 75 (44-117) million girls and 117 (70-178) million boys worldwide were moderately or severely underweight. In the same year, 50 (24-89) million girls and 74 (39-125) million boys worldwide were obese. INTERPRETATION: The rising trends in children's and adolescents' BMI have plateaued in many high-income countries, albeit at high levels, but have accelerated in parts of Asia, with trends no longer correlated with those of adults. FUNDING: Wellcome Trust, AstraZeneca Young Health Programme

Significant benefits of AIP testing and clinical screening in familial isolated and young-onset pituitary tumors

Context Germline mutations in the aryl hydrocarbon receptor-interacting protein (AIP) gene are responsible for a subset of familial isolated pituitary adenoma (FIPA) cases and sporadic pituitary neuroendocrine tumors (PitNETs). Objective To compare prospectively diagnosed AIP mutation-positive (AIPmut) PitNET patients with clinically presenting patients and to compare the clinical characteristics of AIPmut and AIPneg PitNET patients. Design 12-year prospective, observational study. Participants & Setting We studied probands and family members of FIPA kindreds and sporadic patients with disease onset ≤18 years or macroadenomas with onset ≤30 years (n = 1477). This was a collaborative study conducted at referral centers for pituitary diseases. Interventions & Outcome AIP testing and clinical screening for pituitary disease. Comparison of characteristics of prospectively diagnosed (n = 22) vs clinically presenting AIPmut PitNET patients (n = 145), and AIPmut (n = 167) vs AIPneg PitNET patients (n = 1310). Results Prospectively diagnosed AIPmut PitNET patients had smaller lesions with less suprasellar extension or cavernous sinus invasion and required fewer treatments with fewer operations and no radiotherapy compared with clinically presenting cases; there were fewer cases with active disease and hypopituitarism at last follow-up. When comparing AIPmut and AIPneg cases, AIPmut patients were more often males, younger, more often had GH excess, pituitary apoplexy, suprasellar extension, and more patients required multimodal therapy, including radiotherapy. AIPmut patients (n = 136) with GH excess were taller than AIPneg counterparts (n = 650). Conclusions Prospectively diagnosed AIPmut patients show better outcomes than clinically presenting cases, demonstrating the benefits of genetic and clinical screening. AIP-related pituitary disease has a wide spectrum ranging from aggressively growing lesions to stable or indolent disease course

Worldwide trends in underweight and obesity from 1990 to 2022: a pooled analysis of 3663 population-representative studies with 222 million children, adolescents, and adults

Background Underweight and obesity are associated with adverse health outcomes throughout the life course. We estimated the individual and combined prevalence of underweight or thinness and obesity, and their changes, from 1990 to 2022 for adults and school-aged children and adolescents in 200 countries and territories. Methods We used data from 3663 population-based studies with 222 million participants that measured height and weight in representative samples of the general population. We used a Bayesian hierarchical model to estimate trends in the prevalence of different BMI categories, separately for adults (age ≥20 years) and school-aged children and adolescents (age 5–19 years), from 1990 to 2022 for 200 countries and territories. For adults, we report the individual and combined prevalence of underweight (BMI <18·5 kg/m2) and obesity (BMI ≥30 kg/m2). For schoolaged children and adolescents, we report thinness (BMI <2 SD below the median of the WHO growth reference) and obesity (BMI >2 SD above the median). Findings From 1990 to 2022, the combined prevalence of underweight and obesity in adults decreased in 11 countries (6%) for women and 17 (9%) for men with a posterior probability of at least 0·80 that the observed changes were true decreases. The combined prevalence increased in 162 countries (81%) for women and 140 countries (70%) for men with a posterior probability of at least 0·80. In 2022, the combined prevalence of underweight and obesity was highest in island nations in the Caribbean and Polynesia and Micronesia, and countries in the Middle East and north Africa. Obesity prevalence was higher than underweight with posterior probability of at least 0·80 in 177 countries (89%) for women and 145 (73%) for men in 2022, whereas the converse was true in 16 countries (8%) for women, and 39 (20%) for men. From 1990 to 2022, the combined prevalence of thinness and obesity decreased among girls in five countries (3%) and among boys in 15 countries (8%) with a posterior probability of at least 0·80, and increased among girls in 140 countries (70%) and boys in 137 countries (69%) with a posterior probability of at least 0·80. The countries with highest combined prevalence of thinness and obesity in school-aged children and adolescents in 2022 were in Polynesia and Micronesia and the Caribbean for both sexes, and Chile and Qatar for boys. Combined prevalence was also high in some countries in south Asia, such as India and Pakistan, where thinness remained prevalent despite having declined. In 2022, obesity in school-aged children and adolescents was more prevalent than thinness with a posterior probability of at least 0·80 among girls in 133 countries (67%) and boys in 125 countries (63%), whereas the converse was true in 35 countries (18%) and 42 countries (21%), respectively. In almost all countries for both adults and school-aged children and adolescents, the increases in double burden were driven by increases in obesity, and decreases in double burden by declining underweight or thinness. Interpretation The combined burden of underweight and obesity has increased in most countries, driven by an increase in obesity, while underweight and thinness remain prevalent in south Asia and parts of Africa. A healthy nutrition transition that enhances access to nutritious foods is needed to address the remaining burden of underweight while curbing and reversing the increase in obesit

Height and body-mass index trajectories of school-aged children and adolescents from 1985 to 2019 in 200 countries and territories: a pooled analysis of 2181 population-based studies with 65 million participants

Summary Background Comparable global data on health and nutrition of school-aged children and adolescents are scarce. We aimed to estimate age trajectories and time trends in mean height and mean body-mass index (BMI), which measures weight gain beyond what is expected from height gain, for school-aged children and adolescents. Methods For this pooled analysis, we used a database of cardiometabolic risk factors collated by the Non-Communicable Disease Risk Factor Collaboration. We applied a Bayesian hierarchical model to estimate trends from 1985 to 2019 in mean height and mean BMI in 1-year age groups for ages 5–19 years. The model allowed for non-linear changes over time in mean height and mean BMI and for non-linear changes with age of children and adolescents, including periods of rapid growth during adolescence. Findings We pooled data from 2181 population-based studies, with measurements of height and weight in 65 million participants in 200 countries and territories. In 2019, we estimated a difference of 20 cm or higher in mean height of 19-year-old adolescents between countries with the tallest populations (the Netherlands, Montenegro, Estonia, and Bosnia and Herzegovina for boys; and the Netherlands, Montenegro, Denmark, and Iceland for girls) and those with the shortest populations (Timor-Leste, Laos, Solomon Islands, and Papua New Guinea for boys; and Guatemala, Bangladesh, Nepal, and Timor-Leste for girls). In the same year, the difference between the highest mean BMI (in Pacific island countries, Kuwait, Bahrain, The Bahamas, Chile, the USA, and New Zealand for both boys and girls and in South Africa for girls) and lowest mean BMI (in India, Bangladesh, Timor-Leste, Ethiopia, and Chad for boys and girls; and in Japan and Romania for girls) was approximately 9–10 kg/m2. In some countries, children aged 5 years started with healthier height or BMI than the global median and, in some cases, as healthy as the best performing countries, but they became progressively less healthy compared with their comparators as they grew older by not growing as tall (eg, boys in Austria and Barbados, and girls in Belgium and Puerto Rico) or gaining too much weight for their height (eg, girls and boys in Kuwait, Bahrain, Fiji, Jamaica, and Mexico; and girls in South Africa and New Zealand). In other countries, growing children overtook the height of their comparators (eg, Latvia, Czech Republic, Morocco, and Iran) or curbed their weight gain (eg, Italy, France, and Croatia) in late childhood and adolescence. When changes in both height and BMI were considered, girls in South Korea, Vietnam, Saudi Arabia, Turkey, and some central Asian countries (eg, Armenia and Azerbaijan), and boys in central and western Europe (eg, Portugal, Denmark, Poland, and Montenegro) had the healthiest changes in anthropometric status over the past 3·5 decades because, compared with children and adolescents in other countries, they had a much larger gain in height than they did in BMI. The unhealthiest changes—gaining too little height, too much weight for their height compared with children in other countries, or both—occurred in many countries in sub-Saharan Africa, New Zealand, and the USA for boys and girls; in Malaysia and some Pacific island nations for boys; and in Mexico for girls. Interpretation The height and BMI trajectories over age and time of school-aged children and adolescents are highly variable across countries, which indicates heterogeneous nutritional quality and lifelong health advantages and risks

Rising rural body-mass index is the main driver of the global obesity epidemic in adults

Body-mass index (BMI) has increased steadily in most countries in parallel with a rise in the proportion of the population who live in cities(.)(1,2) This has led to a widely reported view that urbanization is one of the most important drivers of the global rise in obesity(3-6). Here we use 2,009 population-based studies, with measurements of height and weight in more than 112 million adults, to report national, regional and global trends in mean BMI segregated by place of residence (a rural or urban area) from 1985 to 2017. We show that, contrary to the dominant paradigm, more than 55% of the global rise in mean BMI from 1985 to 2017-and more than 80% in some low- and middle-income regions-was due to increases in BMI in rural areas. This large contribution stems from the fact that, with the exception of women in sub-Saharan Africa, BMI is increasing at the same rate or faster in rural areas than in cities in low- and middle-income regions. These trends have in turn resulted in a closing-and in some countries reversal-of the gap in BMI between urban and rural areas in low- and middle-income countries, especially for women. In high-income and industrialized countries, we noted a persistently higher rural BMI, especially for women. There is an urgent need for an integrated approach to rural nutrition that enhances financial and physical access to healthy foods, to avoid replacing the rural undernutrition disadvantage in poor countries with a more general malnutrition disadvantage that entails excessive consumption of low-quality calories.Peer reviewe