Bayesian Policy Reuse

A long-lived autonomous agent should be able to respond online to novel instances of tasks from a familiar domain. Acting online requires 'fast' responses, in terms of rapid convergence, especially when the task instance has a short duration, such as in applications involving interactions with humans. These requirements can be problematic for many established methods for learning to act. In domains where the agent knows that the task instance is drawn from a family of related tasks, albeit without access to the label of any given instance, it can choose to act through a process of policy reuse from a library, rather than policy learning from scratch. In policy reuse, the agent has prior knowledge of the class of tasks in the form of a library of policies that were learnt from sample task instances during an offline training phase. We formalise the problem of policy reuse, and present an algorithm for efficiently responding to a novel task instance by reusing a policy from the library of existing policies, where the choice is based on observed 'signals' which correlate to policy performance. We achieve this by posing the problem as a Bayesian choice problem with a corresponding notion of an optimal response, but the computation of that response is in many cases intractable. Therefore, to reduce the computation cost of the posterior, we follow a Bayesian optimisation approach and define a set of policy selection functions, which balance exploration in the policy library against exploitation of previously tried policies, together with a model of expected performance of the policy library on their corresponding task instances. We validate our method in several simulated domains of interactive, short-duration episodic tasks, showing rapid convergence in unknown task variations.Comment: 32 pages, submitted to the Machine Learning Journa

Higher socioeconomic status is related to healthier levels of fatness and fitness already at 3 to 5 years of age: The PREFIT project: Relation between socioeconomic status, fatness and fitness in preschoolers

This study aimed to analyse the association between socioeconomic status (SES) and fatness and fitness in preschoolers. 2, 638 preschoolers (3–5 years old; 47.2% girls) participated. SES was estimated from the parental educational and occupational levels, and the marital status. Fatness was assessed by body mass index (BMI), waist circumference (WC), and waist-to-height ratio (WHtR). Physical fitness components were assessed using the PREFIT battery. Preschoolers whose parents had higher educational levels had lower fatness (P < 0.05). BMI significantly differed across occupational levels of each parent (P < 0.05) and WHtR across paternal levels (P = 0.004). Musculoskeletal fitness was different across any SES factor (P < 0.05), except handgrip across paternal occupational levels (P = 0.05). Preschoolers with high paternal occupation had higher speed/agility (P = 0.005), and those with high or low maternal education had higher VO2max (P = 0.046). Odds of being obese and having low musculoskeletal fitness was lower as SES was higher (P < 0.05). Those with married parents had higher cardiorespiratory fitness than single-parent ones (P = 0.010). School-based interventions should be aware of that children with low SES are at a higher risk of obesity and low fitness already in the first years of life

RICORS2040 : The need for collaborative research in chronic kidney disease

Chronic kidney disease (CKD) is a silent and poorly known killer. The current concept of CKD is relatively young and uptake by the public, physicians and health authorities is not widespread. Physicians still confuse CKD with chronic kidney insufficiency or failure. For the wider public and health authorities, CKD evokes kidney replacement therapy (KRT). In Spain, the prevalence of KRT is 0.13%. Thus health authorities may consider CKD a non-issue: very few persons eventually need KRT and, for those in whom kidneys fail, the problem is 'solved' by dialysis or kidney transplantation. However, KRT is the tip of the iceberg in the burden of CKD. The main burden of CKD is accelerated ageing and premature death. The cut-off points for kidney function and kidney damage indexes that define CKD also mark an increased risk for all-cause premature death. CKD is the most prevalent risk factor for lethal coronavirus disease 2019 (COVID-19) and the factor that most increases the risk of death in COVID-19, after old age. Men and women undergoing KRT still have an annual mortality that is 10- to 100-fold higher than similar-age peers, and life expectancy is shortened by ~40 years for young persons on dialysis and by 15 years for young persons with a functioning kidney graft. CKD is expected to become the fifth greatest global cause of death by 2040 and the second greatest cause of death in Spain before the end of the century, a time when one in four Spaniards will have CKD. However, by 2022, CKD will become the only top-15 global predicted cause of death that is not supported by a dedicated well-funded Centres for Biomedical Research (CIBER) network structure in Spain. Realizing the underestimation of the CKD burden of disease by health authorities, the Decade of the Kidney initiative for 2020-2030 was launched by the American Association of Kidney Patients and the European Kidney Health Alliance. Leading Spanish kidney researchers grouped in the kidney collaborative research network Red de Investigación Renal have now applied for the Redes de Investigación Cooperativa Orientadas a Resultados en Salud (RICORS) call for collaborative research in Spain with the support of the Spanish Society of Nephrology, Federación Nacional de Asociaciones para la Lucha Contra las Enfermedades del Riñón and ONT: RICORS2040 aims to prevent the dire predictions for the global 2040 burden of CKD from becoming true