Layered direct policy search for learning hierarchical skills

Solutions to real world robotic tasks often require complex behaviors in high dimensional continuous state and action spaces. Reinforcement Learning (RL) is aimed at learning such behaviors but often fails for lack of scalability. To address this issue, Hierarchical RL (HRL) algorithms leverage hierarchical policies to exploit the structure of a task. However, many HRL algorithms rely on task specific knowledge such as a set of predefined sub-policies or sub-goals. In this paper we propose a new HRL algorithm based on information theoretic principles to autonomously uncover a diverse set of sub-policies and their activation policies. Moreover, the learning process mirrors the policys structure and is thus also hierarchical, consisting of a set of independent optimization problems. The hierarchical structure of the learning process allows us to control the learning rate of the sub-policies and the gating individually and add specific information theoretic constraints to each layer to ensure the diversification of the subpolicies. We evaluate our algorithm on two high dimensional continuous tasks and experimentally demonstrate its ability to autonomously discover a rich set of sub-policies

Wfs1 Is Expressed In Dopaminoceptive Regions Of The Amniote Brain And Modulates Levels Of D1-Like Receptors

During amniote evolution, the construction of the forebrain has diverged across different lineages, and accompanying the structural changes, functional diversification of the homologous brain regions has occurred. This can be assessed by studying the expression patterns of marker genes that are relevant in particular functional circuits. In all vertebrates, the dopaminergic system is responsible for the behavioral responses to environmental stimuli. Here we show that the brain regions that receive dopaminergic input through dopamine receptor D1 are relatively conserved, but with some important variations between three evolutionarily distant vertebrate lines–house mouse (Mus musculus), domestic chick (Gallus gallus domesticus) / common quail (Coturnix coturnix) and red-eared slider turtle (Trachemys scripta). Moreover, we find that in almost all instances, those brain regions expressing D1-like dopamine receptor genes also express Wfs1. Wfs1 has been studied primarily in the pancreas, where it regulates the endoplasmic reticulum (ER) stress response, cellular Ca2+ homeostasis, and insulin production and secretion. Using radioligand binding assays in wild type and Wfs1-/- mouse brains, we show that the number of binding sites of D1-like dopamine receptors is increased in the hippocampus of the mutant mice. We propose that the functional link between Wfs1 and D1-like dopamine receptors is evolutionarily conserved and plays an important role in adjusting behavioral reactions to environmental stimuli

Applying Market Shaping Approaches to Increase Access to Assistive Technology: Summary of the Wheelchair Product Narrative

To accelerate access to assistive technology (AT), we need to leverage the capabilities and resources of the public, private, and non-profit sectors to harness innovation and break down barriers to access. Market shaping interventions can play a role in enhancing market efficiencies, coordinating and incentivizing the number of stakeholders involved in demand and supply-side activities. Across health sectors, market shaping has demonstrated its potential to enhance national governments’ or donors’ value-for-money, diversify the supply base, and increase reliability – ultimately increasing product and service delivery access for end users. These market shaping successes in other health areas have led practitioners to hypothesize that market shaping could also be applied to assistive technology markets. ATscale, the Global Partnership for AT, aims to mobilise global stakeholders to shape markets in line with a unified strategy. To inform this strategy, it is critical to identify specific interventions required to shape markets and overcome barriers. The first product undergoing analysis by ATscale is wheelchairs. The market for appropriate wheelchairs in low-and middle income countries (LMICs) is highly fragmented and characterized by limited government interest, investment, and a low willingness-to-pay. Moreover, the market is dominated by cheaper, low quality wheelchairs which fail to meet the needs of end-users. Non-profit organizations have attempted to fill the need for context-appropriate wheelchairs, but market uptake is limited. These initial findings led ATscale to believe that market shaping could support increased access to appropriate wheelchairs. This paper outlines what market shaping is, and how it can be applied to assistive technology at large -- using the aforementioned wheelchair product narrative as an illustrative case study and presents the proposed market shaping strategy for wheelchairs. ATscale will develop a framework to evaluate short-term interventions identified to achieve a healthy market and increase access. This paper provides an opportunity to obtain feedback from interested stakeholders on the market shaping strategy for wheelchairs, as well as the product narrative process to be undertaken for other priority AT

Troubling meanings of family and competing moral imperatives in the family lives of young people with a parent who is at the end of life

This article draws on a narrative study of young people with a parent who is at the end of life to examine how family lives are troubled by life-limiting parental illness. Young people struggled to reconcile the physical and emotional absence of family members with meanings of ‘family’; the extent to which young people could rely on family to ‘be there’ in these troubling circumstances was of practical, emotional and moral significance. Our discussion is situated in the context of an English end of life care policy predicated on the ideal of a good death as one that takes place at home accompanied by family members. We explore how the shift away from family as a site for nurturing children towards family as a space to care for the dying is experienced by young people, and consider how these competing moral imperatives are negotiated through relational practices of care

Differences in Lower Extremity Kinematics Between High School Cross-Country and Young Adult Recreational Runners

# Background While previous research has assessed running kinematics for age-related differences that could increase the risk of a running-related injury, none of these studies have included high school aged runners or assessed running kinematics using 2-dimensional video analysis. # Purpose The purpose of this study was to compare sagittal plane kinematics during treadmill running in high school cross-country and young adult recreational runners using 2-dimensional motion analysis techniques. # Methods Twenty-five high school cross-country runners (13 women, 12 men) and 25 young adult recreational runners (12 women, 13 men) consented to participate in this study. Reflective markers were placed on each lower extremity over multiple anatomical landmarks. After a five-minute acclimation period in which the participants ran on a treadmill at their preferred running speed, video data were recorded at 240 frames per second for all participants while they continued to run on the treadmill. # Results There were no significant differences between left and right extremities. The young adult recreational runners exhibited significantly greater vertical excursion of the center of mass (*t* = 4.64, p = .0001) compared to the high school runners. There was no significant difference between the two age groups regarding the six other sagittal plane variables. # Conclusions The young adult recreational runners demonstrated an increased center-of-mass vertical excursion in comparison to high school cross-country runners. In addition, the results obtained in this study for kinematic variables using 2-dimensional motion analysis were similar to previously reported studies using 3-dimensional motion analysis, demonstrating that 2-dimensional motion analysis could be used for analyzing sagittal plane running kinematics in clinical settings. # Level of Evidence 4, Controlled laboratory stud

Effect of a farnesyl transferase inhibitor (R115777) on ductal carcinoma in situ of the breast in a human xenograft model and on breast and ovarian cancer cell growth in vitro and in vivo

INTRODUCTION: The ras pathway is essential for cell growth and proliferation. The effects of R115777, a farnesyl transferase inhibitor, were investigated in cancer cell lines expressing varying levels of growth factor receptors and with differing ras status. Effects on tumour xenografts and human ductal carcinoma in situ (DCIS) of the breast in a xenograft mouse model were also tested. METHOD: In vitro, the concentrations required to reduce cell numbers by 50% (50% inhibitory concentration) were established (MDA-MB231, MCF-7, MCF-7/HER2-18, BT-474, SK-BR3 and SKOV3). Human DCIS was implanted in nude mice or, in separate experiments, cultured cells were injected (MDA-MB231, MCF-7/HER2-18, SKOV3) and allowed to form tumours. Proliferation and apoptosis were determined by immunohistochemistry in xenografts and cell tumours. RESULTS: The 50% inhibitory concentrations varied a hundred-fold, from 39 nmol/l (± 26 nmol/l) for SKBR3 to 5.9 μmol/l(± 0.8 μmol/l) for MDA-MB231. In MCF-7/HER2-18 and SKOV3 cells the levels of tumour growth inhibition were approximately 85% and 40%, respectively. There was a significant decrease in the cell turnover index (CTI; proliferation/apoptosis). In MDA-MB 231 with activated k-ras no inhibition was observed. In treated DCIS xenografts proliferation decreased and apoptosis increased. The CTI ratio between the start and 1 and 2 weeks of treatment were 1.99 and 1.50, respectively, for controls and 0.85 (P = 0.005) and 0.75 (P = 0.08) for treated xenografts. CONCLUSION: Treatment with the farnesyl transferase inhibitor reduced cell growth in vitro and cell tumour growth in vivo. In DCIS treatment resulted in a reduced CTI. R115777 is a promising treatment for breast cancer but the relation between effect and growth factor receptor and ras status has to be established

Implementing a quality improvement programme in palliative care in care homes: a qualitative study

<p>Abstract</p> <p>Background</p> <p>An increasing number of older people reach the end of life in care homes. The aim of this study is to explore the perceived benefits of, and barriers to, implementation of the Gold Standards Framework for Care Homes (GSFCH), a quality improvement programme in palliative care.</p> <p>Methods</p> <p>Nine care homes involved in the GSFCH took part. We conducted semi-structured interviews with nine care home managers, eight nurses, nine care assistants, eleven residents and seven of their family members. We used the Framework approach to qualitative analysis. The analysis was deductive based on the key tasks of the GSFCH, the 7Cs: communication, coordination, control of symptoms, continuity, continued learning, carer support, and care of the dying. This enabled us to consider benefits of, and barriers to, individual components of the programme, as well as of the programme as a whole.</p> <p>Results</p> <p>Perceived benefits of the GSFCH included: improved symptom control and team communication; finding helpful external support and expertise; increasing staff confidence; fostering residents' choice; and boosting the reputation of the home. Perceived barriers included: increased paperwork; lack of knowledge and understanding of end of life care; costs; and gaining the cooperation of GPs. Many of the tools and tasks in the GSFCH focus on improving communication. Participants described effective communication within the homes, and with external providers such as general practitioners and specialists in palliative care. However, many had experienced problems with general practitioners. Although staff described the benefits of supportive care registers, coding predicted stage of illness and advance care planning, which included improved communication, some felt the need for more experience of using these, and there were concerns about discussing death.</p> <p>Conclusions</p> <p>Most of the barriers described by participants are relevant to other interventions to improve end of life care in care homes. There is a need to investigate the impact of quality improvement programmes in care homes, such as the GSFCH, on a wider range of outcomes for residents and their families, and to monitor the sustainability of any resulting improvements. It is also important to explore the impact of the different components of these complex interventions.</p