Learning Reward for Physical Skills using Large Language Model

Learning reward functions for physical skills are challenging due to the vast spectrum of skills, the high-dimensionality of state and action space, and nuanced sensory feedback. The complexity of these tasks makes acquiring expert demonstration data both costly and time-consuming. Large Language Models (LLMs) contain valuable task-related knowledge that can aid in learning these reward functions. However, the direct application of LLMs for proposing reward functions has its limitations such as numerical instability and inability to incorporate the environment feedback. We aim to extract task knowledge from LLMs using environment feedback to create efficient reward functions for physical skills. Our approach consists of two components. We first use the LLM to propose features and parameterization of the reward function. Next, we update the parameters of this proposed reward function through an iterative self-alignment process. In particular, this process minimizes the ranking inconsistency between the LLM and our learned reward functions based on the new observations. We validated our method by testing it on three simulated physical skill learning tasks, demonstrating effective support for our design choices.Comment: CoRL 2023, LangRob worksho

An investigation on China’s NGOs in urban heritage preservation

As for China’s urban heritage preservation, besides the top-down actors, some non-governmental organizations have emerged and started to play increasingly noticeable parts. The paper accordingly explores China's increasingly pluralistic situation in urban heritage preservation, and reveals the roles of these non-governmental players. The paper selects Tianjin Memory as a specific case, and elucidates its developments and transformations from 2006 to the present via documental research, in-depth interviews and internet big data analyses. Informed by internal ecological relationship analyses, the paper divides its development process into four main stages: start, rapid development, differentiation and reorganization. Each stage is examined from six factors, i.e. human resources, structure and management, finance, social resources, products and achievements, based on NGOs’ influence evaluation in sociology. The research further identifies the main limitations and challenges for Tianjin Memory. As a part of the discussions about China’s current pluralistic urbanism, this paper brings forwards some suggestions for a healthy and sustainable future of Tianjin Memory and other similar NGOs in China

Sn(II)-containing phosphates as optoelectronic materials

We theoretically investigate Sn(II) phosphates as optoelectronic materials using first principles calculations. We focus on known prototype materials Sn$_n$P$_2$O$_{5+n}$ (n=2, 3, 4, 5) and a previously unreported compound, SnP$_2$O$_6$ (n=1), which we find using global optimization structure prediction. The electronic structure calculations indicate that these compounds all have large band gaps above 3.2 eV, meaning their transparency to visible light. Several of these compounds show relatively low hole effective masses ($\sim$2-3 m$_0$), comparable the electron masses. This suggests potential bipolar conductivity depending on doping. The dispersive valence band-edges underlying the low hole masses, originate from the anti-bonding hybridization between the Sn 5s orbitals and the phosphate groups. Analysis of structure-property relationships for the metastable structures generated during structure search shows considerable variation in combinations of band gap and carrier effective masses, implying chemical tunability of these properties. The unusual combinations of relatively high band gap, low carrier masses and high chemical stability suggests possible optoelectronic applications of these Sn(II) phosphates, including p-type transparent conductors. Related to this, calculations for doped material indicate low visible light absorption, combined with high plasma frequencies.Comment: 10 pages, 10 figures, Supplementary informatio

Gender-diverse practitioners in early years education and care (EYEC): a cross-cultural study of Scotland, Hong Kong, and Mainland China

This paper discusses whether practitioners’ gender subjectivities influence pedagogies and practices in early years education and care (EYEC) settings and whether an increase of men’s participation can improve gender diversity in EYEC. It draws on poststructuralist theories, understanding gender as the product/outcome of the social formation of subjects and the process of subjectification. This is illustrated through accounts for how individual practitioners from Scotland, Hong Kong, and Mainland China discursively construct their gender subjectivities, in accordance with the respective cultural discourses that shape work with young children in EYEC in the three contexts. Thirty-four practitioners from 17 EYEC settings (1 male and 1 female practitioner from each setting) in the cities of Edinburgh, Hong Kong, and Tianjin were interviewed. The study finds that participant practitioners’ constructions of gender subjectivities vary within and across contexts, and gender-binary discourses are to different extents prevalent in all three contexts. This paper argues for a cross-cultural approach to gender-sensitive teacher training, to interrogate popular discourses that advocate for men to fulfil complementary roles in EYEC to women and to challenge gender binary thinking that persists in EYEC and beyond

A cross-cultural analysis of gender and practitioner-child interactions in Early Childhood Education and Care (ECEC) settings in Scotland, Hong Kong, and Mainland China

This study is conducted in the global contexts of policy calls for more men to work in early childhood education and care (ECEC) and of concerns over the assumed ‘feminisation’ of ECEC. The overarching aim is to critically interrogate whether men should be encouraged to work in the ECEC workforce in greater numbers in both the UK and China (Mainland China and Hong Kong). Framed by the poststructuralist theoretical framework of gender, this research aims to address four research questions: 1. How do practitioners posit themselves as women/men working with young children in ECEC? 2. How do children view their practitioners’ gender in relation to their daily interactions? 3. What is the nature of interactions between practitioners and children in ECEC settings? How far and to what extent can these interactions be seen to be gendered, and in what ways? 4. How far and to what extent can culturally-specific gender discourses be seen to have an impact on practitioner-child interactions in Scotland, Hong Kong and Mainland China, and in what ways? Qualitative, multiple-method and cross-cultural approaches were adopted. Research methods employed include observations in ECEC settings, interviews with ECEC practitioners, and pictorial activities with children. 17 ECEC settings were recruited from the cities of Edinburgh, Hong Kong, and Tianjin, and 34 ECEC practitioners and 280 children aged 3-6 years old participated in the research. The findings suggest that practitioners’ and children’s constructions of gender subjectivities can be diverse and dynamic processes through which individuals embody and ‘perform’ their gender with references to a variety of cultural and gender discourses that situate them. This study therefore argues that ECEC pedagogies and practices need to enable practitioners and children to interrogate dominant gender discourses and to become gender-sensitive and –flexible performers, in order to achieve gender equality, diversity and inclusion in ECEC. Current political drives in the UK, China and elsewhere to recruit more men to work in ECEC and to achieve a gender-balanced ECEC workforce need to reconsider their theoretical underpinnings and to make sure that such policies will not reinforce binary, hegemonic gender structures. A gender-diverse and –flexible approach to gender and ECEC is preferable for equitable and inclusive ECEC