Learning Complicated Manipulation Skills via Deterministic Policy with Limited Demonstrations

Combined with demonstrations, deep reinforcement learning can efficiently develop policies for manipulators. However, it takes time to collect sufficient high-quality demonstrations in practice. And human demonstrations may be unsuitable for robots. The non-Markovian process and over-reliance on demonstrations are further challenges. For example, we found that RL agents are sensitive to demonstration quality in manipulation tasks and struggle to adapt to demonstrations directly from humans. Thus it is challenging to leverage low-quality and insufficient demonstrations to assist reinforcement learning in training better policies, and sometimes, limited demonstrations even lead to worse performance. We propose a new algorithm named TD3fG (TD3 learning from a generator) to solve these problems. It forms a smooth transition from learning from experts to learning from experience. This innovation can help agents extract prior knowledge while reducing the detrimental effects of the demonstrations. Our algorithm performs well in Adroit manipulator and MuJoCo tasks with limited demonstrations

A Versatile Agent for Fast Learning from Human Instructors

In recent years, a myriad of superlative works on intelligent robotics policies have been done, thanks to advances in machine learning. However, inefficiency and lack of transfer ability hindered algorithms from pragmatic applications, especially in human-robot collaboration, when few-shot fast learning and high flexibility become a wherewithal. To surmount this obstacle, we refer to a "Policy Pool", containing pre-trained skills that can be easily accessed and reused. An agent is employed to govern the "Policy Pool" by unfolding requisite skills in a flexible sequence, contingent on task specific predilection. This predilection can be automatically interpreted from one or few human expert demonstrations. Under this hierarchical setting, our algorithm is able to pick up a sparse-reward, multi-stage knack with only one demonstration in a Mini-Grid environment, showing the potential for instantly mastering complex robotics skills from human instructors. Additionally, the innate quality of our algorithm also allows for lifelong learning, making it a versatile agent

Organic matter provenance and depositional environment of marine-to-continental mudstones and coals in eastern Ordos Basin, China—Evidence from molecular geochemistry and petrology

Cyclothems, composed of interbedded mudstone, coal and sandstone layers, make up the Taiyuan and Shanxi Formations in the Late Carboniferous to Early Permian in North China under a marine-to-continental depositional environment. The cyclothems act as important fossil energy hosts, such as coalbeds, hydrocarbon source rocks and unconventional natural gas reservoirs. Organic geochemistry and petrology of mudstones and coals in the Taiyuan and Shanxi Formations in the eastern Ordos Basin were studied to reveal the organic matter sources and paleoenvironments. Total organic carbon (TOC) contents vary from 1.1 wt% (mudstone) to 72.6 wt% (coal). The samples are mainly within the oil window, with the Tmax values ranging from 433 to 469 °C. Organic petrology and source biomarkers indicate that the mudstones were sourced from a mixed organic matter input, and terrigenous organic matter predominates over aquatic organic matter. The coals are mostly sourced by terrigenous organic matter inputs. High concentrations of hopanes argue for a strong bacterial input. Some m/z 217 mass chromatograms have peaks at the hopanes' retention times as a result of high hopane to sterane ratios. These hopane-derived peaks do not interfere the identification of the steranes because the hopanes and the steranes have different retention times. Maturity-dependent biomarkers demonstrate that the samples have been thermally mature, which agree with the Tmax values. Anomalously low C29 20S/(20S + 20R) and C29 ββ/(ββ + αα) sterane ratios are present in all the samples, and are interpreted as due to the terrigenous organic matter input or the coal-related depositional environment. In addition, biomarkers and iron sulfide morphology indicate that the organic matter of the mudstones deposited in a proximal setting with shallow, brackish/fresh water bodies. With consideration of preservation of organic matter, the redox conditions are dysoxic. Redox oscillations resulted in the records of oxic conditions in some samples. Finally, the coals and the mudstones mainly generate gas and have poor oil generative potential

Proteomic Studies on the Mechanism of Myostatin Regulating Cattle Skeletal Muscle Development

Myostatin (MSTN) is an important negative regulator of muscle growth and development. In this study, we performed comparatively the proteomics analyses of gluteus tissues from MSTN+/− Mongolian cattle (MG.MSTN+/−) and wild type Mongolian cattle (MG.WT) using a shotgun-based tandem mass tag (TMT) 6-plex labeling method to investigate the regulation mechanism of MSTN on the growth and development of bovine skeletal muscle. A total of 1,950 proteins were identified in MG.MSTN+/− and MG.WT. Compared with MG.WT cattle, a total of 320 differentially expressed proteins were identified in MG.MSTN cattle, including 245 up-regulated differentially expressed proteins and 75 down-regulated differentially expressed proteins. Bioinformatics analysis showed that knockdown of the MSTN gene increased the expression of extracellular matrix and ribosome-related proteins, induced activation of focal adhesion, PI3K-AKT, and Ribosomal pathways. The results of proteomic analysis were verified by muscle tissue Western blot test and in vitro MSTN gene knockdown test, and it was found that knockdown MSTN gene expression could promote the proliferation and myogenic differentiation of bovine skeletal muscle satellite cells (BSMSCs). At the same time, Co-Immunoprecipitation (CO-IP) assay showed that MSTN gene interacted with extracellular matrix related protein type I collagen α 1 (COL1A1), and knocking down the expression of COL1A1 could inhibit the activity of adhesion, PI3K-AKT and ribosome pathway, thus inhibit BSMSCs proliferation. These results suggest that the MSTN gene regulates focal adhesion, PI3K-AKT, and Ribosomal pathway through the COL1A1 gene. In general, this study provides new insights into the regulatory mechanism of MSTN involved in muscle growth and development

Strategies for enhancing electrochemical CO 2 reduction to multi-carbon fuels on copper

Productively harnessing CO2 as a reactant is of practical interest due to addressing the dual pressures of resource sustainability and environmental sustainability. Electrochemical CO2 reduction (ECR) offers a promising approach for driving the chemical transformation of CO2 by exploiting green renewably generated electricity at (near) room temperature and ambient pressure, facilitating a sustainable, low-carbon footprint future. In this work, we provide a comprehensive and timely review of the various avenues that have been developed to date to modulate product selectivity, stability, and efficiency toward C2+ using Cu-based electrocatalysts. We discuss how the electrocatalyst structure can be effectively designed in order to boost performance. Special attention is paid to some of the critical intermediate species that shed light on CO2 reduction paths. We will also discuss the application of in situ and operando spectroscopy, along with computational techniques, that help to improve our fundamental understanding of ECR. Finally, development opportunities and challenge in the conversion of CO2 into multi-carbon fuels by Cu-based electrocatalysts are presented

Perceived Roles and Barriers in Delivering Community-Based Care: A Qualitative Study of Health and Social Care Professionals

Introduction: As healthcare systems increasingly embrace population health management, the integration of health and social care to improve the health and well-being of individuals is crucial. Thus, we conducted a qualitative study in Singapore to understand health and social care professionals’ (HCPs and SCPs) perception of the roles they played in delivering community-based care. Methods: A descriptive phenomenological research design was adopted. HCPs and SCPs (n = 53) providing services in community settings were recruited purposefully and interviewed through eleven focus group discussions. Each session was recorded and transcribed. Thematic analysis was applied. Results: Our results revealed eight themes in three main categories describing the roles played by HCPs and SCPs, including: (1) delivering needs-based care in community settings; (2) activating and empowering clients in health care, and (3) fostering community-based sustainable support networks. Six barriers encountered while performing these roles were also identified. Discussion and Conclusion: Our results highlight that the roles of HCPs and SCPs go beyond the provision of direct medical and social care. They were involved in activating and empowering clients to take care of their health, and importantly, fostering community-based sustainable support networks to better empower individuals in coping with health challenges. The identified barriers shed light on areas for potential improvements for integrated community care

Measurement of t(t)over-bar normalised multi-differential cross sections in pp collisions at root s=13 TeV, and simultaneous determination of the strong coupling strength, top quark pole mass, and parton distribution functions

Peer reviewe