Robustness of Utilizing Feedback in Embodied Visual Navigation

This paper presents a framework for training an agent to actively request help in object-goal navigation tasks, with feedback indicating the location of the target object in its field of view. To make the agent more robust in scenarios where a teacher may not always be available, the proposed training curriculum includes a mix of episodes with and without feedback. The results show that this approach improves the agent's performance, even in the absence of feedback.Comment: Accepted at the ICRA Workshop for Communicating Robot Learning across Human-Robot Interactio

A Survey of Embodied AI: From Simulators to Research Tasks

There has been an emerging paradigm shift from the era of "internet AI" to "embodied AI", where AI algorithms and agents no longer learn from datasets of images, videos or text curated primarily from the internet. Instead, they learn through interactions with their environments from an egocentric perception similar to humans. Consequently, there has been substantial growth in the demand for embodied AI simulators to support various embodied AI research tasks. This growing interest in embodied AI is beneficial to the greater pursuit of Artificial General Intelligence (AGI), but there has not been a contemporary and comprehensive survey of this field. This paper aims to provide an encyclopedic survey for the field of embodied AI, from its simulators to its research. By evaluating nine current embodied AI simulators with our proposed seven features, this paper aims to understand the simulators in their provision for use in embodied AI research and their limitations. Lastly, this paper surveys the three main research tasks in embodied AI -- visual exploration, visual navigation and embodied question answering (QA), covering the state-of-the-art approaches, evaluation metrics and datasets. Finally, with the new insights revealed through surveying the field, the paper will provide suggestions for simulator-for-task selections and recommendations for the future directions of the field.Comment: Under Review for IEEE TETC

Subgraph Networks Based Contrastive Learning

Graph contrastive learning (GCL), as a self-supervised learning method, can solve the problem of annotated data scarcity. It mines explicit features in unannotated graphs to generate favorable graph representations for downstream tasks. Most existing GCL methods focus on the design of graph augmentation strategies and mutual information estimation operations. Graph augmentation produces augmented views by graph perturbations. These views preserve a locally similar structure and exploit explicit features. However, these methods have not considered the interaction existing in subgraphs. To explore the impact of substructure interactions on graph representations, we propose a novel framework called subgraph network-based contrastive learning (SGNCL). SGNCL applies a subgraph network generation strategy to produce augmented views. This strategy converts the original graph into an Edge-to-Node mapping network with both topological and attribute features. The single-shot augmented view is a first-order subgraph network that mines the interaction between nodes, node-edge, and edges. In addition, we also investigate the impact of the second-order subgraph augmentation on mining graph structure interactions, and further, propose a contrastive objective that fuses the first-order and second-order subgraph information. We compare SGNCL with classical and state-of-the-art graph contrastive learning methods on multiple benchmark datasets of different domains. Extensive experiments show that SGNCL achieves competitive or better performance (top three) on all datasets in unsupervised learning settings. Furthermore, SGNCL achieves the best average gain of 6.9\% in transfer learning compared to the best method. Finally, experiments also demonstrate that mining substructure interactions have positive implications for graph contrastive learning.Comment: 12 pages, 6 figure

Bias-reduced Multi-step Hindsight Experience Replay for Efficient Multi-goal Reinforcement Learning

Multi-goal reinforcement learning is widely applied in planning and robot manipulation. Two main challenges in multi-goal reinforcement learning are sparse rewards and sample inefficiency. Hindsight Experience Replay (HER) aims to tackle the two challenges via goal relabeling. However, HER-related works still need millions of samples and a huge computation. In this paper, we propose Multi-step Hindsight Experience Replay (MHER), incorporating multi-step relabeled returns based on $n$-step relabeling to improve sample efficiency. Despite the advantages of $n$-step relabeling, we theoretically and experimentally prove the off-policy $n$-step bias introduced by $n$-step relabeling may lead to poor performance in many environments. To address the above issue, two bias-reduced MHER algorithms, MHER($\lambda$) and Model-based MHER (MMHER) are presented. MHER($\lambda$) exploits the $\lambda$ return while MMHER benefits from model-based value expansions. Experimental results on numerous multi-goal robotic tasks show that our solutions can successfully alleviate off-policy $n$-step bias and achieve significantly higher sample efficiency than HER and Curriculum-guided HER with little additional computation beyond HER.Comment: 20pages, 8 figure

Research on Friction Compensation Control for Electric Power Steering System

A novel friction compensation control method is proposed to compensate both the dynamic and static friction torque of steering system. The change of EPS assist torque under fixed amplitude friction compensation torque can cause the diver’s steering feeling fuzzy. That is due to the fact that the friction torque felt by driver varies with EPS assist gain. Therefore, a further modified friction compensation control method is proposed based on EPS assist gain to make the driver have similar friction feeling. Finally, computer simulation and vehicle test are performed to verify the effectiveness of adaptation method in the proposed controller. Test results indicate that the proposed controller improved the driver’s steering performance

Vitamin C supramolecular hydrogel for enhanced cancer immunotherapy

Vitamin C (VitC) has shown great promise to promote cancer immunotherapy, however, its high hydrophilicity makes it quickly excreted, leading to limited therapeutic efficiency even with frequent high-dose administration. Herein, we provide a pioneering report about the employment of VitC amphiphile self-assembled nanofiber hydrogels for enhanced cancer immunotherapy. Specifically, driven by hydrogen bonding and hydrophobic interactions, the synthesized VitC amphiphile, consisting of a hydrophilic VitC headgroup and a hydrophobic alkyl chain, could self-assemble into an injectable nanofiber hydrogel with self-healing properties. The formed VitC hydrogel not only serves as a reservoir for VitC but also acts as an effective delivery platform for stimulator of interferon genes (STING) agonist-4 (SA). Interestingly, the VitC hydrogel itself exhibits antitumor effects by upregulating genes related to interferon (IFN) signaling, apoptotic signaling and viral recognition and defense. Moreover, the SA-encapsulated VitC hydrogel (SA@VitC hydrogel) synergistically activated the immune system to inhibit the progression of both local and abscopal tumors

Failure of enhanced recovery after surgery in liver surgery: a systematic review and meta analysis

PurposeThis study aimed to conduct a systematic review of the literature to identify and summarize the existing evidence regarding ERAS failure and related risk factors after hepatic surgery. The objective was to provide physicians with a better understanding of these factors so that they can take appropriate action to minimize ERAS failure and improve patient outcomes.MethodA literature search of the PubMed MEDLINE, OVID, EMBASE, Cochrane Library, and Web of Science was performed. The search strategy involved terms related to ERAS, failure, and hepatectomy.ResultA meta-analysis was conducted on four studies encompassing a total of 1,535 patients, resulting in the identification of 20 risk factors associated with ERAS failure after hepatic surgery. Four of these risk factors were selected for pooling, including major resection, ASA classification of ≥3, advanced age, and male gender. Major resection and ASA ≥ 3 were identified as statistically significant factors of ERAS failure.ConclusionThe comprehensive literature review results indicated that the frequently identified risk factors for ERAS failure after hepatic surgery are linked to operative and anesthesia factors, including substantial resection and an American Society of Anesthesiologists score of 3 or higher. These insights will assist healthcare practitioners in taking prompt remedial measures. Nevertheless, there is a requirement for future high-quality randomized controlled trials with standardized evaluation frameworks for ERAS programs

RpoS Regulates a Novel Type of Plasmid DNA Transfer in Escherichia coli

Spontaneous plasmid transformation of Escherichia coli is independent of the DNA uptake machinery for single-stranded DNA (ssDNA) entry. The one-hit kinetic pattern of plasmid transformation indicates that double-stranded DNA (dsDNA) enters E. coli cells on agar plates. However, DNA uptake and transformation regulation remain unclear in this new type of plasmid transformation. In this study, we developed our previous plasmid transformation system and induced competence at early stationary phase. Despite of inoculum size, the development of competence was determined by optical cell density. DNase I interruption experiment showed that DNA was taken up exponentially within the initial 2 minutes and most transforming DNA entered E. coli cells within 10 minutes on LB-agar plates. A half-order kinetics between recipient cells and transformants was identified when cell density was high on plates. To determine whether the stationary phase master regulator RpoS plays roles in plasmid transformation, we investigated the effects of inactivating and over-expressing its encoding gene rpoS on plasmid transformation. The inactivation of rpoS systematically reduced transformation frequency, while over-expressing rpoS increased plasmid transformation. Normally, RpoS recognizes promoters by its lysine 173 (K173). We found that the K173E mutation caused RpoS unable to promote plasmid transformation, further confirming a role of RpoS in regulating plasmid transformation. In classical transformation, DNA was transferred across membranes by DNA uptake proteins and integrated by DNA processing proteins. At stationary growth phase, RpoS regulates some genes encoding membrane/periplasmic proteins and DNA processing proteins. We quantified transcription of 22 of them and found that transcription of only 4 genes (osmC, yqjC, ygiW and ugpC) encoding membrane/periplasmic proteins showed significant differential expression when wildtype RpoS and RpoSK173E mutant were expressed. Further investigation showed that inactivation of any one of these genes did not significantly reduce transformation, suggesting that RpoS may regulate plasmid transformation through other/multiple target genes