Model-Based Runtime Monitoring with Interactive Imitation Learning

Robot learning methods have recently made great strides, but generalization and robustness challenges still hinder their widespread deployment. Failing to detect and address potential failures renders state-of-the-art learning systems not combat-ready for high-stakes tasks. Recent advances in interactive imitation learning have presented a promising framework for human-robot teaming, enabling the robots to operate safely and continually improve their performances over long-term deployments. Nonetheless, existing methods typically require constant human supervision and preemptive feedback, limiting their practicality in realistic domains. This work aims to endow a robot with the ability to monitor and detect errors during task execution. We introduce a model-based runtime monitoring algorithm that learns from deployment data to detect system anomalies and anticipate failures. Unlike prior work that cannot foresee future failures or requires failure experiences for training, our method learns a latent-space dynamics model and a failure classifier, enabling our method to simulate future action outcomes and detect out-of-distribution and high-risk states preemptively. We train our method within an interactive imitation learning framework, where it continually updates the model from the experiences of the human-robot team collected using trustworthy deployments. Consequently, our method reduces the human workload needed over time while ensuring reliable task execution. Our method outperforms the baselines across system-level and unit-test metrics, with 23% and 40% higher success rates in simulation and on physical hardware, respectively. More information at https://ut-austin-rpl.github.io/sirius-runtime-monitor

Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment

With the rapid growth of computing powers and recent advances in deep learning, we have witnessed impressive demonstrations of novel robot capabilities in research settings. Nonetheless, these learning systems exhibit brittle generalization and require excessive training data for practical tasks. To harness the capabilities of state-of-the-art robot learning models while embracing their imperfections, we present Sirius, a principled framework for humans and robots to collaborate through a division of work. In this framework, partially autonomous robots are tasked with handling a major portion of decision-making where they work reliably; meanwhile, human operators monitor the process and intervene in challenging situations. Such a human-robot team ensures safe deployments in complex tasks. Further, we introduce a new learning algorithm to improve the policy's performance on the data collected from the task executions. The core idea is re-weighing training samples with approximated human trust and optimizing the policies with weighted behavioral cloning. We evaluate Sirius in simulation and on real hardware, showing that Sirius consistently outperforms baselines over a collection of contact-rich manipulation tasks, achieving an 8% boost in simulation and 27% on real hardware than the state-of-the-art methods, with twice faster convergence and 85% memory size reduction. Videos and code are available at https://ut-austin-rpl.github.io/sirius

Interactive Robot Learning from Verbal Correction

The ability to learn and refine behavior after deployment has become ever more important for robots as we design them to operate in unstructured environments like households. In this work, we design a new learning system based on large language model (LLM), OLAF, that allows everyday users to teach a robot using verbal corrections when the robot makes mistakes, e.g., by saying "Stop what you're doing. You should move closer to the cup." A key feature of OLAF is its ability to update the robot's visuomotor neural policy based on the verbal feedback to avoid repeating mistakes in the future. This is in contrast to existing LLM-based robotic systems, which only follow verbal commands or corrections but not learn from them. We demonstrate the efficacy of our design in experiments where a user teaches a robot to perform long-horizon manipulation tasks both in simulation and on physical hardware, achieving on average 20.0% improvement in policy success rate. Videos and more results are at https://ut-austin-rpl.github.io/olaf

A Novel Graphical Lasso based approach towards Segmentation Analysis in Energy Game-Theoretic Frameworks

Energy game-theoretic frameworks have emerged to be a successful strategy to encourage energy efficient behavior in large scale by leveraging human-in-the-loop strategy. A number of such frameworks have been introduced over the years which formulate the energy saving process as a competitive game with appropriate incentives for energy efficient players. However, prior works involve an incentive design mechanism which is dependent on knowledge of utility functions for all the players in the game, which is hard to compute especially when the number of players is high, common in energy game-theoretic frameworks. Our research proposes that the utilities of players in such a framework can be grouped together to a relatively small number of clusters, and the clusters can then be targeted with tailored incentives. The key to above segmentation analysis is to learn the features leading to human decision making towards energy usage in competitive environments. We propose a novel graphical lasso based approach to perform such segmentation, by studying the feature correlations in a real-world energy social game dataset. To further improve the explainability of the model, we perform causality study using grangers causality. Proposed segmentation analysis results in characteristic clusters demonstrating different energy usage behaviors. We also present avenues to implement intelligent incentive design using proposed segmentation method.Comment: Proceedings of the Special Session on Machine Learning in Energy Application, International Conference on Machine Learning and Applications (ICMLA) 2019. arXiv admin note: text overlap with arXiv:1810.1053

Cancer/testis antigens: promising immunotherapy targets for digestive tract cancers

Digestive tract cancers, including esophageal, gastric, and colorectal cancers, are the major cause of death among cancer patients worldwide due to the heterogeneity of cancer cells, which limits the effectiveness of traditional treatment methods. Immunotherapy represents a promising treatment strategy for improving the prognosis of patients with digestive tract cancers. However, the clinical application of this approach is limited by the absence of optimal targets. Cancer/testis antigens are characterized by low or absent expression in normal tissues, but high expression in tumor tissues, making them an attractive target for antitumor immunotherapy. Recent preclinical trials have shown promising results for cancer/testis antigen-targeted immunotherapy in digestive cancer. However, practical problems and difficulties in clinical application remain. This review presents a comprehensive analysis of cancer/testis antigens in digestive tract cancers, covering their expression, function, and potential as an immunotherapy target. Additionally, the current state of cancer/testis antigens in digestive tract cancer immunotherapy is discussed, and we predict that these antigens hold great promise as an avenue for breakthroughs in the treatment of digestive tract cancers

The Role of Ferroptosis and Cuproptosis in Curcumin against Hepatocellular Carcinoma

Background: Among cancer-related deaths, hepatocellular carcinoma (HCC) ranks fourth, and traditional Chinese medicine (TCM) treatment is an important complementary alternative therapy for HCC. Curcumin is a natural ingredient extracted from Curcuma longa with anti-HCC activity, while the therapeutic mechanisms of curcumin remain unclear, especially on ferroptosis and cuproptosis. Methods: Differentially expressed genes (DEGs) of curcumin treatment in PLC, KMCH, and Huh7 cells were identified, respectively. The common genes among them were then obtained to perform functional enrichment analysis and prognostic analysis. Moreover, weighted gene co-expression network analysis (WGCNA) was carried out for the construction of the co-expression network. The ferroptosis potential index (FPI) and the cuproptosis potential index (CPI) were subsequently used to quantitatively analyze the levels of ferroptosis and cuproptosis. Finally, single-cell transcriptome analysis of liver cancer was conducted. Results: We first identified 702, 515, and 721 DEGs from curcumin-treated PLC, KMCH, and Huh7 cells, respectively. Among them, HMOX1, CYP1A1, HMGCS2, LCN2, and MTTP may play an essential role in metal ion homeostasis. By WGCNA, grey60 co-expression module was associated with curcumin treatment and involved in the regulation of ion homeostasis. Furthermore, FPI and CPI assessment showed that curcumin had cell-specific effects on ferroptosis and cuproptosis in different HCC cells. In addition, there are also significant differences in ferroptosis and cuproptosis levels among 16 HCC cell subtypes according to single-cell transcriptome data analysis. Conclusions: We developed CPI and combined it with FPI to quantitatively analyze curcumin-treated HCC cells. It was found that ferroptosis and cuproptosis, two known metal ion-mediated forms of programmed cell death, may have a vital effect in treating HCC with curcumin, and there are significant differences in various liver cancer cell types and curcumin treatment which should be considered in the clinical application of curcumin