Safety-Critical Learning of Robot Control with Temporal Logic Specifications

Reinforcement learning (RL) is a promising approach. However, success is limited to real-world applications, because ensuring safe exploration and facilitating adequate exploitation is a challenge for controlling robotic systems with unknown models and measurement uncertainties. The learning problem becomes even more difficult for complex tasks over continuous state-action. In this paper, we propose a learning-based robotic control framework consisting of several aspects: (1) we leverage Linear Temporal Logic (LTL) to express complex tasks over infinite horizons that are translated to a novel automaton structure; (2) we detail an innovative reward scheme for LTL satisfaction with a probabilistic guarantee. Then, by applying a reward shaping technique, we develop a modular policy-gradient architecture exploiting the benefits of the automaton structure to decompose overall tasks and enhance the performance of learned controllers; (3) by incorporating Gaussian Processes (GPs) to estimate the uncertain dynamic systems, we synthesize a model-based safe exploration during the learning process using Exponential Control Barrier Functions (ECBFs) that generalize systems with high-order relative degrees; (4) to further improve the efficiency of exploration, we utilize the properties of LTL automata and ECBFs to propose a safe guiding process. Finally, we demonstrate the effectiveness of the framework via several robotic environments. We show an ECBF-based modular deep RL algorithm that achieves near-perfect success rates and safety guarding with high probability confidence during training.Comment: Under Review. arXiv admin note: text overlap with arXiv:2102.1285

Control Barrier Function for Linearizable Systems with High Relative Degrees from Signal Temporal Logics: A Reference Governor Approach

This paper considers the safety-critical navigation problem with Signal Temporal Logic (STL) tasks. We developed an explicit reference governor-guided control barrier function (ERG-guided CBF) method that enables the application of first-order CBFs to high-order linearizable systems. This method significantly reduces the conservativeness of the existing CBF approaches for high-order systems. Furthermore, our framework provides safety-critical guarantees in the sense of obstacle avoidance by constructing the margin of safety and updating direction of safe evolution in the agent's state space. To improve control performance and enhance STL satisfaction, we employ efficient gradient-based methods for iteratively learning optimal parameters of ERG-guided CBF. We validate the algorithm through both high-order linear and nonlinear systems. A video demonstration can be found on: \url{https://youtu.be/ZRmsA2FeFR4

A Robotic Visual Grasping Design: Rethinking Convolution Neural Network with High-Resolutions

High-resolution representations are important for vision-based robotic grasping problems. Existing works generally encode the input images into low-resolution representations via sub-networks and then recover high-resolution representations. This will lose spatial information, and errors introduced by the decoder will be more serious when multiple types of objects are considered or objects are far away from the camera. To address these issues, we revisit the design paradigm of CNN for robotic perception tasks. We demonstrate that using parallel branches as opposed to serial stacked convolutional layers will be a more powerful design for robotic visual grasping tasks. In particular, guidelines of neural network design are provided for robotic perception tasks, e.g., high-resolution representation and lightweight design, which respond to the challenges in different manipulation scenarios. We then develop a novel grasping visual architecture referred to as HRG-Net, a parallel-branch structure that always maintains a high-resolution representation and repeatedly exchanges information across resolutions. Extensive experiments validate that these two designs can effectively enhance the accuracy of visual-based grasping and accelerate network training. We show a series of comparative experiments in real physical environments at Youtube: https://youtu.be/Jhlsp-xzHFY

A Case report: Synovial sarcoma of the mediastinum in an 18-year-old teenager

Synovial sarcomas (SSs) are a rare group of malignant tumors originating from pluripotential mesenchymal cells, which commonly occur as the primary tumor in the soft tissues near the articular surface, tendons, and articular synovium. Herein, we report a rare case of mediastinal SS in an 18-year-old teenager who initially presented with cough as the primary symptom. In this case, plain chest CT and contrast-enhanced CT clearly revealed the lesion presenting as a round-like and uneven density mass in the mediastinum with heterogeneous enhancement, which compressed the trachea and invaded the adjacent vessels. Based on the results of immunohistochemistry and fluorescence in situ hybridization (FISH), combined with the differential diagnosis with other types of tumors in the mediastinum on imaging, we were able to diagnose the tumor as an SS located in the mediastinum. Subsequent resection of the lesion coupled with chemotherapy and immunotherapy led to an improvement in the patient’s symptoms

Burst phase distribution of SGR J1935+2154 based on Insight-HXMT

On April 27, 2020, the soft gamma ray repeater SGR J1935+2154 entered its intense outburst episode again. Insight-HXMT carried out about one month observation of the source. A total number of 75 bursts were detected during this activity episode by Insight-HXMT, and persistent emission data were also accumulated. We report on the spin period search result and the phase distribution of burst start times and burst photon arrival times of the Insight-HXMT high energy detectors and Fermi Gamma-ray Burst Monitor (GBM). We find that the distribution of burst start times is uniform within its spin phase for both Insight-HXMT and Fermi-GBM observations, whereas the phase distribution of burst photons is related to the type of a burst's energy spectrum. The bursts with the same spectrum have different distribution characteristics in the initial and decay episodes for the activity of magnetar SGR J1935+2154.Comment: 12 pages, 9 figure

CircUBAP2 Promotes MMP9-Mediated Oncogenic Effect via Sponging miR-194-3p in Hepatocellular Carcinoma

BackgroundThe physiological regulatory functions of circRNAs have become a topic of intensive research in recent years. Increasing evidence supports a significant role of circRNAs during cancer initiation and progression, including hepatocellular carcinoma (HCC).Materials and MethodsA bioinformatics analysis from three independent Gene Expression Omnibus (GEO) databases was performed to profile and screen the dysregulated circRNAs in HCC. RT-qPCR was used to examine the expression level of circUBAP2 in HCC and adjacent non-tumor tissues. Then, proliferation assays (CCK8 and colony formation) and migration assays (transwell and wound healing) were performed to examine effect of circUBAP2 in vitro. Immunoprecipitation, RNA pulldown, FISH, and dual-luciferase reporter assay was conducted to explore the circUBAP2-related mechanism for regulating HCC progression. Moreover, a mouse xenograft model and a mouse lung metastasis model confirmed the effect of circUBAP2 in vivo.ResultsIn this study, we found a novel circRNA: circUBAP2, which was identified by bioinformatics analysis. Among 91 HCC patients, circUBAP2 was significantly upregulated in HCC tissues, and negatively correlated with aggressive clinical characteristics and prognosis. Functional assays demonstrated that circUBAP2 promoted cell proliferation, colony formation, migration, and invasion in vitro. Moreover, circUBAP2 enhanced tumor growth and pulmonary metastasis in vivo. Mechanistically, circUBAP2 acts as a competing endogenous RNA (ceRNA) for miR-194-3p, a tumor suppressor in HCC. We confirmed that MMP9 was direct target for miR-194-3p, which was regulated by circUBAP2.ConclusionCircUBAP2 plays a significant role in promoting HCC via the miR-194-3p/MMP9 pathway and could serve as a promising prognostic biomarker and novel therapeutic target for HCC patients