Mind Your Data! Hiding Backdoors in Offline Reinforcement Learning Datasets

A growing body of research works has focused on the Offline Reinforcement Learning (RL) paradigm. Data providers share large pre-collected datasets on which others can train high-quality agents without interacting with the environments. Such an offline RL paradigm has demonstrated effectiveness in many critical tasks, including robot control, autonomous driving, etc. A well-trained agent can be regarded as a software system. However, less attention is paid to investigating the security threats to the offline RL system. In this paper, we focus on a critical security threat: backdoor attacks. Given normal observations, an agent implanted with backdoors takes actions leading to high rewards. However, the same agent takes actions that lead to low rewards if the observations are injected with triggers that can activate the backdoor. In this paper, we propose Baffle (Backdoor Attack for Offline Reinforcement Learning) and evaluate how different Offline RL algorithms react to this attack. Our experiments conducted on four tasks and four offline RL algorithms expose a disquieting fact: none of the existing offline RL algorithms is immune to such a backdoor attack. More specifically, Baffle modifies $10\%$ of the datasets for four tasks (3 robotic controls and 1 autonomous driving). Agents trained on the poisoned datasets perform well in normal settings. However, when triggers are presented, the agents' performance decreases drastically by $63.6\%$, $57.8\%$, $60.8\%$ and $44.7\%$ in the four tasks on average. The backdoor still persists after fine-tuning poisoned agents on clean datasets. We further show that the inserted backdoor is also hard to be detected by a popular defensive method. This paper calls attention to developing more effective protection for the open-source offline RL dataset.Comment: 13 pages, 6 figure

Study on standard response spectrum parameters of special long-period ground movements

Multiple earthquake damages show that the long-period ground movements have an amplification effect on the dynamic response and seismic damage of long-period structures. To form an improved code design spectrum suitable for long-period ground movements, the standard response spectra consisting of four segments for near-fault pulse-like (NFPL) and far-field harmonic (FFH) ground movements are proposed. Firstly, two types of special long-period ground movements with reliable information are selected for this research. Then, the fundamental period of a single-degree-of-freedom (SDOF) system is extended to 16 s with the damping ratio of 5 %, and the seismic response spectrum and the normalized response spectrum are analyzed. Next, the normalized acceleration spectrum of long-period ground movements is calibrated to the standard response spectrum of regular variation. Finally, the parameters of standard response spectrum about long-period ground movements and the ones of the current code design spectrum are discussed. The results show that the current code design spectrum overestimates the structural seismic response in the short period under NFPL and FFH ground movements, while it underestimates the structural seismic response in the long period. The fundamental period of a SDOF system of china’s current code design spectrum should be extended to consider the influence on long-period structures acted by a long-period earthquake. Further, it accounts for the resonance effect of long-period ground movements which ensures the seismic design safety of long-period structures

Tislelizumab in Patients with Previously Treated Advanced Hepatocellular Carcinoma (RATIONALE-208): A Multicenter, Non-Randomized, Open-Label, Phase 2 Trial

Introduction: Tislelizumab (anti-programmed cell death protein 1 antibody) showed preliminary antitumor activity and tolerability in patients with advanced solid tumors, including hepatocellular carcinoma (HCC). This study aimed to assess the efficacy and safety of tislelizumab in patients with previously treated advanced HCC. Methods: The multi-regional phase 2 study, RATIONALE-208, examined single-agent tislelizumab (200 mg intravenously every three weeks) in patients with advanced HCC with Child-Pugh A, Barcelona Clinic Liver Cancer stage B or C, and who had received one or more prior lines of systemic therapy. The primary endpoint was objective response rate (ORR), radiologically confirmed per Response Evaluation Criteria in Solid Tumors version 1.1 by Independent Review Committee. Safety was assessed in patients who received ≥1 dose of tislelizumab. Results: Between April 9, 2018 and February 27, 2019, 249 eligible patients were enrolled and treated. After a median study follow-up of 12.7 months, ORR was 13% (n = 32/249; 95% confidence interval [CI], 9–18), including five complete and 27 partial responses. Number of prior lines of therapy did not impact ORR (one prior line, 13% [95% CI, 8–20]; two or more prior lines, 13% [95% CI, 7–20]). Median duration of response was not reached. Disease control rate was 53% and median overall survival was 13.2 months. Of the 249 total patients, grade ≥3 treatment-related adverse events were reported in 38 (15%) patients; the most common was liver transaminase elevations in 10 (4%) patients. Treatment-related adverse events led to treatment discontinuation in 13 (5%) patients or dose delay in 46 (19%) patients. No deaths were attributed to the treatment per investigator assessment. Conclusion: Tislelizumab demonstrated durable objective responses, regardless of the number of prior lines of therapy, and acceptable tolerability in patients with previously treated advanced HCC

Evaluation of Intra-Host Variants of the Entire Hepatitis B Virus Genome

Genetic analysis of hepatitis B virus (HBV) frequently involves study of intra-host variants, identification of which is commonly achieved using short regions of the HBV genome. However, the use of short sequences significantly limits evaluation of genetic relatedness among HBV strains. Although analysis of HBV complete genomes using genetic cloning has been developed, its application is highly labor intensive and practiced only infrequently. We describe here a novel approach to whole genome (WG) HBV quasispecies analysis based on end-point, limiting-dilution real-time PCR (EPLD-PCR) for amplification of single HBV genome variants, and their subsequent sequencing. EPLD-PCR was used to analyze WG quasispecies from serum samples of patients (n = 38) infected with HBV genotypes A, B, C, D, E and G. Phylogenetic analysis of the EPLD-isolated HBV-WG quasispecies showed the presence of mixed genotypes, recombinant variants and sub-populations of the virus. A critical observation was that HBV-WG consensus sequences obtained by direct sequencing of PCR fragments without EPLD are genetically close, but not always identical to the major HBV variants in the intra-host population, thus indicating that consensus sequences should be judiciously used in genetic analysis. Sequence-based studies of HBV WG quasispecies should afford a more accurate assessment of HBV evolution in various clinical and epidemiological settings

Clonal evolution in liver cancer at single-cell and single-variant resolution.

Genetic heterogeneity of tumor is closely related to its clonal evolution, phenotypic diversity and treatment resistance, and such heterogeneity has only been characterized at single-cell sub-chromosomal scale in liver cancer. Here we reconstructed the single-variant resolution clonal evolution in human liver cancer based on single-cell mutational profiles. The results indicated that key genetic events occurred early during tumorigenesis, and an early metastasis followed by independent evolution was observed in primary liver tumor and intrahepatic metastatic portal vein tumor thrombus. By parallel single-cell RNA-Seq, the transcriptomic phenotype of HCC was found to be related with genetic heterogeneity. For the first time we reconstructed the single-cell and single-variant clonal evolution in human liver cancer, and dissection of both genetic and phenotypic heterogeneity will facilitate better understanding of their relationship

Performance Analysis of Restraint Grouting-Anchoring Lap Connection of Rebar in Prefabricated Shear Walls

This study aims to obtain the key engineering parameters for the reliability performance of restraint grouting-anchoring lap connection of rebar, so as to achieve the promotion of prefabricated building. To study the performance of restraint grouting-anchoring lap connection of rebar in prefabricated shear walls, four parameters rebar diameter, lap length, bellows diameter and concrete strength were selected as the main subjects of the test. Specimens were prepared according to four important parameters. The test results show that the connection performance is reliable in the specimen of restraint grouting-anchoring lap connection of rebar for pre-drilled bellows. Through theoretical analysis, a force model was proposed to provide a quantitative analysis method for the theoretical calculation of the mechanical properties of restraint grouting-anchoring lap connection of rebar. As the diameter of the rebar increases, the spread of concrete stresses in the specimens becomes more pronounced; as the diameter of the rebar increases, the spread of concrete stresses in the specimens becomes more pronounced; the shorter the lap length, the higher the bond stress between the rebar and the concrete interface in the specimen, the damage of the specimen is from mild to severe; as the strength of the concrete increases, a larger diameter of rebar and a shorter lap length are required to achieve the same damage pattern in the specimen

Experimental Study on Cumulative Damage Behavior of Steel-Reinforced Concrete Columns

The cumulative damage behavior of SRC columns under far-field long-period ground motions was simulated and studied by quasi-static tests with the same displacement for 10 times. Quasi-static tests of 8 SRC columns were conducted under the horizontal cyclic loading with the same displacement for 10 times or 3 times, and then the effects of steel ratio, stirrup ratio, axial compression ratio, and number of cyclic loading on the cumulative damage of SRC columns under the far-field long-period ground motions were studied. The results showed that the number of cyclic loading had little effect on the peak load of the specimens, but had a significant effect on the deformation capacity, stiffness degradation, and energy dissipation capacity. Compared with the specimens after 3 cycles, the displacement ductility coefficient of specimens after 10 cycles was reduced by about 20%–26%, the ultimate hysteresis energy dissipation was reduced by 35%–48%, while the stiffness degradation rate was accelerated. After the peak load, the cumulative damage caused by multiple cyclic loading with the same displacement was more significant, which aggravated the reduction of bearing capacity and stiffness degradation. The smaller the steel ratio and stirrup ratio, the larger the axial compression ratio, and the greater the reduction of the bearing capacity and stiffness of specimens. However, accumulated damage caused by multiple cyclic loading with the same displacement had a slight impact on the energy dissipation capacity. Increasing the steel ratio and stirrup ratio can effectively improve the deformation capacity and energy dissipation capacity of the specimens and reduce the bearing capacity and stiffness degradation caused by cumulative damage