Your Room is not Private: Gradient Inversion Attack on Reinforcement Learning

The prominence of embodied Artificial Intelligence (AI), which empowers robots to navigate, perceive, and engage within virtual environments, has attracted significant attention, owing to the remarkable advancements in computer vision and large language models. Privacy emerges as a pivotal concern within the realm of embodied AI, as the robot accesses substantial personal information. However, the issue of privacy leakage in embodied AI tasks, particularly in relation to reinforcement learning algorithms, has not received adequate consideration in research. This paper aims to address this gap by proposing an attack on the value-based algorithm and the gradient-based algorithm, utilizing gradient inversion to reconstruct states, actions, and supervision signals. The choice of using gradients for the attack is motivated by the fact that commonly employed federated learning techniques solely utilize gradients computed based on private user data to optimize models, without storing or transmitting the data to public servers. Nevertheless, these gradients contain sufficient information to potentially expose private data. To validate our approach, we conduct experiments on the AI2THOR simulator and evaluate our algorithm on active perception, a prevalent task in embodied AI. The experimental results demonstrate the effectiveness of our method in successfully reconstructing all information from the data across 120 room layouts.Comment: 7 pages, 4 figures, 2 table

Effects of Cations and PH on Antimicrobial Activity of Thanatin and s-Thanatin against _Escherichia coli_ ATCC25922 and _B. subtilis_ ATCC 21332

Thanatin and s-thanatin were insect antimicrobial peptides which have shown potent antimicrobial activities on a variety of microbes. In order to investigate the effect of cations and pH on the activity of these peptides against Gram-negative bacteria and Gram-positive bacteria, the antimicrobial activities of both peptides were studied in increasing concentrations of monovalent cations (K^+^ and Na^+^), divalent cations (Ca^2+^ and Mg^2+^) and H^+^. The NCCLS broth microdilution method showed that both peptides were sensitive to the presence of cations. The divalent cations showed more antagonized effect on the activity against Gram-negative bacteria than the monovalent cations, since the two peptides lost the ability to inhibit bacterial growth at a very low concentration. In addition, the activities of both peptides tested were not significantly affected by pH. Comparing to studies of other antibacterial peptide activities, our data support a hypothesis that positive ions affect the sensitivity to cation peptides

Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoning

As a pivotal component to attaining generalizable solutions in human intelligence, reasoning provides great potential for reinforcement learning (RL) agents' generalization towards varied goals by summarizing part-to-whole arguments and discovering cause-and-effect relations. However, how to discover and represent causalities remains a huge gap that hinders the development of causal RL. In this paper, we augment Goal-Conditioned RL (GCRL) with Causal Graph (CG), a structure built upon the relation between objects and events. We novelly formulate the GCRL problem into variational likelihood maximization with CG as latent variables. To optimize the derived objective, we propose a framework with theoretical performance guarantees that alternates between two steps: using interventional data to estimate the posterior of CG; using CG to learn generalizable models and interpretable policies. Due to the lack of public benchmarks that verify generalization capability under reasoning, we design nine tasks and then empirically show the effectiveness of the proposed method against five baselines on these tasks. Further theoretical analysis shows that our performance improvement is attributed to the virtuous cycle of causal discovery, transition modeling, and policy training, which aligns with the experimental evidence in extensive ablation studies.Comment: 28 pages, 5 figures, under revie

Applicability of the Friedberg-Lee-Zhao method

Friedberg, Lee and Zhao proposed a method for effectively evaluating the eigenenergies and eigen wavefunctions of quantum systems. In this work, we study several special cases to investigate applicability of the method. Concretely, we calculate the ground-state eigenenergy of the Hellmann potential as well as the Cornell potential, and also evaluate the energies of the systems where linear term is added to the Coulomb and harmonic oscillator potentials as a perturbation. The results obtained in this method have a surprising agreement with the traditional method or the numerical results. Since the results in this method have obvious analyticity compared to that in other methods, and because of the simplicity for calculations this method can be applied to solving the Schr\"{o}dinger equation and provides us better understanding of the physical essence of the concerned systems. But meanwhile applications of the FLZ method are restricted at present, especially for certain potential forms, but due to its obvious advantages, it should be further developed.Comment: 14 pages,no figure