Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Tasks

Meta reinforcement learning (meta-RL) aims to learn a policy solving a set of training tasks simultaneously and quickly adapting to new tasks. It requires massive amounts of data drawn from training tasks to infer the common structure shared among tasks. Without heavy reward engineering, the sparse rewards in long-horizon tasks exacerbate the problem of sample efficiency in meta-RL. Another challenge in meta-RL is the discrepancy of difficulty level among tasks, which might cause one easy task dominating learning of the shared policy and thus preclude policy adaptation to new tasks. This work introduces a novel objective function to learn an action translator among training tasks. We theoretically verify that the value of the transferred policy with the action translator can be close to the value of the source policy and our objective function (approximately) upper bounds the value difference. We propose to combine the action translator with context-based meta-RL algorithms for better data collection and more efficient exploration during meta-training. Our approach empirically improves the sample efficiency and performance of meta-RL algorithms on sparse-reward tasks.Comment: Published in AAAI 202

The Real-Time and Embedded Soccer Robot Control System

Non

Federated Learning Attacks and Defenses: A Survey

In terms of artificial intelligence, there are several security and privacy deficiencies in the traditional centralized training methods of machine learning models by a server. To address this limitation, federated learning (FL) has been proposed and is known for breaking down ``data silos" and protecting the privacy of users. However, FL has not yet gained popularity in the industry, mainly due to its security, privacy, and high cost of communication. For the purpose of advancing the research in this field, building a robust FL system, and realizing the wide application of FL, this paper sorts out the possible attacks and corresponding defenses of the current FL system systematically. Firstly, this paper briefly introduces the basic workflow of FL and related knowledge of attacks and defenses. It reviews a great deal of research about privacy theft and malicious attacks that have been studied in recent years. Most importantly, in view of the current three classification criteria, namely the three stages of machine learning, the three different roles in federated learning, and the CIA (Confidentiality, Integrity, and Availability) guidelines on privacy protection, we divide attack approaches into two categories according to the training stage and the prediction stage in machine learning. Furthermore, we also identify the CIA property violated for each attack method and potential attack role. Various defense mechanisms are then analyzed separately from the level of privacy and security. Finally, we summarize the possible challenges in the application of FL from the aspect of attacks and defenses and discuss the future development direction of FL systems. In this way, the designed FL system has the ability to resist different attacks and is more secure and stable.Comment: IEEE BigData. 10 pages, 2 figures, 2 table

HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds

We present a novel deep neural network architecture for end-to-end scene flow estimation that directly operates on large-scale 3D point clouds. Inspired by Bilateral Convolutional Layers (BCL), we propose novel DownBCL, UpBCL, and CorrBCL operations that restore structural information from unstructured point clouds, and fuse information from two consecutive point clouds. Operating on discrete and sparse permutohedral lattice points, our architectural design is parsimonious in computational cost. Our model can efficiently process a pair of point cloud frames at once with a maximum of 86K points per frame. Our approach achieves state-of-the-art performance on the FlyingThings3D and KITTI Scene Flow 2015 datasets. Moreover, trained on synthetic data, our approach shows great generalization ability on real-world data and on different point densities without fine-tuning

Topological Transformation and Free-Space Transport of Photonic Hopfions

Structured light fields embody strong spatial variations of polarisation, phase and amplitude. Understanding, characterization and exploitation of such fields can be achieved through their topological properties. Three-dimensional (3D) topological solitons, such as hopfions, are 3D localized continuous field configurations with nontrivial particle-like structures, that exhibit a host of important topologically protected properties. Here, we propose and demonstrate photonic counterparts of hopfions with exact characteristics of Hopf fibration, Hopf index, and Hopf mapping from real-space vector beams to homotopic hyperspheres representing polarisation states. We experimentally generate photonic hopfions with on-demand high-order Hopf indices and independently controlled topological textures, including N\'eel-, Bloch-, and anti-skyrmionic types. We also demonstrate a robust free-space transport of photonic hopfions, thus, showing potential of hopfions for developing optical topological informatics and communications

A Coarse-to-fine Framework for Automated Kidney and Kidney Tumor Segmentation from Volumetric CT Images

Automatic semantic segmentation of kidney and kidney tumor is a promising tool for the treatment of kidney cancer. Due to the wide variety in kidney and kidney tumor morphology, it is still a great challenge to complete accurate segmentation of kidney and kidney tumor. We propose a new framework based on our previous work accepted by MICCAI2019, which is a coarse-to-fine segmentation framework to realize accurate and fast segmentation of kidney and kidney tumor

The inner nuclear membrane protein NEMP1 supports nuclear envelope openings and enucleation of erythroblasts

Nuclear envelope membrane proteins (NEMPs) are a conserved family of nuclear envelope (NE) proteins that reside within the inner nuclear membrane (INM). Even though Nemp1 knockout (KO) mice are overtly normal, they display a pronounced splenomegaly. This phenotype and recent reports describing a requirement for NE openings during erythroblasts terminal maturation led us to examine a potential role for Nemp1 in erythropoiesis. Here, we report that Nemp1 KO mice show peripheral blood defects, anemia in neonates, ineffective erythropoiesis, splenomegaly, and stress erythropoiesis. The erythroid lineage of Nemp1 KO mice is overrepresented until the pronounced apoptosis of polychromatophilic erythroblasts. We show that NEMP1 localizes to the NE of erythroblasts and their progenitors. Mechanistically, we discovered that NEMP1 accumulates into aggregates that localize near or at the edge of NE openings and Nemp1 deficiency leads to a marked decrease of both NE openings and ensuing enucleation. Together, our results for the first time demonstrate that NEMP1 is essential for NE openings and erythropoietic maturation in vivo and provide the first mouse model of defective erythropoiesis directly linked to the loss of an INM protein