Efficient Deep Reinforcement Learning via Adaptive Policy Transfer

Transfer Learning (TL) has shown great potential to accelerate Reinforcement Learning (RL) by leveraging prior knowledge from past learned policies of relevant tasks. Existing transfer approaches either explicitly computes the similarity between tasks or select appropriate source policies to provide guided explorations for the target task. However, how to directly optimize the target policy by alternatively utilizing knowledge from appropriate source policies without explicitly measuring the similarity is currently missing. In this paper, we propose a novel Policy Transfer Framework (PTF) to accelerate RL by taking advantage of this idea. Our framework learns when and which source policy is the best to reuse for the target policy and when to terminate it by modeling multi-policy transfer as the option learning problem. PTF can be easily combined with existing deep RL approaches. Experimental results show it significantly accelerates the learning process and surpasses state-of-the-art policy transfer methods in terms of learning efficiency and final performance in both discrete and continuous action spaces.Comment: Accepted by IJCAI'202

NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illumination

Inverse rendering methods aim to estimate geometry, materials and illumination from multi-view RGB images. In order to achieve better decomposition, recent approaches attempt to model indirect illuminations reflected from different materials via Spherical Gaussians (SG), which, however, tends to blur the high-frequency reflection details. In this paper, we propose an end-to-end inverse rendering pipeline that decomposes materials and illumination from multi-view images, while considering near-field indirect illumination. In a nutshell, we introduce the Monte Carlo sampling based path tracing and cache the indirect illumination as neural radiance, enabling a physics-faithful and easy-to-optimize inverse rendering method. To enhance efficiency and practicality, we leverage SG to represent the smooth environment illuminations and apply importance sampling techniques. To supervise indirect illuminations from unobserved directions, we develop a novel radiance consistency constraint between implicit neural radiance and path tracing results of unobserved rays along with the joint optimization of materials and illuminations, thus significantly improving the decomposition performance. Extensive experiments demonstrate that our method outperforms the state-of-the-art on multiple synthetic and real datasets, especially in terms of inter-reflection decomposition.Comment: Accepted in CVPR 202

Mitochondrial Dysfunction Promotes Breast Cancer Cell Migration and Invasion through HIF1α Accumulation via Increased Production of Reactive Oxygen Species

Although mitochondrial dysfunction has been observed in various types of human cancer cells, the molecular mechanism underlying mitochondrial dysfunction mediated tumorigenesis remains largely elusive. To further explore the function of mitochondria and their involvement in the pathogenic mechanisms of cancer development, mitochondrial dysfunction clones of breast cancer cells were generated by rotenone treatment, a specific inhibitor of mitochondrial electron transport complex I. These clones were verified by mitochondrial respiratory defect measurement. Moreover, those clones exhibited increased reactive oxygen species (ROS), and showed higher migration and invasive behaviors compared with their parental cells. Furthermore, antioxidant N-acetyl cysteine, PEG-catalase, and mito-TEMPO effectively inhibited cell migration and invasion in these clones. Notably, ROS regulated malignant cellular behavior was in part mediated through upregulation of hypoxia-inducible factor-1 α and vascular endothelial growth factor. Our results suggest that mitochondrial dysfunction promotes cancer cell motility partly through HIF1α accumulation mediated via increased production of reactive oxygen species