Efficient Meta Reinforcement Learning for Preference-based Fast Adaptation

Learning new task-specific skills from a few trials is a fundamental challenge for artificial intelligence. Meta reinforcement learning (meta-RL) tackles this problem by learning transferable policies that support few-shot adaptation to unseen tasks. Despite recent advances in meta-RL, most existing methods require the access to the environmental reward function of new tasks to infer the task objective, which is not realistic in many practical applications. To bridge this gap, we study the problem of few-shot adaptation in the context of human-in-the-loop reinforcement learning. We develop a meta-RL algorithm that enables fast policy adaptation with preference-based feedback. The agent can adapt to new tasks by querying human's preference between behavior trajectories instead of using per-step numeric rewards. By extending techniques from information theory, our approach can design query sequences to maximize the information gain from human interactions while tolerating the inherent error of non-expert human oracle. In experiments, we extensively evaluate our method, Adaptation with Noisy OracLE (ANOLE), on a variety of meta-RL benchmark tasks and demonstrate substantial improvement over baseline algorithms in terms of both feedback efficiency and error tolerance.Comment: Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022

Direct tunneling through high-κ\kappa amorphous HfO2_2: effects of chemical modification

We report first principles modeling of quantum tunneling through amorphous HfO$_2$ dielectric layer of metal-oxide-semiconductor (MOS) nanostructures in the form of n-Si/HfO$_2$/Al. In particular we predict that chemically modifying the amorphous HfO$_2$ barrier by doping N and Al atoms in the middle region - far from the two interfaces of the MOS structure, can reduce the gate-to-channel tunnel leakage by more than one order of magnitude. Several other types of modification are found to enhance tunneling or induce substantial band bending in the Si, both are not desired from leakage point of view. By analyzing transmission coefficients and projected density of states, the microscopic physics of electron traversing the tunnel barrier with or without impurity atoms in the high-$\kappa$ dielectric is revealed.Comment: 5 pages, 5 figure

Expression and Clinical Relevance of uPA and ET-1 in Non-small Cell Lung Cancer

Background and objective uPA and ET-1 proteins have been reported to be up-regulated in some of human cancers. The aim of this study is to investigate the alteration and clinical relevance of uPA and ET-1 protein levels in non-small cell lung cancer (NSCLC). Methods Expressions of uPA and ET-1 protein were detected in 155 cases of NSCLC with tissue microarrays and immunohistochemistry (TMA-IHC) technique. The correlations between the alteration of the two proteins and clinicopathological parameters were analyzed. Results Negative/weak, moderate and high expression of uPA were observed in 12.3%, 64.4% and 23.3% of squamous cell carcinomas, in 12.2%, 53.7% and 34.1% of adenocarcinomas, and in 12.3%, 58.7% and 29.0% of all cases. ET-1 presented negative/weak, moderate and high expression in 2.7%, 42.5% and 54.8% of squamous cell carcinomas, in 11.0%, 30.5% and 58.5% of adenocarcinomas, and in 7.1%, 36.1% and 56.8% of all cases. Simultaneously high expression of uPA and ET-1 were found in adenocarcinomas without lymph node metastasis (P=0.017). Adenocarcinoma patients with high expression of uPA or with high expression of both ET-1 and uPA had the longer survival time (P=0.007 and 0.016). Conclusion Detection of uPA and ET-1 protein levels might contribute to the prognosis evaluation of NSCLC

A Compact Dual-Band MIMO Antenna for Sub-6 GHz 5G Terminals

In this paper, a dual-band multiple-input-multiple-output (MIMO) antenna is proposed for fifth-generation (5G) wireless communication terminals. The measured −10 dB impedance bandwidths of 380 MHz (3.34–3.72 GHz) and 560 MHz (4.57–5.13 GHz) can cover the 3.4–3.6 GHz and 4.8–5 GHz 5G bands. The single antenna element of this proposed MIMO is composed of an F-shaped feed strip and an inverted L-shaped radiation strip. A defected ground structure is employed to obtain a good isolation performance, whereby the measured isolation between the antenna elements is observed to be larger than 23 dB. The measured total radiation efficiencies at 3.5 GHz and 4.9 GHz are 76.65% and 71.93%, respectively. Besides, the calculated envelope correlation coefficients (ECC) are less than 0.00125 and 0.01164 at the low-frequency and high-frequency bands, respectively. Furthermore, the specific absorption ratio (SAR) analysis of the antenna verifies that it qualifies for 5G terminals

Soil Moisture Retrieval Using BuFeng-1 A/B Based on Land Surface Clustering Algorithm

A new land surface clustering algorithm is developed to retrieve soil moisture (SM) using the Global Navigation Satellite System reflectometry (GNSS-R) technique. Data from the BuFeng-1 (BF-1) twin satellites A/B, a pilot mission for the Chinese GNSS-R constellation, is used for SM retrieval. The core concept of the algorithm is to cluster global land areas into different types according to the land properties and calculate the SM type by type, based on the linear relationship between equivalent specular reflectivity and SM. The global comparison between the results and SM product from the Soil Moisture Active Passive mission shows the correlation coefficient (R) is 0.82, and unbiased root mean square error (ubRMSE) is 0.070 cm3·cm-3. The results also show good agreement compared with in situ SM measurements with the mean ubRMSE of 0.036 cm3·cm-3. This study proves that the global SM can be retrieved successfully from the BF-1 mission with the land surface clustering algorithm. By taking full advantage of the similarity of land surface physical properties in different regions, the algorithm provides a practical approach for global SM retrieval using spaceborne GNSS-R data.10.13039/501100001809-National Natural Science Foundation of China (Grant Number: 41971377). China Spacesat Company, Ltd. ESA-MOST China Dragon5 Programme (Grant Number: ID.58070) 10.13039/501100003392-Natural Science Foundation of Fujian Province (Grant Number: 2019J01853

Integrated multi-omics identified the novel intratumor microbiome-derived subtypes and signature to predict the outcome, tumor microenvironment heterogeneity, and immunotherapy response for pancreatic cancer patients

Background: The extremely malignant tumour known as pancreatic cancer (PC) lacks efficient prognostic markers and treatment strategies. The microbiome is crucial to how cancer develops and responds to treatment. Our study was conducted in order to better understand how PC patients’ microbiomes influence their outcome, tumour microenvironment, and responsiveness to immunotherapy.Methods: We integrated transcriptome and microbiome data of PC and used univariable Cox regression and Kaplan–Meier method for screening the prognostic microbes. Then intratumor microbiome-derived subtypes were identified using consensus clustering. We utilized LASSO and Cox regression to build the microbe-related model for predicting the prognosis of PC, and utilized eight algorithms to assess the immune microenvironment feature. The OncoPredict package was utilized to predict drug treatment response. We utilized qRT-PCR to verify gene expression and single-cell analysis to reveal the composition of PC tumour microenvironment.Results: We obtained a total of 26 prognostic genera in PC. And PC samples were divided into two microbiome-related subtypes: Mcluster A and B. Compared with Mcluster A, patients in Mcluster B had a worse prognosis and higher TNM stage and pathological grade. Immune analysis revealed that neutrophils, regulatory T cell, CD8+ T cell, macrophages M1 and M2, cancer associated fibroblasts, myeloid dendritic cell, and activated mast cell had remarkably higher infiltrated levels within the tumour microenvironment of Mcluster B. Patients in Mcluster A were more likely to benefit from CTLA-4 blockers and were highly sensitive to 5-fluorouracil, cisplatin, gemcitabine, irinotecan, oxaliplatin, and epirubicin. Moreover, we built a microbe-derived model to assess the outcome. The ROC curves showed that the microbe-related model has good predictive performance. The expression of LAMA3 and LIPH was markedly increased within pancreatic tumour tissues and was linked to advanced stage and poor prognosis. Single-cell analysis indicated that besides cancer cells, the tumour microenvironment of PC was also rich in monocytes/macrophages, endothelial cells, and fibroblasts. LIPH and LAMA3 exhibited relatively higher expression in cancer cells and neutrophils.Conclusion: The intratumor microbiome-derived subtypes and signature in PC were first established, and our study provided novel perspectives on PC prognostic indicators and treatment options