Reward Teaching for Federated Multi-armed Bandits

Most of the existing federated multi-armed bandits (FMAB) designs are based on the presumption that clients will implement the specified design to collaborate with the server. In reality, however, it may not be possible to modify the clients' existing protocols. To address this challenge, this work focuses on clients who always maximize their individual cumulative rewards, and introduces a novel idea of ``reward teaching'', where the server guides the clients towards global optimality through implicit local reward adjustments. Under this framework, the server faces two tightly coupled tasks of bandit learning and target teaching, whose combination is non-trivial and challenging. A phased approach, called Teaching-After-Learning (TAL), is first designed to encourage and discourage clients' explorations separately. General performance analyses of TAL are established when the clients' strategies satisfy certain mild requirements. With novel technical approaches developed to analyze the warm-start behaviors of bandit algorithms, particularized guarantees of TAL with clients running UCB or epsilon-greedy strategies are then obtained. These results demonstrate that TAL achieves logarithmic regrets while only incurring logarithmic adjustment costs, which is order-optimal w.r.t. a natural lower bound. As a further extension, the Teaching-While-Learning (TWL) algorithm is developed with the idea of successive arm elimination to break the non-adaptive phase separation in TAL. Rigorous analyses demonstrate that when facing clients with UCB1, TWL outperforms TAL in terms of the dependencies on sub-optimality gaps thanks to its adaptive design. Experimental results demonstrate the effectiveness and generality of the proposed algorithms.Comment: Accepted to IEEE Transactions on Signal Processin

HIF-1α Contributes to Hypoxia-induced Invasion and Metastasis of Esophageal Carcinoma via Inhibiting E-cadherin and Promoting MMP-2 Expression

Hypoxia-inducible factor-1α (HIF-1α) has been found to enhance tumor invasion and metastasis, but no study has reported its action in esophageal carcinoma. The goal of this study was to explore the probable mechanism of HIF-1α in the invasion and metastasis of esophageal carcinoma Eca109 cells in vitro and in vivo. mRNA and protein expression of HIF-1α, E-cadherin and matrix metalloproteinase-2 (MMP-2) under hypoxia were detected by RT-PCR and Western blotting. The effects of silencing HIF-1α on E-cadherin, MMP-2 mRNA and protein expression under hypoxia or normoxia were detected by RT-PCR and Western blotting, respectively. The invasive ability of Eca109 cells was tested using a transwell chambers. We established an Eca109-implanted tumor model and observed tumor growth and lymph node metastasis. The expression of HIF-1α, E-cadherin and MMP-2 in xenograft tumors was detected by Western blotting. After exposure to hypoxia, HIF-1α protein was up-regulated, both mRNA and protein levels of E-cadherin were down-regulated and MMP-2 was up-regulated, while HIF-1α mRNA showed no significant change. SiRNA could block HIF-1α effectively, increase E-cadherin expression and inhibit MMP-2 expression. The number of invading cells decreased after HIF-1α was silenced. Meanwhile, the tumor volume was much smaller, and the metastatic rate of lymph nodes and the positive rate were lower in vivo. Our observations suggest that HIF-1α inhibition might be an effective strategy to weaken invasion and metastasis in the esophageal carcinoma Eca109 cell line