Local Clustering in Contextual Multi-Armed Bandits

We study identifying user clusters in contextual multi-armed bandits (MAB). Contextual MAB is an effective tool for many real applications, such as content recommendation and online advertisement. In practice, user dependency plays an essential role in the user's actions, and thus the rewards. Clustering similar users can improve the quality of reward estimation, which in turn leads to more effective content recommendation and targeted advertising. Different from traditional clustering settings, we cluster users based on the unknown bandit parameters, which will be estimated incrementally. In particular, we define the problem of cluster detection in contextual MAB, and propose a bandit algorithm, LOCB, embedded with local clustering procedure. And, we provide theoretical analysis about LOCB in terms of the correctness and efficiency of clustering and its regret bound. Finally, we evaluate the proposed algorithm from various aspects, which outperforms state-of-the-art baselines.Comment: 12 page

Graph Neural Bandits

Contextual bandits algorithms aim to choose the optimal arm with the highest reward out of a set of candidates based on the contextual information. Various bandit algorithms have been applied to real-world applications due to their ability of tackling the exploitation-exploration dilemma. Motivated by online recommendation scenarios, in this paper, we propose a framework named Graph Neural Bandits (GNB) to leverage the collaborative nature among users empowered by graph neural networks (GNNs). Instead of estimating rigid user clusters as in existing works, we model the "fine-grained" collaborative effects through estimated user graphs in terms of exploitation and exploration respectively. Then, to refine the recommendation strategy, we utilize separate GNN-based models on estimated user graphs for exploitation and adaptive exploration. Theoretical analysis and experimental results on multiple real data sets in comparison with state-of-the-art baselines are provided to demonstrate the effectiveness of our proposed framework.Comment: Accepted to SIGKDD 202

Genome-wide transcriptome analysis of gametophyte development in Physcomitrella patens

<p>Abstract</p> <p>Background</p> <p>Regulation of gene expression plays a pivotal role in controlling the development of multicellular plants. To explore the molecular mechanism of plant developmental-stage transition and cell-fate determination, a genome-wide analysis was undertaken of sequential developmental time-points and individual tissue types in the model moss <it>Physcomitrella patens </it>because of the short life cycle and relative structural simplicity of this plant.</p> <p>Results</p> <p>Gene expression was analyzed by digital gene expression tag profiling of samples taken from <it>P. patens </it>protonema at 3, 14 and 24 days, and from leafy shoot tissues at 30 days, after protoplast isolation, and from 14-day-old caulonemal and chloronemal tissues. In total, 4333 genes were identified as differentially displayed. Among these genes, 4129 were developmental-stage specific and 423 were preferentially expressed in either chloronemal or caulonemal tissues. Most of the differentially displayed genes were assigned to functions in organic substance and energy metabolism or macromolecule biosynthetic and catabolic processes based on gene ontology descriptions. In addition, some regulatory genes identified as candidates might be involved in controlling the developmental-stage transition and cell differentiation, namely MYB-like, HB-8, AL3, zinc finger family proteins, bHLH superfamily, GATA superfamily, GATA and bZIP transcription factors, protein kinases, genes related to protein/amino acid methylation, and auxin, ethylene, and cytokinin signaling pathways.</p> <p>Conclusions</p> <p>These genes that show highly dynamic changes in expression during development in <it>P. patens </it>are potential targets for further functional characterization and evolutionary developmental biology studies.</p

Contextual Bandits with Online Neural Regression

Recent works have shown a reduction from contextual bandits to online regression under a realizability assumption [Foster and Rakhlin, 2020, Foster and Krishnamurthy, 2021]. In this work, we investigate the use of neural networks for such online regression and associated Neural Contextual Bandits (NeuCBs). Using existing results for wide networks, one can readily show a ${\mathcal{O}}(\sqrt{T})$ regret for online regression with square loss, which via the reduction implies a ${\mathcal{O}}(\sqrt{K} T^{3/4})$ regret for NeuCBs. Departing from this standard approach, we first show a $\mathcal{O}(\log T)$ regret for online regression with almost convex losses that satisfy QG (Quadratic Growth) condition, a generalization of the PL (Polyak-\L ojasiewicz) condition, and that have a unique minima. Although not directly applicable to wide networks since they do not have unique minima, we show that adding a suitable small random perturbation to the network predictions surprisingly makes the loss satisfy QG with unique minima. Based on such a perturbed prediction, we show a ${\mathcal{O}}(\log T)$ regret for online regression with both squared loss and KL loss, and subsequently convert these respectively to $\tilde{\mathcal{O}}(\sqrt{KT})$ and $\tilde{\mathcal{O}}(\sqrt{KL^*} + K)$ regret for NeuCB, where $L^*$ is the loss of the best policy. Separately, we also show that existing regret bounds for NeuCBs are $\Omega(T)$ or assume i.i.d. contexts, unlike this work. Finally, our experimental results on various datasets demonstrate that our algorithms, especially the one based on KL loss, persistently outperform existing algorithms

Endogenous small-noncoding RNAs and potential functions in desiccation tolerance in Physcomitrella patens

Early land plants like moss Physcomitrella patens have developed remarkable drought tolerance. Phytohormone abscisic acid (ABA) protects seeds during water stress by activating genes through transcription factors such as ABSCISIC ACID INSENSITIVE (ABI3). Small noncoding RNA (sncRNA), including microRNAs (miRNAs) and endogenous small-interfering RNAs (endo-siRNAs), are key gene regulators in eukaryotes, playing critical roles in stress tolerance in plants. Combining next-generation sequencing and computational analysis, we profiled and characterized sncRNA species from two ABI3 deletion mutants and the wild type P. patens that were subject to ABA treatment in dehydration and rehydration stages. Small RNA profiling using deep sequencing helped identify 22 novel miRNAs and 6 genomic loci producing trans-acting siRNAs (ta-siRNAs) including TAS3a to TAS3e and TAS6. Data from degradome profiling showed that ABI3 genes (ABI3a/b/c) are potentially regulated by the plant-specific miR536 and that other ABA-relevant genes are regulated by miRNAs and ta-siRNAs. We also observed broad variations of miRNAs and ta-siRNAs expression across different stages, suggesting that they could potentially influence desiccation tolerance. This study provided evidence on the potential roles of sncRNA in mediating desiccation-responsive pathways in early land plants