Stability-penalty-adaptive Follow-the-regularized-leader: Sparsity, Game-dependency, and Best-of-both-worlds

Adaptivity to the difficulties of a problem is a key property in sequential decision-making problems to broaden the applicability of algorithms. Follow-the-Regularized-Leader (FTRL) has recently emerged as one of the most promising approaches for obtaining various types of adaptivity in bandit problems. Aiming to further generalize this adaptivity, we develop a generic adaptive learning rate, called Stability-Penalty-Adaptive (SPA) learning rate for FTRL. This learning rate yields a regret bound jointly depending on stability and penalty of the algorithm, into which the regret of FTRL is typically decomposed. With this result, we establish several algorithms with three types of adaptivity: sparsity, game-dependency, and Best-of-Both-Worlds (BOBW). Sparsity frequently appears in real-world problems. However, existing sparse multi-armed bandit algorithms with $k$-arms assume that the sparsity level $s \leq k$ is known in advance, which is often not the case in real-world scenarios. To address this problem, with the help of the new learning rate framework, we establish $s$-agnostic algorithms with regret bounds of $\tilde{O}(\sqrt{sT})$ in the adversarial regime for $T$ rounds, which matches the existing lower bound up to a logarithmic factor. Meanwhile, BOBW algorithms aim to achieve a near-optimal regret in both the stochastic and adversarial regimes. Leveraging the new adaptive learning rate framework and a novel analysis to bound the variation in FTRL output in response to changes in a regularizer, we establish the first BOBW algorithm with a sparsity-dependent bound. Additionally, we explore partial monitoring and demonstrate that the proposed learning rate framework allows us to achieve a game-dependent bound and the BOBW simultaneously.Comment: 30 page

Best-of-Both-Worlds Algorithms for Partial Monitoring

This study considers the partial monitoring problem with $k$-actions and $d$-outcomes and provides the first best-of-both-worlds algorithms, whose regrets are favorably bounded both in the stochastic and adversarial regimes. In particular, we show that for non-degenerate locally observable games, the regret is $O(m^2 k^4 \log(T) \log(k_{\Pi} T) / \Delta_{\min})$ in the stochastic regime and $O(m k^{2/3} \sqrt{T \log(T) \log k_{\Pi}})$ in the adversarial regime, where $T$ is the number of rounds, $m$ is the maximum number of distinct observations per action, $\Delta_{\min}$ is the minimum suboptimality gap, and $k_{\Pi}$ is the number of Pareto optimal actions. Moreover, we show that for globally observable games, the regret is $O(c_{\mathcal{G}}^2 \log(T) \log(k_{\Pi} T) / \Delta_{\min}^2)$ in the stochastic regime and $O((c_{\mathcal{G}}^2 \log(T) \log(k_{\Pi} T))^{1/3} T^{2/3})$ in the adversarial regime, where $c_{\mathcal{G}}$ is a game-dependent constant. We also provide regret bounds for a stochastic regime with adversarial corruptions. Our algorithms are based on the follow-the-regularized-leader framework and are inspired by the approach of exploration by optimization and the adaptive learning rate in the field of online learning with feedback graphs.Comment: 31 page

Adaptive Learning Rate for Follow-the-Regularized-Leader: Competitive Analysis and Best-of-Both-Worlds

Follow-The-Regularized-Leader (FTRL) is known as an effective and versatile approach in online learning, where appropriate choice of the learning rate is crucial for smaller regret. To this end, we formulate the problem of adjusting FTRL's learning rate as a sequential decision-making problem and introduce the framework of competitive analysis. We establish a lower bound for the competitive ratio and propose update rules for learning rate that achieves an upper bound within a constant factor of this lower bound. Specifically, we illustrate that the optimal competitive ratio is characterized by the (approximate) monotonicity of components of the penalty term, showing that a constant competitive ratio is achievable if the components of the penalty term form a monotonically non-increasing sequence, and derive a tight competitive ratio when penalty terms are $\xi$-approximately monotone non-increasing. Our proposed update rule, referred to as \textit{stability-penalty matching}, also facilitates constructing the Best-Of-Both-Worlds (BOBW) algorithms for stochastic and adversarial environments. In these environments our result contributes to achieve tighter regret bound and broaden the applicability of algorithms for various settings such as multi-armed bandits, graph bandits, linear bandits, and contextual bandits

Novel insights into the ontogeny of basophils

Basophils are the least common granulocytes, accounting for <1% of peripheral blood leukocytes. In the last 20 years, analytical tools for mouse basophils have been developed, and we now recognize that basophils play critical roles in various immune reactions, including the development of allergic inflammation and protective immunity against parasites. Moreover, the combined use of flow cytometric analyses and knockout mice has uncovered several progenitor cells committed to basophils in mice. Recently, advancements in single-cell RNA sequencing (scRNA-seq) technologies have challenged the classical view of the differentiation of various hematopoietic cell lineages. This is also true for basophil differentiation, and studies using scRNA-seq analysis have provided novel insights into basophil differentiation, including the association of basophil differentiation with that of erythrocyte/megakaryocyte and the discovery of novel basophil progenitor cells in the mouse bone marrow. In this review, we summarize the recent findings of basophil ontogeny in both mice and humans, mainly focusing on studies using scRNA-seq analyses

Synthesis of donor-acceptor chromophores by the [2+2] cycloaddition of arylethynyl-2H-cyclohepta[b]furan-2-ones with 7,7,8,8-tetracyanoquinodimethane

Arylethynyl-2H-cyclohepta[b]furan-2-ones reacted with 7,7,8,8-tetracyanoquinodimethane (TCNQ) in a formal [2 + 2] cycloaddition reaction, followed by ring opening of the initially formed cyclobutene derivatives, to afford the corresponding dicyanoquinodimethane (DCNQ) chromophores in excellent yields. The intramolecular charge-transfer (ICT) interactions between the 2H-cyclohepta[b]furan-2-one ring and DCNQ acceptor moiety were investigated by UV/Vis spectroscopy and theoretical calculations. The redox behavior of the novel DCNQ derivatives was examined by cyclic voltammetry (CV) and differential pulse voltammetry (DPV), which revealed their multistep electrochemical reduction properties depended on the number of DCNQ units in the molecule. Moreover, a significant color change was observed by visible spectroscopy under electrochemical reduction conditions.ArticleORGANIC & BIOMOLECULAR CHEMISTRY. 10(12):2431-2438 (2012)journal articl

Nephrin is an important component of the barrier system in the testis.

Nephrin, a gene product of the congenital nephrotic syndrome of the Finnish type (NPHS1), is a 1242-residue putative transmembrane protein of the immunoglobulin family of cell adhesion molecules. The expression of this gene is localized in rat and human glomerular epithelial cells. Here we report the expression of nephrin in various tissues other than the kidneys in mice. The expression of nephrin mRNA in various tissues of mice, including the kidneys, testes, spleen, thymus and brain, were first investigated by the RT-PCR method, and it was shown that a high level of nephrin mRNA could be detected in the testes of mice 1-6 weeks old. In situ hybridization revealed the expression of the nephrin gene in the Sertoli cells. Additionally, immunofluorescent staining studies indicated that nephrin was colocalized with anchoring protein ZO-1 in the mouse testis. From these results, it is inferred that nephrin is an important component of the barrier system in testes.</p

Synthesis of 2-Aryl- and 6-Heteroaryl-1,3-di(4-pyridyl)azulenes by Katritzky’s Pyridylation of 2-Aryl- and 6-Heteroarylazulenes

Preparation of 2-aryl- and 6-heteroarylazulenes 2-6 and 8-10 was established by the palladium-catalyzed cross-coupling reaction of the corresponding haloazulenes with lithium aryl- and heteroarylmagnesium ate complexes, which were readily prepared from the corresponding aryl and heteroaryl halides. The reaction of 2-6 and 8-10, except for 3 and 8, with pyridine in the presence of Tf2O, followed by treatment with KOH in MeOH afforded the corresponding 2-aryl- and 6-heteroary1-1,3-di(4-pyridyl)azulenes 11-16 in good yields.ArticleHETEROCYCLES. 89(11):2588-2603 (2014)journal articl