Improved Best-of-Both-Worlds Guarantees for Multi-Armed Bandits: FTRL with General Regularizers and Multiple Optimal Arms

We study the problem of designing adaptive multi-armed bandit algorithms that perform optimally in both the stochastic setting and the adversarial setting simultaneously (often known as a best-of-both-world guarantee). A line of recent works shows that when configured and analyzed properly, the Follow-the-Regularized-Leader (FTRL) algorithm, originally designed for the adversarial setting, can in fact optimally adapt to the stochastic setting as well. Such results, however, critically rely on an assumption that there exists one unique optimal arm. Recently, Ito (2021) took the first step to remove such an undesirable uniqueness assumption for one particular FTRL algorithm with the $\frac{1}{2}$-Tsallis entropy regularizer. In this work, we significantly improve and generalize this result, showing that uniqueness is unnecessary for FTRL with a broad family of regularizers and a new learning rate schedule. For some regularizers, our regret bounds also improve upon prior results even when uniqueness holds. We further provide an application of our results to the decoupled exploration and exploitation problem, demonstrating that our techniques are broadly applicable.Comment: Update the camera-ready version for NeurIPS 202

The contact mechanics and occurrence of edge loading in modular metal-on-polyethylene total hip replacement during daily activities

The occurrence of edge loading in hip joint replacement has been associated with many factors such as prosthetic design, component malposition and activities of daily living. The present study aimed to quantify the occurrence of edge loading/contact at the articulating surface and to evaluate the effect of cup angles and edge loading on the contact mechanics of a modular metal-on-polyethylene (MoP) total hip replacement (THR) during different daily activities. A three-dimensional finite element model was developed based on a modular MoP bearing system. Different cup inclination and anteversion angles were modelled and six daily activities were considered. The results showed that edge loading was predicted during normal walking, ascending and descending stairs activities under steep cup inclination conditions (≥55°) while no edge loading was observed during standing up, sitting down and knee bending activities. The duration of edge loading increased with increased cup inclination angles and was affected by the cup anteversion angles. Edge loading caused elevated contact pressure at the articulating surface and substantially increased equivalent plastic strain of the polyethylene liner. The present study suggested that correct positioning the component to avoid edge loading that may occur during daily activities is important for MoP THR in clinical practice

Adenosine deaminase acting on RNA 1 (ADAR1) as crucial regulators in cardiovascular diseases: structures, pathogenesis, and potential therapeutic approach

Cardiovascular diseases (CVDs) are a group of diseases that have a major impact on global health and are the leading cause of death. A large number of chemical base modifications in ribonucleic acid (RNA) are associated with cardiovascular diseases. A variety of ribonucleic acid modifications exist in cells, among which adenosine deaminase-dependent modification is one of the most common ribonucleic acid modifications. Adenosine deaminase acting on ribonucleic acid 1 (Adenosine deaminase acting on RNA 1) is a widely expressed double-stranded ribonucleic acid adenosine deaminase that forms inosine (A-to-I) by catalyzing the deamination of adenosine at specific sites of the target ribonucleic acid. In this review, we provide a comprehensive overview of the structure of Adenosine deaminase acting on RNA 1 and summarize the regulatory mechanisms of ADAR1-mediated ribonucleic acid editing in cardiovascular diseases, indicating Adenosine deaminase acting on RNA 1 as a promising therapeutic target in cardiovascular diseases

No-Regret Online Reinforcement Learning with Adversarial Losses and Transitions

Existing online learning algorithms for adversarial Markov Decision Processes achieve ${O}(\sqrt{T})$ regret after $T$ rounds of interactions even if the loss functions are chosen arbitrarily by an adversary, with the caveat that the transition function has to be fixed. This is because it has been shown that adversarial transition functions make no-regret learning impossible. Despite such impossibility results, in this work, we develop algorithms that can handle both adversarial losses and adversarial transitions, with regret increasing smoothly in the degree of maliciousness of the adversary. More concretely, we first propose an algorithm that enjoys $\widetilde{{O}}(\sqrt{T} + C^{\textsf{P}})$ regret where $C^{\textsf{P}}$ measures how adversarial the transition functions are and can be at most ${O}(T)$. While this algorithm itself requires knowledge of $C^{\textsf{P}}$, we further develop a black-box reduction approach that removes this requirement. Moreover, we also show that further refinements of the algorithm not only maintains the same regret bound, but also simultaneously adapts to easier environments (where losses are generated in a certain stochastically constrained manner as in Jin et al. [2021]) and achieves $\widetilde{{O}}(U + \sqrt{UC^{\textsf{L}}} + C^{\textsf{P}})$ regret, where $U$ is some standard gap-dependent coefficient and $C^{\textsf{L}}$ is the amount of corruption on losses.Comment: 66 page

Effects of food restriction on growth, body composition and gene expression related in regulation of lipid metabolism and food intake in grass carp

It is well known that most fish would prefer to use body lipid stores for energy expenditure when receiving a long-term food restriction. However, the mechanism of this is still not clear. In the present study, a growth experiment was carried out to investigate the effects of food restriction on growth performance, gene expression related in regulation of lipid metabolism and food ingestion in grass carp (Ctenopharyngodon idellus). Four rations, satiation (S), 80% S, 60% S and 40% S, were adopted in this study. Each treatment was randomly assigned to triplicate net cages of 15 fish (177.3 +/- 3.3 g) per cage. The experiment lasted for 49 days at 30.0 +/- 3.0 degrees C. The experimental results showed that a significant increase in feeding rate and weight gain was found in grass carp with the increased ration level. The body lipid and energy content of the grass carp exhibited a significant decrease when receiving food restriction. The transcriptional levels of the genes involved in lipogenesis (srebp-1c, fas, ppar gamma) were down-regulated at the rations of food restriction. The relative expression of hepatic fas (fatty acid synthetase) and srebp-1c (sterol regulatory element-binding protein 1c) in the fish at satiation were significantly higher than the restricted-fed groups. Similarly, the expressions of hepatic ppar. (peroxisome proliferator-activated receptor-gamma) in the fish at the ration of satiation and 80% S were significantly higher than the group at the low ration of 40% S. However, the expression of hepatic cpt-1a (carnitine palmitoyl transferase I) involved in fatty acid beta-oxidation in fish was significantly up-regulated when receiving food restriction. Other hepatic lipolysis genes of ppar alpha (peroxisome proliferators-activated receptor alpha) and hl (hepatic lipase) didn&#39;t show any significant changes in restricted-fed fish. The transcriptional levels of hepatic leptin and hypothalamus pomc (proopiomelanocortin) were significantly down-regulated in fish fed with restricted rations. But the hypothalamus npy (neuropeptide Y) and lepr (leptin receptor) had no change. The present results indicated that a long-term food restriction could cause less accumulation of lipid and could be through a way of down-regulating lipogenesis genes and up-regulating lipolysis genes. Long-term restriction could also activate the appetite of grass carp by down-regulating some anorexigenic genes. Statement of relevance: Food restriction for some time could lead to a suitable lipid storage, in case of accumulation of fatty acid profile and lipid, in cultured grass carp. (C) 2016 Elsevier B.V. All rights reserved.</p

Dynamics of quantum dissipation systems interacting with Fermion and Boson grand canonical bath ensembles: Hierarchical equations of motion approach

A hierarchical equations of motion formalism for a quantum dissipation system in a grand canonical bath ensemble surrounding is constructed, on the basis of the calculus-on-path-integral algorithm, together with the parametrization of arbitrary non-Markovin bath that satisfies fluctuation-dissipation theorem. The influence functionals for both the Fermion or Boson bath interaction are found to be of the same path-integral expression as the canonical bath, assuming they all satisfy the Gaussian statistics. However, the equation of motion formalism are different, due to the fluctuation-dissipation theories that are distinct and used explicitly. The implications of the present work to quantum transport through molecular wires and electron transfer in complex molecular systems are discussed.Comment: 12page

Enhancement of liver mitochondrial complex I and energy metabolism induced by enteritis: The key role of gut microbiota derived endotoxins

publishedVersio

Acupuncture for chronic, stable angina pectoris and an investigation of the characteristics of acupoint specificity: study protocol for a multicenter randomized controlled trial

BACKGROUND: Chronic stable angina pectoris (CSAP) is a common cardiovascular condition that endangers a patient’s life quality and longevity. As demonstrated in several clinical trials, acupuncture is attested to be effective for CSAP. Current trials are not adequate enough to provide high-quality evidence for clinical decision making, as a result of inadequate methodology design and small sample size. Notably, stark controversy toward acupoint specificity also exists in the clinical acupuncture trials for CSAP. Therefore, we designed the present study as a randomized controlled trial primarily to investigate the effectiveness of acupuncture in addition to routine care among patients with CSAP. Meanwhile, we examined whether acupoint on the disease-affected meridian (DAM) is superior to either acupoint on the non-affected meridian (NAM) or non-acupoint (NA), to further investigate the meridian-based characteristics of acupoint specificity. METHODS/DESIGN: This study was a multicenter, assessor and statistician blinded, randomized controlled trial in China. In this study, 404 participants in sum will be randomly assigned to four groups through central randomization in a 1:1:1:1 ratio. The whole study period is 20 weeks including a 4-week baseline period, a 4-week treatment period and a 12-week follow-up. Participants in the DAM group receive acupuncture stimulation at acupoints on the disease-affected meridian, and three different control groups will undergo acupuncture stimulation at the NAM, the non-acupoint and no intervention respectively, in addition to basic treatment. Participants in the acupuncture groups will receive 12 sessions of acupuncture treatment over 4 weeks, while the wait-listed (WL) group would receive free acupuncture treatment after the completion of the study. The outcome measures in this trial include the frequency of angina attack during 4 weeks as the primary outcome and eight other secondary outcomes. DISCUSSION: This trial will provide new and relatively high-quality evidence in acupuncture treatment for CSAP. Moreover, this trial may further validate the meridian-based characteristics of acupoint specificity by comparing the strength of acupoints on the disease-affected meridian versus that of the non-affected meridian, to further inspire optimization of acupuncture therapy for CSAP. TRIAL REGISTRATION: Clinical Trials.gov NCT0168623