The Sliding Regret in Stochastic Bandits: Discriminating Index and Randomized Policies

This paper studies the one-shot behavior of no-regret algorithms for stochastic bandits. Although many algorithms are known to be asymptotically optimal with respect to the expected regret, over a single run, their pseudo-regret seems to follow one of two tendencies: it is either smooth or bumpy. To measure this tendency, we introduce a new notion: the sliding regret, that measures the worst pseudo-regret over a time-window of fixed length sliding to infinity. We show that randomized methods (e.g. Thompson Sampling and MED) have optimal sliding regret, while index policies, although possibly asymptotically optimal for the expected regret, have the worst possible sliding regret under regularity conditions on their index (e.g. UCB, UCB-V, KL-UCB, MOSS, IMED etc.). We further analyze the average bumpiness of the pseudo-regret of index policies via the regret of exploration, that we show to be suboptimal as well.Comment: 31 page

A renewal proposal for the North Slope of Beacon Hill

Thesis (M.C.P.)--Massachusetts Institute of Technology, Dept. of City Planning, 1960.Includes bibliographical references (leaves 83-84).by Victor Boone Powell.M.C.P

Aerobic interval training impacts muscle and brain oxygenation responses to incremental exercise

The purpose of the present study was to assess the effects of aerobic interval training on muscle and brain oxygenation to incremental ramp exercise. Eleven physically active subjects performed a 6-week interval training period, proceeded and followed by an incremental ramp exercise to exhaustion (25 W min\u20131). Throughout the tests pulmonary gas exchange and muscle (Vastus Lateralis) and brain (prefrontal cortex) oxygenation [concentration of deoxygenated and oxygenated hemoglobin, HHb and O2Hb, and tissue oxygenation index (TOI)] were continuously recorded. Following the training intervention V.O2peak had increased with 7.8 \ub1 5.0% (P < 0.001). The slope of the decrease in muscle TOI had decreased (P = 0.017) 16.6 \ub1 6.4% and the amplitude of muscle HHb and totHb had increased (P < 0.001) 40.4 \ub1 15.8 and 125.3 \ub1 43.1%, respectively. The amplitude of brain O2Hb and totHb had increased (P < 0.05) 40.1 \ub1 18.7 and 26.8 \ub1 13.6%, respectively. The training intervention shifted breakpoints in muscle HHb, totHb and TOI, and brain O2Hb, HHb, totHb and TOI to a higher absolute work rate and V.O2 (P < 0.05). The relative (in %) change in V.O2peak was significantly correlated to relative (in %) change slope of muscle TOI (r = 0.69, P = 0.011) and amplitude of muscle HHb (r = 0.72, P = 0.003) and totHb (r = 0.52, P = 0.021), but not to changes in brain oxygenation. These results indicate that interval training affects both muscle and brain oxygenation, coinciding with an increase in aerobic fitness (i.e., V.O2peak). The relation between the change in V.O2peak and muscle but not brain oxygenation suggests that brain oxygenation per se is not a primary factor limiting exercise tolerance during incremental exercise

Guidelines for the use and interpretation of assays for monitoring autophagy (4th edition)1.

In 2008, we published the first set of guidelines for standardizing research in autophagy. Since then, this topic has received increasing attention, and many scientists have entered the field. Our knowledge base and relevant new technologies have also been expanding. Thus, it is important to formulate on a regular basis updated guidelines for monitoring autophagy in different organisms. Despite numerous reviews, there continues to be confusion regarding acceptable methods to evaluate autophagy, especially in multicellular eukaryotes. Here, we present a set of guidelines for investigators to select and interpret methods to examine autophagy and related processes, and for reviewers to provide realistic and reasonable critiques of reports that are focused on these processes. These guidelines are not meant to be a dogmatic set of rules, because the appropriateness of any assay largely depends on the question being asked and the system being used. Moreover, no individual assay is perfect for every situation, calling for the use of multiple techniques to properly monitor autophagy in each experimental setting. Finally, several core components of the autophagy machinery have been implicated in distinct autophagic processes (canonical and noncanonical autophagy), implying that genetic approaches to block autophagy should rely on targeting two or more autophagy-related genes that ideally participate in distinct steps of the pathway. Along similar lines, because multiple proteins involved in autophagy also regulate other cellular pathways including apoptosis, not all of them can be used as a specific marker for bona fide autophagic responses. Here, we critically discuss current methods of assessing autophagy and the information they can, or cannot, provide. Our ultimate goal is to encourage intellectual and technical innovation in the field

Precision mouse models with expanded tropism for human pathogens

A major limitation of current humanized mouse models is that they primarily enable the analysis of human-specific pathogens that infect hematopoietic cells. However, most human pathogens target other cell types, including epithelial, endothelial and mesenchymal cells. Here, we show that implantation of human lung tissue, which contains up to 40 cell types, including nonhematopoietic cells, into immunodeficient mice (lung-only mice) resulted in the development of a highly vascularized lung implant. We demonstrate that emerging and clinically relevant human pathogens such as Middle East respiratory syndrome coronavirus, Zika virus, respiratory syncytial virus and cytomegalovirus replicate in vivo in these lung implants. When incorporated into bone marrow/liver/thymus humanized mice, lung implants are repopulated with autologous human hematopoietic cells. We show robust antigen-specific humoral and T-cell responses following cytomegalovirus infection that control virus replication. Lung-only mice and bone marrow/liver/thymus-lung humanized mice substantially increase the number of human pathogens that can be studied in vivo, facilitating the in vivo testing of therapeutics

When do discounted-optimal policies also optimize the gain?

In this technical note, we establish an upper-bound on the threshold on the discount factor starting from which all discounted-optimal deterministic policies are gain-optimal, that we prove to be tight on an example. To address computability issues of that theoretical threshold, we provide a weaker bound which is tractable on ergodic MDPs in polynomial time