Multi-Armed Bandits for Correlated Markovian Environments with Smoothed Reward Feedback

We study a multi-armed bandit problem in a dynamic environment where arm rewards evolve in a correlated fashion according to a Markov chain. Different than much of the work on related problems, in our formulation a learning algorithm does not have access to either a priori information or observations of the state of the Markov chain and only observes smoothed reward feedback following time intervals we refer to as epochs. We demonstrate that existing methods such as UCB and $\varepsilon$-greedy can suffer linear regret in such an environment. Employing mixing-time bounds on Markov chains, we develop algorithms called EpochUCB and EpochGreedy that draw inspiration from the aforementioned methods, yet which admit sublinear regret guarantees for the problem formulation. Our proposed algorithms proceed in epochs in which an arm is played repeatedly for a number of iterations that grows linearly as a function of the number of times an arm has been played in the past. We analyze these algorithms under two types of smoothed reward feedback at the end of each epoch: a reward that is the discount-average of the discounted rewards within an epoch, and a reward that is the time-average of the rewards within an epoch.Comment: Significant revision of prior version including deeper discussion of related work, gap-independent regret bounds, and regret bounds for discounted reward

Assessing fish–fishery dynamics from a spatially explicit metapopulation perspective reveals winners and losers in fisheries management

Sustainable management of living resources must reconcile biodiversity conservation and socioeconomic viability of human activities. In the case of fisheries, sustainable management design is made challenging by the complex spatiotemporal interactions between fish and fisheries. We develop a comprehensive metapopulation framework integrating data on species life-history traits, connectivity and habitat distribution to identify priority areas for fishing regulation and assess how management impacts are spatially distributed. We trial this approach on European hake fisheries in the north-western Mediterranean, where we assess area-based management scenarios in terms of stock status and fishery productivity to prioritize areas for protection. Model simulations show that local fishery closures have the potential to enhance both spawning stock biomass and landings on a regional scale compared to a status quo scenario, but that improving protection is easier than increasing productivity. Moreover, the interaction between metapopulation dynamics and the redistribution of fishing effort following local closures implies that benefits and drawbacks are heterogeneously distributed in space, the former being concentrated in the proximity of the protected site. A network analysis shows that priority areas for protection are those with the highest connectivity (as expressed by network metrics) if the objective is to improve the spawning stock, while no significant relationship emerges between connectivity and potential for increased landings. Synthesis and applications. Our framework provides a tool for (1) assessing area-based management measures aimed at improving fisheries outcomes in terms of both conservation and socioeconomic viability and (2) describing the spatial distribution of costs and benefits, which can help guide effective management and gain stakeholder support. Adult dispersal remains the main source of uncertainty that needs to be investigated to effectively apply our model to fisheries regulation

Making protected areas effective for biodiversity, climate and food

The spatial extent of marine and terrestrial protected areas (PAs) was among the most intensely debated issues prior to the decision about the post-2020 Global Biodiversity Framework (GBF) of the Convention on Biological Diversity. Positive impacts of PAs on habitats, species diversity and abundance are well documented. Yet, biodiversity loss continues unabated despite efforts to protect 17% of land and 10% of the oceans by 2020. This casts doubt on whether extending PAs to 30%, the agreed target in the Kunming-Montreal GBF, will indeed achieve meaningful biodiversity benefits. Critically, the focus on area coverage obscures the importance of PA effectiveness and overlooks concerns about the impact of PAs on other sustainability objectives. We propose a simple means of assessing and visualising the complex relationships between PA area coverage and effectiveness and their effects on biodiversity conservation, nature-based climate mitigation and food production. Our analysis illustrates how achieving a 30% PA global target could be beneficial for biodiversity and climate. It also highlights important caveats: (i) achieving lofty area coverage objectives alone will be of little benefit without concomitant improvements in effectiveness, (ii) trade-offs with food production particularly for high levels of coverage and effectiveness are likely and (iii) important differences in terrestrial and marine systems need to be recognized when setting and implementing PA targets. The CBD's call for a significant increase in PA will need to be accompanied by clear PA effectiveness goals to reduce and revert dangerous anthropogenic impacts on socio-ecological systems and biodiversity

Commissioning of the cryogenics of the LHC long straight sections

The LHC is made of eight circular arcs interspaced with eight Long Straight Sections (LSS). Most powering interfaces to the LHC are located in these sections where the particle beams are focused and shaped for collision, cleaning and acceleration. The LSSs are constituted of several unique cryogenic devices and systems like electrical feed-boxes, standalone superconducting magnets, superconducting links, RF cavities and final focusing superconducting magnets. This paper presents the cryogenic commissioning and the main results obtained during the first operation of the LHC Long Straight Sections.Comment: 8 pp. Cryogenic Engineering Conference and International Cryogenic Materials Conference, 28 Jun - 2 Jul 2009. Tucson, Arizon

“A plea for a renaissance”:Dorothy Todd’s Modernist experiment in British Vogue, 1922 -1926

This is not a fashion paper: Modernism, Dorothy Todd and British Vogue "Style is thinking." In 1922, six years after its initial inception in England, Vogue magazine began to be edited by Dorothy Todd. Her spell in charge of the already renowned magazine, which had begun its life in America in 1892, lasted until only 1926. These years represent somewhat of an anomaly in the flawless history of the world's most famous fashion magazine, and study of the editions from this era reveal a Vogue that few would expect. Dorothy Todd, the most enigmatic and undocumented figure in the history of the magazine and, arguably within the sphere of popular publications in general, used Vogue as the vehicle through which to promote the innovative forms in art and literature that were emerging at the beginning of the twentieth century. Through her inclusion of artists and writers whom we would now consider to be the influential makers of modernism, Todd turned Vogue into an advanced literary and social review and thus a magazine of modernism. Preconceptions which regard Vogue as a mere mass circulated fashion glossy need necessarily be dismissed before reading this work, as the Vogue of 1922-1926 presented the fashions of the body alongside the “fashions of the mind"2 This research will demonstrate both the extent of Vogue's transformation into a modernist magazine and to seek to locate the lost editor of Dorothy Todd. Such a meticulous project has never yet to be undertaken. Dorothy Todd's Vogue can be no longer dismissed as mere frivolity in the frenzied and tumultuous intellectual climate of the inter-war period

Baseline Configuration of the Cryogenic System for the International Linear Collider

The paper discusses the main constraints and boundary conditions and describes the baseline configuration of the International Linear Collider (ILC) cryogenic system. The cryogenic layout, architecture and the cooling principle are presented. The paper addresses a plan for study and development required to demonstrate and improve the performance, to reduce cost and to attain the desired reliability

Design and manufacturing of complex optics: the dragonfly eye optic.

The ''Design and Manufacturing of Complex Optics'' LDRD sought to develop new advanced methods for the design and manufacturing of very complex optical systems. The project team developed methods for including manufacturability into optical designs and also researched extensions of manufacturing techniques to meet the challenging needs of aspherical, 3D, multi-level lenslet arrays on non-planar surfaces. In order to confirm the applicability of the developed techniques, the team chose the Dragonfly Eye optic as a testbed. This optic has arrays of aspherical micro-lenslets on both the exterior and the interior of a 4mm diameter hemispherical shell. Manufacturing of the dragonfly eye required new methods of plunge milling aspherical optics and the development of a method to create the milling tools using focused ion beam milling. The team showed the ability to create aspherical concave milling tools which will have great significance to the optical industry. A prototype dragonfly eye exterior was created during the research, and the methods of including manufacturability in the optical design process were shown to be successful as well

Fishers' behaviour in response to the implementation of a marine protected area

Marine Protected Areas (MPAs) have been widely proposed as a fisheries management tool in addition to their conservation purposes. Despite this, few studies have satisfactorily assessed the dynamics of fishers' adaptations to the loss of fishing grounds. Here we used data from before, during and after the implementation of the management plan of a temperate Atlantic multiple-use MPA to examine the factors affecting the spatial and temporal distribution of different gears used by the artisanal fishing fleet. The position of vessels and gear types were obtained by visual surveys and related to spatial features of the marine park. A hotspot analysis was conducted to identify heavily utilized patches for each fishing gear and time period. The contribution of individual vessels to each significant cluster was assessed to better understand fishers' choices. Different fisheries responded differently to the implementation of protection measures, with preferred habitats of target species driving much of the fishers' choices. Within each fishery, individual fishers showed distinct strategies with some operating in a broader area whereas others kept preferred territories. Our findings are based on reliable methods that can easily be applied in coastal multipurpose MPAs to monitor and assess fisheries and fishers responses to different management rules and protection levels. This paper is the first in-depth empirical study where fishers' choices from artisanal fisheries were analysed before, during and after the implementation of a MPA, thereby allowing a clearer understanding of the dynamics of local fisheries and providing significant lessons for marine conservation and management of coastal systems