Multi-Armed Bandits for Correlated Markovian Environments with Smoothed Reward Feedback

We study a multi-armed bandit problem in a dynamic environment where arm rewards evolve in a correlated fashion according to a Markov chain. Different than much of the work on related problems, in our formulation a learning algorithm does not have access to either a priori information or observations of the state of the Markov chain and only observes smoothed reward feedback following time intervals we refer to as epochs. We demonstrate that existing methods such as UCB and $\varepsilon$-greedy can suffer linear regret in such an environment. Employing mixing-time bounds on Markov chains, we develop algorithms called EpochUCB and EpochGreedy that draw inspiration from the aforementioned methods, yet which admit sublinear regret guarantees for the problem formulation. Our proposed algorithms proceed in epochs in which an arm is played repeatedly for a number of iterations that grows linearly as a function of the number of times an arm has been played in the past. We analyze these algorithms under two types of smoothed reward feedback at the end of each epoch: a reward that is the discount-average of the discounted rewards within an epoch, and a reward that is the time-average of the rewards within an epoch.Comment: Significant revision of prior version including deeper discussion of related work, gap-independent regret bounds, and regret bounds for discounted reward

Assessing fish–fishery dynamics from a spatially explicit metapopulation perspective reveals winners and losers in fisheries management

Sustainable management of living resources must reconcile biodiversity conservation and socioeconomic viability of human activities. In the case of fisheries, sustainable management design is made challenging by the complex spatiotemporal interactions between fish and fisheries. We develop a comprehensive metapopulation framework integrating data on species life-history traits, connectivity and habitat distribution to identify priority areas for fishing regulation and assess how management impacts are spatially distributed. We trial this approach on European hake fisheries in the north-western Mediterranean, where we assess area-based management scenarios in terms of stock status and fishery productivity to prioritize areas for protection. Model simulations show that local fishery closures have the potential to enhance both spawning stock biomass and landings on a regional scale compared to a status quo scenario, but that improving protection is easier than increasing productivity. Moreover, the interaction between metapopulation dynamics and the redistribution of fishing effort following local closures implies that benefits and drawbacks are heterogeneously distributed in space, the former being concentrated in the proximity of the protected site. A network analysis shows that priority areas for protection are those with the highest connectivity (as expressed by network metrics) if the objective is to improve the spawning stock, while no significant relationship emerges between connectivity and potential for increased landings. Synthesis and applications. Our framework provides a tool for (1) assessing area-based management measures aimed at improving fisheries outcomes in terms of both conservation and socioeconomic viability and (2) describing the spatial distribution of costs and benefits, which can help guide effective management and gain stakeholder support. Adult dispersal remains the main source of uncertainty that needs to be investigated to effectively apply our model to fisheries regulation

Marine reserves: Fish life history and ecological traits matter

Copyright by the Ecological Society of AmericaMarine reserves are assumed to protect a wide range of species from deleterious effects stemming from exploitation. However, some species, due to their ecological characteristics, may not respond positively to protection. Very little is known about the effects of life history and ecological traits (e.g., mobility, growth, and habitat) on responses of fish species to marine reserves. Using 40 data sets from 12 European marine reserves, we show that there is significant variation in the response of different species of fish to protection and that this heterogeneity can be explained, in part, by differences in their traits. Densities of targeted size-classes of commercial species were greater in protected than unprotected areas. This effect of protection increased as the maximum body size of the targeted species increased, and it was greater for species that were not obligate schoolers. However, contrary to previous theoretical findings, even mobile species with wide home ranges benefited from protection: the effect of protection was at least as strong for mobile species as it was for sedentary ones. Noncommercial bycatch and unexploited species rarely responded to protection, and when they did (in the case of unexploited bentho-pelagic species), they exhibited the opposite response: their densities were lower inside reserves. The use of marine reserves for marine conservation and fisheries management implies that they should ensure protection for a wide range of species with different life-history and ecological traits. Our results suggest this is not the case, and instead that effects vary with economic value, body size, habitat, depth range, and schooling behavior.Publicado

Fishers' behaviour in response to the implementation of a marine protected area

Marine Protected Areas (MPAs) have been widely proposed as a fisheries management tool in addition to their conservation purposes. Despite this, few studies have satisfactorily assessed the dynamics of fishers' adaptations to the loss of fishing grounds. Here we used data from before, during and after the implementation of the management plan of a temperate Atlantic multiple-use MPA to examine the factors affecting the spatial and temporal distribution of different gears used by the artisanal fishing fleet. The position of vessels and gear types were obtained by visual surveys and related to spatial features of the marine park. A hotspot analysis was conducted to identify heavily utilized patches for each fishing gear and time period. The contribution of individual vessels to each significant cluster was assessed to better understand fishers' choices. Different fisheries responded differently to the implementation of protection measures, with preferred habitats of target species driving much of the fishers' choices. Within each fishery, individual fishers showed distinct strategies with some operating in a broader area whereas others kept preferred territories. Our findings are based on reliable methods that can easily be applied in coastal multipurpose MPAs to monitor and assess fisheries and fishers responses to different management rules and protection levels. This paper is the first in-depth empirical study where fishers' choices from artisanal fisheries were analysed before, during and after the implementation of a MPA, thereby allowing a clearer understanding of the dynamics of local fisheries and providing significant lessons for marine conservation and management of coastal systems

Human Stressors Are Driving Coastal Benthic Long-Lived Sessile Fan Mussel Pinna nobilis Population Structure More than Environmental Stressors.

Coastal degradation and habitat disruption are severely compromising sessile marine species. The fan shell Pinna nobilis is an endemic, vulnerable species and the largest bivalve in the Mediterranean basin. In spite of species legal protection, fan shell populations are declining. Models analyzed the contributions of environmental (mean depth, wave height, maximum wave height, period of waves with high energy and mean direction of wave source) versus human-derived stressors (anchoring, protection status, sewage effluents, fishing activity and diving) as explanatory variables depicting Pinna nobilis populations at a mesoscale level. Human stressors were explaining most of the variability in density spatial distribution of fan shell, significantly disturbing benthic communities. Habitat protection affected P. nobilis structure and physical aggression by anchoring reveals a high impact on densities. Environmental variables instead played a secondary role, indicating that global change processes are not so relevant in coastal benthic communities as human-derived impacts.Versión del editor4,411

Single-Pair FRET Microscopy Reveals Mononucleosome Dynamics

We applied spFRET microscopy for direct observation of intranucleosomal DNA dynamics. Mononucleosomes, reconstituted with DNA containing a FRET pair at the dyad axis and exit of the nucleosome core particle, were immobilized through a 30 bp DNA tether on a polyethyleneglycol functionalized slide and visualized using Total Internal Reflection Fluorescence microscopy. FRET efficiency time-traces revealed two types of dynamics: acceptor blinking and intramolecular rearrangements. Both Cy5 and ATTO647N acceptor dyes showed severe blinking in a deoxygenated buffer in the presence of 2% βME. Replacing the triplet quencher βME with 1 mM Trolox eliminated most blinking effects. After suppression of blinking three subpopulations were observed: 90% appeared as dissociated complexes; the remaining 10% featured an average FRET efficiency in agreement with intact nucleosomes. In 97% of these intact nucleosomes no significant changes in FRET efficiency were observed in the experimentally accessible time window ranging from 10 ms to 10’s of seconds. However, 3% of the intact nucleosomes showed intervals with reduced FRET efficiency, clearly distinct from blinking, with a lifetime of 120 ms. These fluctuations can unambiguously be attributed to DNA breathing. Our findings illustrate not only the merits but also typical caveats encountered in single-molecule FRET studies on complex biological systems

Spearfishing Regulation Benefits Artisanal Fisheries: The ReGS Indicator and Its Application to a Multiple-Use Mediterranean Marine Protected Area

The development of fishing efficiency coupled with an increase of fishing effort led to the overexploitation of numerous natural marine resources. In addition to this commercial pressure, the impact of recreational activities on fish assemblages remains barely known. Here we examined the impact of spearfishing limitation on resources in a marine protected area (MPA) and the benefit it provides for the local artisanal fishery through the use of a novel indicator. We analysed trends in the fish assemblage composition using artisanal fisheries data collected in the Bonifacio Strait Natural Reserve (BSNR), a Mediterranean MPA where the spearfishing activity has been forbidden over 15% of its area. Fish species were pooled into three response groups according to their target level by spearfishing. We developed the new flexible ReGS indicator reflecting shifts in species assemblages according to the relative abundance of each response group facing external pressure. The catch per unit effort (CPUE) increased by ca. 60% in the BSNR between 2000 and 2007, while the MPA was established in 1999. The gain of CPUE strongly depended on the considered response group: for the highly targeted group, the CPUE doubled while the CPUE of the untargeted group increased by only 15.5%. The ReGS value significantly increased from 0.31 to 0.45 (on a scale between 0 and 1) in the general perimeter of this MPA while it has reached a threshold of 0.43, considered as a reference point, in the area protected from spearfishing since 1982. Our results demonstrated that limiting recreational fishing by appropriate zoning in multiple-use MPAs represents a real benefit for artisanal fisheries. More generally we showed how our new indicator may reveal a wide range of impacts on coastal ecosystems such as global change or habitat degradation

Selecting Indicator Portfolios for Marine Species and Food Webs: A Puget Sound Case Study

Ecosystem-based management (EBM) has emerged as a promising approach for maintaining the benefits humans want and need from the ocean, yet concrete approaches for implementing EBM remain scarce. A key challenge lies in the development of indicators that can provide useful information on ecosystem status and trends, and assess progress towards management goals. In this paper, we describe a generalized framework for the methodical and transparent selection of ecosystem indicators. We apply the framework to the second largest estuary in the United States – Puget Sound, Washington – where one of the most advanced EBM processes is currently underway. Rather than introduce a new method, this paper integrates a variety of familiar approaches into one step-by-step approach that will lead to more consistent and reliable reporting on ecosystem condition. Importantly, we demonstrate how a framework linking indicators to policy goals, as well as a clearly defined indicator evaluation and scoring process, can result in a portfolio of useful and complementary indicators based on the needs of different users (e.g., policy makers and scientists). Although the set of indicators described in this paper is specific to marine species and food webs, we provide a general approach that could be applied to any set of management objectives or ecological system

Les nouveaux accidents domestiques

International audienc