CRIMED: Lower and Upper Bounds on Regret for Bandits with Unbounded Stochastic Corruption

We investigate the regret-minimisation problem in a multi-armed bandit setting with arbitrary corruptions. Similar to the classical setup, the agent receives rewards generated independently from the distribution of the arm chosen at each time. However, these rewards are not directly observed. Instead, with a fixed $\varepsilon\in (0,\frac{1}{2})$, the agent observes a sample from the chosen arm's distribution with probability $1-\varepsilon$, or from an arbitrary corruption distribution with probability $\varepsilon$. Importantly, we impose no assumptions on these corruption distributions, which can be unbounded. In this setting, accommodating potentially unbounded corruptions, we establish a problem-dependent lower bound on regret for a given family of arm distributions. We introduce CRIMED, an asymptotically-optimal algorithm that achieves the exact lower bound on regret for bandits with Gaussian distributions with known variance. Additionally, we provide a finite-sample analysis of CRIMED's regret performance. Notably, CRIMED can effectively handle corruptions with $\varepsilon$ values as high as $\frac{1}{2}$. Furthermore, we develop a tight concentration result for medians in the presence of arbitrary corruptions, even with $\varepsilon$ values up to $\frac{1}{2}$, which may be of independent interest. We also discuss an extension of the algorithm for handling misspecification in Gaussian model.Comment: 50 pages; 4 figure

Farm-gym: A modular reinforcement learning platform for stochastic agronomic games

International audienceWe introduce Farm-gym, an open-source farming environment written in Python, that models sequential decisionmaking in farms using Reinforcement Learning (RL). Farm-gym conceptualizes a farm as a dynamical system with many interacting entities. Leveraging a modular design, it enables us to instantiate from very simple to highly complicated environments. Contrasting many available gym environments, Farm-gym features intrinsically stochastic games, using stochastic growth models and weather data. Further, it enables to create farm games in a modular way, activating or not the entities (e.g. weeds, pests, pollinators), and yielding non-trivial coupled dynamics. Finally, every game can be customized with .yaml files for rewards, feasible actions, and initial/end-game conditions. We illustrate some interesting features on simple farms. We also showcase the challenges posed by Farm-gym to the deep RL algorithms, in order to stimulate studies in the RL community

Paramètres gouvernant la prolifération bactérienne dans les réseaux de distribution

L'étude a permis de suivre l'évolution des caractéristiques physico-chimiques et microbiologiques des eaux dans un réseau de distribution expérimental de taille industrielle, afin de comparer d'une part l'effet du chlore et de la monochloramine sur la biomasse présente dans le système à l'équilibre et d'autre part d'établir des relations quantitatives entre prolifération bactérienne, oxydant et matière organique biodégradable.Dès les premières heures de transit dans le réseau, une consommation des oxydants est constatée, avec toutefois une plus grande stabilité de la monochloramine (vitesse de consommation de 0,05 mgCl2 l-1h-1 et 0,02 mgCl2 l-1h-1 respectivement pour le chlore et la monochloramine).Même en présence d'un désinfectant résiduel, il est possible de noter une accumulation de bactéries-à ta surface des tuyaux de distribution (105 à 106 cellules. cm-2, dont environ 1 % est cultivable sur gélose) qui augmente avec la diminution de concentration du désinfectant résiduel. Les relations logarithmiques entre densité cellulaire (phase eau ou biofilm) et oxydant résiduel montrent d'une part que pour inactiver totalement les bactéries en suspension dans l'eau il convient de maintenir une chloration en continu avec un résiduel constant supérieur ou égal à 0,5 mgCl2 l-1 et, d'autre part que les chloramines sont au moins 2,5 fois moins efficaces que le chlore, même vis-à-vis des bactéries fixées.La présence de matière organique biodégradable dans les eaux explique la prolifération des bactéries dans le système de distribution. Ainsi une concentration additionnelle de 100 µg.l-1 de carbone organique dissous biodégradable (CODB) dans l'eau entrant dans le réseau de distribution occasionne en 24 heures et à 20°C une augmentation du nombre de bactéries fixées (+7,5.105 cellules.cm-2) ou en suspension (+ 4.104 cellules.ml-1) dans le réseau de distribution, à l'équilibre, déjà largement colonisé par des micro-organismes.Ainsi le contrôle de la fraction biodégradable de la matière organique apparaît toujours comme un objectif primordial.This study was carried out in order to evaluate the variations in the physicochemical characteristics of the water in an experimental distribution system.The primary objectives of the study were :- to compare the disinfectant efficiency of chlorine and monochloramine- to establish quantitative correlations among bacterial density, concentration of residual disinfectant, and concentration of biodegradable organic matter.The finished waters were obtained from a water treatment pilot plant characterized by : prechlorination (average treatment rate : 1.4 mgCl2 l-1 and residual alter sand filtration : 0.08 mgCl2 l-1), coagulation-flocculation-sedimentation (FeCl3 treatment rate : 30 to 60 mg l-1 adjusted to the raw water turbidity below 0.3 NTU), sand filtration (filtration rate : 6 h-1) and post-disinfection with chloramine (average treatment rate : 1.8 mgCl2 l-1) or chloramine (average treatment rate : 1.66 mgCl2 l-1). The concentrations of post-disinfectant used were chosen in order to maintain chlorine at 0.2 to 0.5 mgCl2 l-1 and monochloramine at 1 mgC2 l-1 after the first 24 hours residence time in the experimental distribution system.The experimental distribution system is composed of three parallel loops connected in series (31 m length, 100 mm diameter, cement lined cast iron, water velocity : 1m s-1). The configuration and operation of the system permitted a residence time of 24 hours in each loop (that is 72 hours for the whole system). Appropriate sample tap locations facilitated removal of bulk water samples. Special sampling parts also permitted sampling of cement coupons for determination of attached biofilm.The measured parameters were : residual oxidant (DPD method), DOC, BDOC (28 days of incubation at 28 °C with a bacterial inoculum), cell density in the bulk water phase (CFU ml-1) and in the biofilm (CFU cm-2) after 15 days of incubation at 20-22 °C. Total cells were enumerated using the epifluorescence direct count technique.For each experiment, all the sampling sequences were carried out on each of three days, after quasi steady-state was achieved in the system (4 to 6 weeks after starting each experiment). The data were analysed in order to characterize the treated and distributed waters; the results discussed here are based on the averages of the measured parameters tram the water samples and biofilm samples taken after the system achieved quasi steady-state.Characteristics of the treated watersThe treated waters were characterized by important variations at the DOC, BDOC and cell density. For example, the concentrations of DOC showed a seasonal variation ranging from 0.8 to 1.3 mg Cl-1 in winter to 1.6 to 2.6 mg Cl-1 in summer.The treated waters contained approximately the same concentrations of residual disinfectant, averaging 1.6 mgCl2 l-1 for chlorine and 1.5 mgCl2 l-1 for monochloramine.However, a significant difference cell density was found between the two post-disinfectants. Cell densities by the epifluorescence direct count technic were 1.6 x 103 ml-1 (0.3 % of CFU ml-1) with chlorine and 6.3 x 104 ml-1 (0.03 % of CFU ml-1) with monochloramine. The difference on cell density between the post-chlorination and the post-chloramination treatments has been observed systematically, and may be explained either as cellular lysis with chlorine or an interference when using epifluorescence counting for chlorinated waters.Characteristics of the distributed watersWhatever the season, depletion of oxidant (chlorine or monochloramine), and elimination of dissolved organic matter (DOC, BDOC) occured during the first hours of circulation of water in the distribution system. The net result was an increase in bacterial cell density.During the first hours of circulation of the waters in the distribution system, depletion of the disinfectant occured. Depletion was more rapid for chlorine (-0.05 mgCl2 l-1 h-1) titan for monochloramine (-0.02 mgCl2 l-1 h-1), winch is considered more stable titan chlorine.Even in the presence of a residual disinfectant in the distribution system, microorganisms are present in the water phase (104 to 106 cells ml-1 by epifluorescence direct count; 1 % CFU ml-1 after 15 days of incubation at 20-22 °C) and in the biofilm (105 to 106 cells cm-2 by epifluorescence direct count; 1 % CFU ml-1 after 15 days of incubation at 20-22 °C). The bacterial density increased white the disinfectant residual decreased. The apparent growth rate of the attached biomass (µfix) in loop 2 of the chlorinated distribution system (equivalent to 48 hours detention), was close to the µfix calculated for loop 2 of the chloraminated distribution system : the values were 0.0043 h-1 and 0.005 h-1 respectively.In addition, the change in the organic matter (expressed as DOC) occured in two steps :- a slight increase in DOC during the 24 first hours of residence time (loop 1), when increased residual disinfectant were present.- a bacterial consumption of DOC after 24 hours of residence time (loops 2 and 3), even in the presence of small concentrations of disinfectants.In loop 2 (48 hours residence time of the water in the system; chlorine : 0.01 mgCl2 l-1, chloramine : 0.3 mgCl2 l-1 h-1), the rates of DOC elimination averaged 13 mgCl2 l-1 h-1 and 0.42, mgCl2 l-1 h-1, respectively in the chlorinated and chloraminated distribution systems. This decrease in DOC concentrations was related to the increase in bacterial density.Relationships between cell density, oxidant and organic matterLinear relationships between the concentration of residual oxidant (chlorine or monochloramine) and the cell density in the water phase or in the biofilm show that :- whichever oxidant was used, the pipe loop sections without residual disinfectants were characterized by about 5 x 106 attached cells per cm2 (4 to 10% were able to grow on agar medium in 15 days at 20-22 °C) and by 4 x 105 planktonic cells per ml (1 % CFU ml-1);- consistently, in the sections of the system with a residual disinfectant, the bacteria (CFU and epilluorescence counting) in the water phase were more sensitive to the residual disinfectant (chlorine or monochloramine) than the bacteria attached to the pipe walls (biofilm).However, there was a difference in effectiveness between the two disinfectants; chlorine was more efficient in controlling planktonic bacteria and biofilm bacteria than was monochloramine.For example, to achieve complete inactivation of the planktonic bacteria (CFU ml-1) a constant chlorine residual of 0.5 mgCl2 l-1 was required throughout the whole distribution system, compared to 2.5 times more chloramine to achieve the same efficiency.Finally, with equivalent concentrations of residual disinfectant, the microbiological quality of the chlorinated distribution waters was better than that of the chloraminated distribution waters.From loop to loop, linear relationships between ∆DOC and cell density pointed out that the presence of biodegradable organic matter can explain the bacterial proliferation in the distribution system. For example, a concentration of BDOC as low as 0.1 mgC l-1 resulted in an increase in the cell density : an additional accumulation of 7.5 x 105 attached cells cm-2 and 4 x 104 planktonic cells ml-1 was observed in the experimental distribution system at quasi steady-state.Consequently, the control of the biodegradable organic carbon remains one of the prime objectives in order to achieve biologically stable distribution waters

Une centralisation renforcée par le chiffre ? Les effets contradictoires des indicateurs chiffrés dans la police nationale en France

Les indicateurs chiffrés ont pris une importance croissante au sein de la Police Nationale dans un contexte d’introduction du nouveau management public et de recours croissant aux nouvelles technologies. Au-delà de l’usage politique de ces chiffres, il convient de s’interroger sur leurs effets concrets sur les pratiques et des priorités de la police, en particulier sur un éventuel renforcement du caractère centralisé de l’organisation policière. Le poids grandissant des chiffres s’est traduit par une utilisation accrue de différentes bases de données comme la Main Courante Informatisée et autres tableaux de bord, par l’introduction de primes à la performance ainsi que par un renforcement des équipes chargées de collecter et de transmettre les données. La nécessité de rendre des comptes et de faire remonter l’information à tous les échelons de la hiérarchie, du commissariat au ministère, a contribué à un renforcement de la pyramide hiérarchique tout en réduisant la marge d’autonomie des agents. Ceux-ci ressentent une forme de frustration face à des demandes de plus en plus pressantes d’information et de résultats émanant de la hiérarchie. En découlent un certain nombre de stratégies d’évitement, d’adaptation, voire de tricherie, afin de fournir de bons chiffres en réponse aux objectifs assignés par le niveau supérieur, souvent suspecté de se soucier uniquement des statistiques au détriment de l’efficacité du « vrai » travail policier. Les effets centralisateurs sont ainsi contrebalancés par des pratiques centrifuges.Quantified indicators have become increasingly important within the Police nationale as a result of the introduction of new public management and the growing uses of information technologies. It is necessary to examine the specific effects of these figures on the practices and priorities of the police, especially on a possible strengthening of the centralized nature of the police organization. The growing weight of numbers has resulted in increased use of various databases such as the Main courante informatisée and other dashboards, the introduction of performance bonuses as well as a strengthening of teams in charge of data collection, transmition and analysis. The need for accountability at all levels of the hierarchy has contributed to a reinforcement of the hierarchical nature of the police while reducing the margin of autonomy of agents. The latter feel a form of frustration towards centralised requests for urgent information and results. As a result, one may observe a variety of avoidance, adaptation or gaming strategies aiming at providing good numbers and achieving the objectives of the upper level which is often suspected of worrying about statistics at the expense of the efficiency and effectiveness of “real” police work

La construction du site pédagogique numérique CHIMACTIV : analyse d'une coopération réussie entre enseignants

International audienceUn collectif d'enseignants inter-établissements s'est organisé pour concevoir et co-construire un site pédagogique numérique. L'équipe de conception initiale a fortement coopéré (au sein de chaque établissement et entre établissements) et interagi étroitement avec différents acteurs (cellules TICE, étudiants, prestataires externes) pour aboutir à une version bilingue du site. La volonté d'élargir et de diversifier le champ des utilisateurs (enseignants et étudiants) a conduit à ouvrir ce collectif à de nouveaux enseignants, afin de faire évoluer le site et compléter son contenu. Après une analyse de l'organisation mise en place, nous discuterons des obstacles à surmonter, des facteurs de réussite et du ressenti des enseignants ayant vécu cette coopération, avant de conclure sur ce qu'apporte l'aspect « numérique » des ressources développées dans la coopération entre enseignants sur la base de notre expérience

Chromosomal contacts connect loci associated with autism, BMI and head circumference phenotypes

Copy number variants (CNVs) are major contributors to genomic imbalance disorders. Phenotyping of 137 unrelated deletion and reciprocal duplication carriers of the distal 16p11.2 220 kb BP2-BP3 interval showed that these rearrangements are associated with autism spectrum disorders and mirror phenotypes of obesity/underweight and macrocephaly/microcephaly. Such phenotypes were previously associated with rearrangements of the non-overlapping proximal 16p11.2 600 kb BP4-BP5 interval. These two CNV-prone regions at 16p11.2 are reciprocally engaged in complex chromatin looping, as successfully confirmed by 4C-seq, fluorescence in situ hybridization and Hi-C, as well as coordinated expression and regulation of encompassed genes. We observed that genes differentially expressed in 16p11.2 BP4-BP5 CNV carriers are concomitantly modified in their chromatin interactions, suggesting that disruption of chromatin interplays could participate in the observed phenotypes. We also identified cis- and trans-acting chromatin contacts to other genomic regions previously associated with analogous phenotypes. For example, we uncovered that individuals with reciprocal rearrangements of the trans-contacted 2p15 locus similarly display mirror phenotypes on head circumference and weight. Our results indicate that chromosomal contacts’ maps could uncover functionally and clinically related genes.Molecular Psychiatry advance online publication, 31 May 2016; doi:10.1038/mp.2016.84