106 research outputs found
Flows and Decompositions of Games: Harmonic and Potential Games
In this paper we introduce a novel flow representation for finite games in
strategic form. This representation allows us to develop a canonical direct sum
decomposition of an arbitrary game into three components, which we refer to as
the potential, harmonic and nonstrategic components. We analyze natural classes
of games that are induced by this decomposition, and in particular, focus on
games with no harmonic component and games with no potential component. We show
that the first class corresponds to the well-known potential games. We refer to
the second class of games as harmonic games, and study the structural and
equilibrium properties of this new class of games. Intuitively, the potential
component of a game captures interactions that can equivalently be represented
as a common interest game, while the harmonic part represents the conflicts
between the interests of the players. We make this intuition precise, by
studying the properties of these two classes, and show that indeed they have
quite distinct and remarkable characteristics. For instance, while finite
potential games always have pure Nash equilibria, harmonic games generically
never do. Moreover, we show that the nonstrategic component does not affect the
equilibria of a game, but plays a fundamental role in their efficiency
properties, thus decoupling the location of equilibria and their payoff-related
properties. Exploiting the properties of the decomposition framework, we obtain
explicit expressions for the projections of games onto the subspaces of
potential and harmonic games. This enables an extension of the properties of
potential and harmonic games to "nearby" games. We exemplify this point by
showing that the set of approximate equilibria of an arbitrary game can be
characterized through the equilibria of its projection onto the set of
potential games
Early Brain Activity Relates to Subsequent Brain Growth in Premature Infants
Recent experimental studies have shown that early brain activity is crucial for neuronal survival and the development of brain networks; however, it has been challenging to assess its role in the developing human brain. We employed serial quantitative magnetic resonance imaging to measure the rate of growth in circumscribed brain tissues from preterm to term age, and compared it with measures of electroencephalographic (EEG) activity during the first postnatal days by 2 different methods. EEG metrics of functional activity were computed: EEG signal peak-to-peak amplitude and the occurrence of developmentally important spontaneous activity transients (SATs). We found that an increased brain activity in the first postnatal days correlates with a faster growth of brain structures during subsequent months until term age. Total brain volume, and in particular subcortical gray matter volume, grew faster in babies with less cortical electrical quiescence and with more SAT events. The present findings are compatible with the idea that (1) early cortical network activity is important for brain growth, and that (2) objective measures may be devised to follow early human brain activity in a biologically reasoned way in future research as well as during intensive care treatmen
Basis Expansion in Natural Actor Critic Methods
International audienceIn reinforcement learning, the aim of the agent is to find a policy that maximizes its expected return. Policy gradient methods try to accomplish this goal by directly approximating the policy using a parametric function approximator; the expected return of the current policy is estimated and its parameters are updated by steepest ascent in the direction of the gradient of the expected return with respect to the policy parameters. In general, the policy is defined in terms of a set of basis functions that capture important features of the problem. Since the quality of the resulting policies directly depend on the set of basis func- tions, and defining them gets harder as the complexity of the problem increases, it is important to be able to find them automatically. In this paper, we propose a new approach which uses cascade-correlation learn- ing architecture for automatically constructing a set of basis functions within the context of Natural Actor-Critic (NAC) algorithms. Such basis functions allow more complex policies be represented, and consequently improve the performance of the resulting policies. We also present the effectiveness of the method empirically
Effect of heat stress on LPS-induced febrile response in D-galactosamine-sensitized rats
. In the present study we have tested whether inhibition of protein synthesis in the liver can reduce the effect of this heat conditioning on the LPS-induced febrile response in the rat. D-galactosamine (D-gal) was used to selectively inhibit liver protein synthesis. D-gal (500 mg/kg) or PBS as control was administered intraperitoneally 1 h before heat stress. LPS (50 g/kg ip) was injected 24 h post-heat exposure. Treatment with D-gal blunted the febrile response to LPS. Moreover, heat-conditioned rats treated first with D-gal and subsequently with LPS demonstrated a profound fall in core temperature 10-18 h post-LPS. A significant increase of serum TNF-⣠accompanied this effect of D-gal on fever. Heat-conditioned animals receiving D-gal showed an inhibition in inducible HSP-70 in the liver. These data support the role of hepatic function in modulating the febrile response to LPS. heat shock proteins; liver; heart; kidney; tumor necrosis factor-âŁ, interleukin-6, temperature regulation; fever; lipopolysaccharide HEAT STRESS PROVOKES metabolic adaptations in the whole organism. One such response is the production of heat shock proteins (HSPs) (26). The accumulation of HSPs within cells helps both cells and the whole organism survive subsequent, otherwise lethal, thermal stress. Interestingly, heat conditioning sufficient to cause cellular HSP accumulation has also been shown to be protective in a subsequent, otherwise lethal, endotoxin challenge (30). Several studies have demonstrated that HSPs regulate cytokine production in peripheral blood monocytes. Intracellular HSP accumulation is associated with a decrease in synthesis of tumor necrosis factor-⣠(TNF-âŁ) and interleukin (IL)-1†(6, 32). Impaired HSP production causes enhanced TNF-induced cytotoxicity in cells Whereas heat conditioning is protective, pretreatment with D-galactosamine (D-gal) increases sensitivity to subsequent LPS (2, 10). D-gal inhibits protein synthesis primarily in the live
Tackling bovine TB
On 18 December Defra revealed that during 2018, 32,601 badgers were killed, bringing the total number slaughtered under licence since 2013 to almost 67,000.1
âEffectivenessâ claims relate not to the impact on cattle TB, but rather to the ability of the contracted shooters to kill sufficient badgers to satisfy their licence requirements, which they can hardly fail to reach given that target numbers are âadjustedâ by Natural England part-way through the culls according to the contractorsâ progress.
Sixty per cent of the badgers have been killed by âcontrolled shootingâ, a method rejected by both the governmentâs Independent Expert Panel2 and the BVA3 because of animal welfare concerns. During 2018 Natural England directly monitored just 89 (0.43 per cent) of controlled shooting events.
It is deplorable that the chief veterinary officer (CVO) continues to support the roll-out of a policy that permits controlled shooting, when veterinary organisations have condemned the method on animal welfare grounds. It is particularly concerning that the CVO appears to have deflected responsibility for humaneness to Natural Englandâs chief scientist who, as far as we are aware, has no background in animal welfare science.
It is also unacceptable for government to attribute reductions in herd bovine TB (bTB) incidents over the first four years of culling in the original âpilotâ cull zones to its badger culling policy.4 Independent analysis of this and more recent data from the Gloucestershire pilot cull zone5 indicate that new herd incidence is rising sharply, with 22 herd breakdowns in the 12 months to September 2017 (an increase of 29.4 per cent when compared to the 17 breakdowns reported by APHA for the previous 12 months). Analysis of the 2018 figures indicates that both incidence and prevalence are now rising even faster, with a further 24 herd breakdowns occurring between 1 January and 5 December 2018.
To date, the government and its officials cite data that are two years out of date, but have declined to comment on this emerging evidence that, far from resulting in a substantial disease control benefit, badger culls may be leading to a sharp increase in bTB in cattle.
Natural Englandâs chief scientist and the UKâs CVO continue to endorse a failing and inhumane policy, bringing their offices into serious disrepute. We urge them, and the BVA, to reconsider their support for further badger culling, and instead focus on the actual cause of bTBâs epidemic spread â a cattle skin test with a sensitivity of only 50 per cent,6,7 and the ongoing problems associated with cattle movements and on-farm biosecurity
Animal welfare impacts of badger culling operations
We are writing to express our extreme concern following recent media coverage1, 2 relating to the methodology being used by contractors to kill badgers under licence, as part of the governmentâs policy to control bovine TB in cattle.
The coverage relates to the shooting of badgers that have been captured in live traps. Covert video footage (https://bit.ly/2Eud1iR ) from Cumbria shows a trapped badger being shot with a firearm at close range, following which it appears to take close to a minute to stop moving. The contractor clearly observes the animal during this time but makes no attempt to expedite the death of the badger and prevent further suffering, as required by the current Natural England best practice guide which states: âImmediately after shooting, the animal should be checked to ensure it is dead, and if there is any doubt, a second shot must be taken as soon as possible.â3 The conversation between the contractor and his companion also suggests they were considering moving the badger to another site before finally bagging the carcase, again breaching the best practice guide.
While the footage only relates to the experience of a single badger, and while the degree to which the badger was conscious in the period immediately following the shot is unclear, we can by no means be certain that the badger did not suffer. It also raises serious questions about the training, competence and behaviour of contractors, in relation to both badger welfare, and biosecurity. This adds to existing concerns relating to the humaneness of âcontrolled shootingâ (targeting free-roaming badgers with rifles), which continues to be a permitted method under culling licences, in spite of the reservations expressed by both the government-commissioned Independent Expert Panel in its 2014 report,4 and the BVA, which concluded in 2015 that it âcan no longer support the continued use of controlled shooting as part of the badger control policyâ.5 (However, it has since continued to support the issuing of licences which permit the method). The BVA has consistently indicated its support for what it calls the âtried and testedâ method of trapping and shooting, but has thus far failed to provide comprehensive and robust evidence for the humaneness of this method.
Figure1
Download figure
Open in new tab
Download powerpoint
Natural England reported that its monitors observed 74 (just over 0.6 per cent) of controlled shooting events for accuracy and humaneness
During 2017, almost 20,000 badgers were killed under licence across 19 cull zones, around 60 per cent of which were killed by controlled shooting, the remainder being trapped and shot.6 Natural England reported that its monitors observed 74 (just over 0.6 per cent) of controlled shooting events for accuracy and humaneness. No information has been provided on the extent to which trapping and shooting activities were monitored. This raises serious concerns about the extent of suffering that might be experienced by very large numbers of animals, for which contractors are not being held to account.
If contractors reach their maximum culling targets set by Natural England for 2018, as many as 41,000 additional badgers could be killed.7 The extent to which these animals will suffer is once again being left in the hands of contractors, with woefully inadequate oversight, and in the face of anecdotal evidence of breaches of best practice guidance. This situation is clearly unacceptable from an animal welfare perspective and it is our view that by endorsing the policy, the BVA is contradicting the principles contained within its own animal welfare strategy.8
We therefore urge the BVA to withdraw its support for any further licensed badger culling, and the RCVS to make it clear that any veterinarian who provides support for culling activities that result in unnecessary and avoidable animal suffering could face disciplinary proceedings. The veterinary profession has no business supporting this licensed mass killing with all its inherent negative welfare and biosecurity implications, and for which the disease control benefits are, at best, extremely uncertain.
We believe the continued support for the culls by veterinary bodies in the face of poor evidence for its efficacy damages the credibility of the profession, and that same support in the face of potential animal suffering on a large scale undermines its reputation.
We stand ready to discuss these issues in more detail
Approximate policy iteration: A survey and some new methods
We consider the classical policy iteration method of dynamic programming (DP), where approximations and simulation are used to deal with the curse of dimensionality. We survey a number of issues: convergence and rate of convergence of approximate policy evaluation methods, singularity and susceptibility to simulation noise of policy evaluation, exploration issues, constrained and enhanced policy iteration, policy oscillation and chattering, and optimistic and distributed policy iteration. Our discussion of policy evaluation is couched in general terms and aims to unify the available methods in the light of recent research developments and to compare the two main policy evaluation approaches: projected equations and temporal differences (TD), and aggregation. In the context of these approaches, we survey two different types of simulation-based algorithms: matrix inversion methods, such as least-squares temporal difference (LSTD), and iterative methods, such as least-squares policy evaluation (LSPE) and TD (λ), and their scaled variants. We discuss a recent method, based on regression and regularization, which rectifies the unreliability of LSTD for nearly singular projected Bellman equations. An iterative version of this method belongs to the LSPE class of methods and provides the connecting link between LSTD and LSPE. Our discussion of policy improvement focuses on the role of policy oscillation and its effect on performance guarantees. We illustrate that policy evaluation when done by the projected equation/TD approach may lead to policy oscillation, but when done by aggregation it does not. This implies better error bounds and more regular performance for aggregation, at the expense of some loss of generality in cost function representation capability. Hard aggregation provides the connecting link between projected equation/TD-based and aggregation-based policy evaluation, and is characterized by favorable error bounds.National Science Foundation (U.S.) (No.ECCS-0801549)Los Alamos National Laboratory. Information Science and Technology InstituteUnited States. Air Force (No.FA9550-10-1-0412
Use of MODIS Satellite Data to Evaluate Juniperus spp. Pollen Phenology to Support a Pollen Dispersal Model, PREAM, to Support Public Health Allergy Alerts
Pollen can be transported great distances. Van de Water et. al., 2003 reported Juniperus spp. pollen was transported 200-600 km. Hence local observations of plant phenology may not be consistent with the timing and source of pollen collected by pollen sampling instruments. The DREAM (Dust REgional Atmospheric Model, Nickovic et al. 2001) is a verified model for atmospheric dust transport modeling using MODIS data products to identify source regions and concentrations of dust. We are modifying the DREAM model to incorporate pollen transport. Pollen emission is based on MODIS-derived phenology of Juniperus spp. communities. Ground-based observational records of pollen release timing and quantities will be used as model verification. This information will be used to support the Centers for Disease Control and Prevention's National Environmental Public Health Tracking Program and the State of New Mexico environmental public health decision support for asthma and allergies alerts
Critical factors in the empirical performance of temporal difference and evolutionary methods for reinforcement learning
- âŠ