Learning to Act with RVRL Agents

The use of reinforcement learning to guide action selection of cognitive agents has been shown to be a powerful technique for stochastic environments. Standard Reinforcement learning techniques used to provide decision theoretic policies rely, however, on explicit state-based computations of value for each state-action pair. This requires the computation of a number of values exponential to the number of state variables and actions in the system. This research extends existing work with an acquired probabilistic rule representation of an agent environment by developing an algorithm to apply reinforcement learning to values attached to the rules themselves. Structure captured by the rules is then used to learn a policy directly. The resulting value attached to each rule represents the utility of taking an action if the conditions of the rule are present in the agent’s current set of percepts. This has several advantages for planning purposes: generalization over many states and over unseen states; effective decisions can therefore be made with less training data than state based modelling systems (e.g. Dyna Q-Learning); and the problem of computation in an exponential state-action space is alleviated. The results of application of this algorithm to rules in a specific environment are presented, with comparison to standard reinforcement learning policies developed from related work

The quality of different types of child care at 10 and 18 months. A comparison between types and factors related to quality.

The quality of care offered in four different types of non-parental child care to 307 infants at 10 months old and 331 infants at 18 months old was compared and factors associated with higher quality were identified. Observed quality was lowest in nurseries at each age point, except that at 18 months they offered more learning activities. There were few differences in the observed quality of care by child-minders, grandparents and nannies, although grandparents had somewhat lower safety and health scores and offered children fewer activities. Cost was largely unrelated to quality of care except in child-minding, where higher cost was associated with higher quality. Observed ratios of children to adults had a significant impact on quality of nursery care; the more infants or toddlers each adult had to care for, the lower the quality of the care she gave them. Mothers' overall satisfaction with their child's care was positively associated with its quality for home-based care but not for nursery settings

Space Charge Limited 2-d Electron Flow between Two Flat Electrodes in a Strong Magnetic Field

An approximate analytic solution is constructed for the 2-d space charge limited emission by a cathode surrounded by non emitting conducting ledges of width Lambda. An essentially exact solution (via conformal mapping) of the electrostatic problem in vacuum is matched to the solution of a linearized problem in the space charge region whose boundaries are sharp due to the presence of a strong magnetic field. The current density growth in a narrow interval near the edges of the cathode depends strongly on Lambda. We obtain an empirical formula for the total current as a function of Lambda which extends to more general cathode geometries.Comment: 4 pages, LaTex, e-mail addresses: [email protected], [email protected]

Counterintuitive transitions in multistate curve crossing involving linear potentials

Two problems incorporating a set of horizontal linear potentials crossed by a sloped linear potential are analytically solved and compared with numerical results: (a) the case where boundary conditions are specified at the ends of a finite interval, and (b) the case where the sloped linear potential is replaced by a piecewise-linear sloped potential and the boundary conditions are specified at infinity. In the approximation of small gaps between the horizontal potentials, an approach similar to the one used for the degenerate problem (Yurovsky V A and Ben-Reuven A 1998 J. Phys. B 31,1) is applicable for both problems. The resulting scattering matrix has a form different from the semiclassical result obtained by taking the product of Landau-Zener amplitudes. Counterintuitive transitions involving a pair of successive crossings, in which the second crossing precedes the first one along the direction of motion, are allowed in both models considered here.Comment: LaTeX 2.09 using ioplppt.sty and psfig.sty, 16 pages with 5 figures. Submitted to J. Phys.

Age structure, dispersion and diet of a population of stoats (Mustela erminea) in southern Fiordland during the decline phase of the beechmast cycle

The dispersion, age structure and diet of stoats (Mustela erminea) in beech forest in the Borland and Grebe Valleys, Fiordland National Park, were examined during December and January 2000/01, 20 months after a heavy seed-fall in 1999. Thirty trap stations were set along a 38-km transect through almost continuous beech forest, at least 1 km apart. Mice were very scarce (nights, C/100TN) along two standard index lines placed at either end of the transect, compared with November 1999 (>60/100TN), but mice were detected (from footprints in stoat tunnels) along an 8 km central section of the transect (stations 14-22). Live trapping with one trap per station (total 317.5 trap nights) in December 2000 caught 2 female and 23 male stoats, of which 10 (including both females) were radio collared. The minimum range lengths of the two females along the transect represented by the trap line were 2.2 and 6.0 km; those of eight radio-tracked males averaged 2.9 ± 1.7 km. Stations 14-22 tended to be visited more often, by more marked individual stoats, than the other 21 stations. Fenn trapping at the same 30 sites, but with multiple traps per station (1333.5 trap nights), in late January 2001 collected carcasses of 35 males and 28 females (including 12 of the marked live-trapped ones). Another two marked males were recovered dead. The stoat population showed no sign of chronic nutritional stress (average fat reserve index = 2.8 on a scale of 1-4 where 4 = highest fat content); and only one of 63 guts analysed was empty. Nevertheless, all 76 stoats handled were adults with 1-3 cementum annuli in their teeth, showing that reproduction had failed that season. Prey categories recorded in descending frequency of occurrence were birds, carabid beetle (ground beetle), weta, possum, rat, and mouse. The frequencies of occurrence of mice and birds in the diet of these stoats (10% and 48%, respectively) were quite different from those in stoats collected in Pig Creek, a tributary of the Borland River (87%, 5%), 12 months previously when mice were still abundant. Five of the six stoat guts containing mice were collected within 1 km of stations 14-22

Non-Euclidean Video Games: Exploring Player Perceptions and Experiences inside Impossible Spaces

Non-Euclidean geometry has the potential to be used for novel interactions in video games and create virtual spaces that are not physically possible in the real world. To explore how players perceive and experience them in video games, we have adapted two well-known 2D games, Snake and Asteroids to create two versions in addition to the conventional virtual space – with hyperbolic and spherical environments – and conducted a within-subject design user study on all three versions of these games. The results show that experienced Mastery and Control are lower when playing the two non-Euclidean versions while perceived Immersion and Challenge do not differ significantly between these three conditions. We also report on the qualitative findings from our participants, which provide further insights into the perception and experiences of these environments

The implementation of NICE guidance on venous thromboembolism risk assessment and prophylaxis: A before- after observational study to assess the impact on patient safety across four hospitals in England.

Exact publication date to be confirmed.Introduction: Venous thromboembolism (VTE) is a major cause of morbidity and mortality in hospitalised patients. VTE prevention has been identified as a major health need internationally to improve patient safety. A National Institute for Health and Clinical Excellence (NICE) guideline was issued in February 2010. Its key priorities were to assess patients for risk of VTE on admission to hospital, assess patients for bleeding risk and evaluate the risks and benefits of prescribing VTE prophylaxis. Objectives: To evaluate the implementation of NICE guidance and its impact on patient safety. Design: Before and after observational study design was used to investigate changes in VTE risk assessment documentation and inappropriate prescribing of prophylaxis between the year prior to (2009) and the year following (2010) the implementation of NICE guidance. A total of 816 patients were sampled in each year in four hospitals in the NHS South region. Results: The percentage of patients for whom a VTE risk assessment was documented increased from 51.5% (210/408) in 2009 to 79.2% (323/408) in 2010; difference 27.7% (95% CI: 21.4% to 33.9%; p<0.001). There was little evidence of change in the percentage who were prescribed prophylaxis amongst patients without a risk assessment (71.7% (142/198) in 2009 and 68.2% (58/85) in 2010; difference -3.5%% (95% CI: -15.2% to 8.2%; p =0.56) nor the percentage who were prescribed low molecular weight heparin amongst patients with a contraindication (14% (4/28) in 2009 and 15% (6/41) in 2010; RD = 0.3% (95% CI: -16.5% to 17.2%; p =0.97). Conclusions: The documentation of risk assessment improved following the implementation of NICE guidance but this did not lead to improved patient safety when prescribing prophylaxis

State-to-State Differential and Relative Integral Cross Sections for Rotationally Inelastic Scattering of H2O by Hydrogen

State-to-state differential cross sections (DCSs) for rotationally inelastic scattering of H2O by H2 have been measured at 71.2 meV (574 cm-1) and 44.8 meV (361 cm-1) collision energy using crossed molecular beams combined with velocity map imaging. A molecular beam containing variable compositions of the (J = 0, 1, 2) rotational states of hydrogen collides with a molecular beam of argon seeded with water vapor that is cooled by supersonic expansion to its lowest para or ortho rotational levels (JKaKc= 000 and 101, respectively). Angular speed distributions of fully specified rotationally excited final states are obtained using velocity map imaging. Relative integral cross sections are obtained by integrating the DCSs taken with the same experimental conditions. Experimental state-specific DCSs are compared with predictions from fully quantum scattering calculations on the most complete H2O-H2 potential energy surface. Comparison of relative total cross sections and state-specific DCSs show excellent agreement with theory in almost all detailsComment: 46 page

Counterintuitive transitions between crossing energy levels

We calculate analytically the probabilities for intuitive and counterintuitive transitions in a three-state system, in which two parallel energies are crossed by a third, tilted energy. The state with the tilted energy is coupled to the other two states in a chainwise linkage pattern with constant couplings of finite duration. The probability for a counterintuitive transition is found to increase with the square of the coupling and decrease with the squares of the interaction duration, the energy splitting between the parallel energies, and the tilt (chirp) rate. Physical examples of this model can be found in coherent atomic excitation and optical shielding in cold atomic collisions

Curve crossing in linear potential grids: the quasidegeneracy approximation

The quasidegeneracy approximation [V. A. Yurovsky, A. Ben-Reuven, P. S. Julienne, and Y. B. Band, J. Phys. B {\bf 32}, 1845 (1999)] is used here to evaluate transition amplitudes for the problem of curve crossing in linear potential grids involving two sets of parallel potentials. The approximation describes phenomena, such as counterintuitive transitions and saturation (incomplete population transfer), not predictable by the assumption of independent crossings. Also, a new kind of oscillations due to quantum interference (different from the well-known St\"uckelberg oscillations) is disclosed, and its nature discussed. The approximation can find applications in many fields of physics, where multistate curve crossing problems occur.Comment: LaTeX, 8 pages, 8 PostScript figures, uses REVTeX and psfig, submitted to Physical Review