Exploiting Multiple Secondary Reinforcers in Policy Gradient Reinforcement Learning

Most formulations of Reinforcement Learning depend on a single reinforcement reward value to guide the search for the optimal policy solution

Localizing Search in Reinforcement Learning

Reinforcement learning (RL) can be impractical for many high dimensional problems because of the computational cost of doing stochastic search in large state spaces. We propose a new RL method, Boundary Localized Reinforcement Learning (BLRL), which maps RL into a mode switching problem where an agent deterministically chooses an action based on its state, and limits stochastic search to small areas around mode boundaries, drastically reducing computational cost. BLRL starts with an initial set of parameterized boundaries that partition the state space into distinct control modes. Reinforcement reward is used to update the boundary parameters using the policy gradient formulation of Sutton et al. (2000). We demonstrate that stochastic search can be limited to regions near mode boundaries, thus greatly reducing search, while still guaranteeing convergence to a locally optimal deterministic mode switching policy. Further, we give conditions under which the policy gradie..

Science and Measurement Requirements for a Plant Physiology and Functional Types Mission: Measuring the Composition, Function and Health of Global Land and Coastal Ocean Ecosystems

This slide presentation reviews the proposed Plant Physiology and Functional Types (PPFT) Mission. The National Academy of Sciences Decadal Survey, placed a critical priority on a Mission to observe distribution and changes in ecosystem functions. The PPFT satellite mission provides the essential measurements needed to assess drivers of change in biodiversity and ecosystem services that affect human welfare. The presentation reviews the science questions that the mission will be designed to answer, the science rationale, the science measurements, the mission concept, the planned instrumentation, the calibration method, and key signal to noise ratios and uniformity requirements

GNSS technology and its application for improved reproductive management in extensive sheep systems

The behaviour of Merino ewes during non-oestrus and oestrus were quantified using Global Navigation Satellite System (GNSS) tracking devices and direct visual observation. GNSS devices were attached to neck collars and deployed on mixed-Age ewes (38 maiden and 40 experienced ewes) following hormonal oestrus synchronisation. The positional accuracy of the GNSS data was validated through a comparative study of GNSS estimates of each animal's location compared with direct visual observations. Positional accuracy was estimated at 90-94%, for a 4-m and 6-m-buffer radius, respectively. Ewe speed of movement was calculated from the GNSS data and plotted against hour of the day to determine diurnal activity patterns during non-oestrus and oestrus days. Ewes showed increased speed of movement during the early morning of the anticipated day of oestrus compared with the non-oestrus day (P < 0.001). In addition, ewes that increased their speed of movement by 0.05 m/s received 1.4-28.4 times more mounts depending on the hour of the day (P ≤ 0.02). Ewes also displayed an increased speed of movement in the period leading up to maximum sexual activity, defined as the hour in which ewes received the maximum number of mounts. Thereafter, ewe activity decreased. No difference in sexual activity was detected between maiden and experienced ewes. The present study has demonstrated a change in ewe diurnal activity at oestrus, suggesting the onset of sexual activity can be identified as a period of increased speed of movement followed by a return to 'normal' activity. The development of commercial remote autonomous monitoring technologies such as GNSS tracking to detect this change in behaviour could facilitate improved reproductive management of sheep in extensive systems. © CSIRO 2015