Using informative behavior to increase engagement while learning from human reward

In this work, we address a relatively unexplored aspect of designing agents that learn from human reward. We investigate how an agent’s non-task behavior can affect a human trainer’s training and agent learning. We use the TAMER framework, which facilitates the training of agents by human-generated reward signals, i.e., judgements of the quality of the agent’s actions, as the foundation for our investigation. Then, starting from the premise that the interaction between the agent and the trainer should be bi-directional, we propose two new training interfaces to increase a human trainer’s active involvement in the training process and thereby improve the agent’s task performance. One provides information on the agent’s uncertainty which is a metric calculated as data coverage, the other on its performance. Our results from a 51-subject user study show that these interfaces can induce the trainers to train longer and give more feedback. The agent’s performance, however, increases only in response to the addition of performance-oriented information, not by sharing uncertainty levels. These results suggest that the organizational maxim about human behavior, “you get what you measure”—i.e., sharing metrics with people causes them to focus on optimizing those metrics while de-emphasizing other objectives—also applies to the training of agents. Using principle component analysis, we show how trainers in the two conditions train agents differently. In addition, by simulating the influence of the agent’s uncertainty–informative behavior on a human’s training behavior, we show that trainers could be distracted by the agent sharing its uncertainty levels about its actions, giving poor feedback for the sake of reducing the agent’s uncertainty without improving the agent’s performance

Observations of whistler mode waves with nonlinear parallel electric fields near the dayside magnetic reconnection separatrix by the Magnetospheric Multiscale mission

We show observations from the Magnetospheric Multiscale (MMS) mission of whistler mode waves in the Earth's low-latitude boundary layer (LLBL) during a magnetic reconnection event. The waves propagated obliquely to the magnetic field toward the X line and were confined to the edge of a southward jet in the LLBL. Bipolar parallel electric fields interpreted as electrostatic solitary waves (ESW) are observed intermittently and appear to be in phase with the parallel component of the whistler oscillations. The polarity of the ESWs suggests that if they propagate with the waves, they are electron enhancements as opposed to electron holes. The reduced electron distribution shows a shoulder in the distribution for parallel velocities between 17,000 and 22,000 km/s, which persisted during the interval when ESWs were observed, and is near the phase velocity of the whistlers. This shoulder can drive Langmuir waves, which were observed in the high-frequency parallel electric field data

Two-Dimensional Velocity of the Magnetic Structure Observed on July 11, 2017 by the Magnetospheric Multiscale Spacecraft

In order to determine particle velocities and electric field in the frame of the magnetic structure, one first needs to determine the velocity of the magnetic structure in the frame of the spacecraft observations. Here, we demonstrate two methods to determine a two-dimensional magnetic structure velocity for the magnetic reconnection event observed in the magnetotail by the Magnetospheric Multiscale (MMS) spacecraft on July 11, 2017, Spatio-Temporal Difference (STD) and the recently developed polynomial reconstruction method. Both of these methods use the magnetic field measurements; the reconstruction technique also uses the current density measured by the particle instrument. We find rough agreement between the results of our methods and with other velocity determinations previously published. We also explain a number of features of STD and show that the polynomial reconstruction technique is most likely to be valid within a distance of 2 spacecraft spacings from the centroid of the MMS spacecraft. Both of these methods are susceptible to contamination by magnetometer calibration errors

Space Physics Effects in the Near-Magnetopause Magnetosheath Elicited by Large-Amplitude Alfvénic Fluctuations Terminating in a Field and Flow Discontinuity

International audienceIn this paper we report on a sequence of large-amplitude Alfvénic fluctuations terminating in a field and flow discontinuity and their effects on electromagnetic fields and plasmas in the near-magnetopause magnetosheath. An arc-polarized structure in the magnetic field was observed by the Time History of Events and Macroscale Interactions during Substorms-C in the solar wind, indicative of nonlinear Alfvén waves. It ends with a combined tangential discontinuity/vortex sheet, which is strongly inclined to the ecliptic plane and at which there is a sharp rise in the density and a drop in temperature. Several effects resulting from this structure were observed by the Magnetospheric Multiscale spacecraft in the magnetosheath close to the subsolar point (11:30 magnetic local time) and somewhat south of the geomagnetic equator (−33 ∘ magnetic latitude): (i) kinetic Alfvén waves; (ii) a peaking of the electric and magnetic field strengths where E ⋅ J becomes strong and negative (−1 nW/m 3) just prior to an abrupt dropout of the fields; (iii) evolution in the pitch angle distribution of energetic (a few tens of kilo-electron-volts) ions (H + , He n+ , and O n+) and electrons inside a high-density region, which we attribute to gyrosounding of the tangential discontinuity/vortex sheet structure passing by the spacecraft; (iv) field-aligned acceleration of ions and electrons that could be associated with localized magnetosheath reconnection inside the high-density region; and (v) variable and strong flow changes, which we argue to be unrelated to reconnection at partial magnetopause crossings and likely result from deflections of magnetosheath flow by a locally deformed, oscillating magnetopause

An assigned responsibility system for robotic teleoperation control

This paper proposes an architecture that explores a gap in the spectrum of existing strategies for robot control mode switching in adjustable autonomy. In situations where the environment is reasonably known and/or predictable, pre-planning these control changes could relieve robot operators of the additional task of deciding when and how to switch. Such a strategy provides a clear division of labour between the automation and the human operator(s) before the job even begins, allowing for individual responsibilities to be known ahead of time, limiting confusion and allowing rest breaks to be planned. Assigned Responsibility is a new form of adjustable autonomy-based teleoperation that allows the selective inclusion of automated control elements at key stages of a robot operation plan’s execution. Progression through these stages is controlled by automatic goal accomplishment tracking. An implementation is evaluated through engineering tests and a usability study, demonstrating the viability of this approach and offering insight into its potential applications

Spatiotemporal Coordination Supports a Sense of Commitment in Human-Robot Interaction

In the current study, we presented participants with videos in which a humanoid robot (iCub) and a human agent were tidying up by moving toys from a table into a container. In the High Coordination condition, the two agents worked together in a coordinated manner, with the human picking up the toys and passing them to the robot. In the Low Coordination condition, they worked in parallel without coordinating. Participants were asked to imagine themselves in the position of the human agent and to respond to a battery of questions to probe the extent to which they felt committed to the joint action. While we did not observe a main effect of our coordination manipulation, the results do reveal that participants who perceived a higher degree of coordination also indicated a greater sense of commitment to the joint action. Moreover, the results show that participants’ sensitivity to the coordination manipulation was contingent on their prior attitudes towards the robot: participants in the High Coordination condition reported a greater sense of commitment than participants in the Low Coordination condition, except among those participants who were a priori least inclined to experience a close sense of relationship with the robot

Electron inflow velocities and reconnection rates at earth's magnetopause and magnetosheath

Electron inflow and outflow velocities during magnetic reconnection at and near the dayside magnetopause are measured using satellites from NASA's Magnetospheric Multiscale (MMS) mission. A case study is examined in detail, and three other events with similar behavior are shown, with one of them being a recently published electron-only reconnection event in the magnetosheath. The measured inflow speeds of 200–400 km/s imply dimensionless reconnection rates of 0.05–0.25 when normalized to the relevant electron Alfvén speed, which are within the range of expectations. The outflow speeds are about 1.5–3 times the inflow speeds, which is consistent with theoretical predictions of the aspect ratio of the inner electron diffusion region. A reconnection rate of 0.04 ± 25% was obtained for the case study event using the reconnection electric field as compared to the 0.12 ± 20% rate determined from the inflow velocity.publishedVersio

The Properties of Lion Roars and Electron Dynamics in Mirror Mode Waves Observed by the Magnetospheric MultiScale Mission

Mirror mode waves are ubiquitous in the Earth's magnetosheath, in particular behind the quasi‐perpendicular shock. Embedded in these nonlinear structures, intense lion roars are often observed. Lion roars are characterized by whistler wave packets at a frequency ∼100 Hz, which are thought to be generated in the magnetic field minima. In this study, we make use of the high time resolution instruments on board the Magnetospheric MultiScale mission to investigate these waves and the associated electron dynamics in the quasi‐perpendicular magnetosheath on 22 January 2016. We show that despite a core electron parallel anisotropy, lion roars can be generated locally in the range 0.05–0.2fce by the perpendicular anisotropy of electrons in a particular energy range. We also show that intense lion roars can be observed up to higher frequencies due to the sharp nonlinear peaks of the signal, which appear as sharp spikes in the dynamic spectra. As a result, a high sampling rate is needed to estimate correctly their amplitude, and the latter might have been underestimated in previous studies using lower time resolution instruments. We also present for the first‐time 3‐D high time resolution electron velocity distribution functions in mirror modes. We demonstrate that the dynamics of electrons trapped in the mirror mode structures are consistent with the Kivelson and Southwood (1996) model. However, these electrons can also interact with the embedded lion roars: first signatures of electron quasi‐linear pitch angle diffusion and possible signatures of nonlinear interaction with high‐amplitude wave packets are presented. These processes can lead to electron untrapping from mirror modes