Representation learning with reward prediction errors

The Reward Prediction Error hypothesis proposes that phasic activity in the midbrain dopaminergic system reflects prediction errors needed for learning in reinforcement learning. Besides the well-documented association between dopamine and reward processing, dopamine is implicated in a variety of functions without a clear relationship to reward prediction error. Fluctuations in dopamine levels influence the subjective perception of time, dopamine bursts precede the generation of motor responses, and the dopaminergic system innervates regions of the brain, including hippocampus and areas in prefrontal cortex, whose function is not uniquely tied to reward. In this manuscript, we propose that a common theme linking these functions is representation, and that prediction errors signaled by the dopamine system, in addition to driving associative learning, can also support the acquisition of adaptive state representations. In a series of simulations, we show how this extension can account for the role of dopamine in temporal and spatial representation, motor response, and abstract categorization tasks. By extending the role of dopamine signals to learning state representations, we resolve a critical challenge to the Reward Prediction Error hypothesis of dopamine function

Cooperative Transmission for Wireless Relay Networks Using Limited Feedback

To achieve the available performance gains in half-duplex wireless relay networks, several cooperative schemes have been earlier proposed using either distributed space-time coding or distributed beamforming for the transmitter without and with channel state information (CSI), respectively. However, these schemes typically have rather high implementation and/or decoding complexities, especially when the number of relays is high. In this paper, we propose a simple low-rate feedback-based approach to achieve maximum diversity with a low decoding and implementation complexity. To further improve the performance of the proposed scheme, the knowledge of the second-order channel statistics is exploited to design long-term power loading through maximizing the receiver signal-to-noise ratio (SNR) with appropriate constraints. This maximization problem is approximated by a convex feasibility problem whose solution is shown to be close to the optimal one in terms of the error probability. Subsequently, to provide robustness against feedback errors and further decrease the feedback rate, an extended version of the distributed Alamouti code is proposed. It is also shown that our scheme can be generalized to the differential transmission case, where it can be applied to wireless relay networks with no CSI available at the receiver.Comment: V1: 27 pages, 1 column, 6 figures. Submitted to IEEE Transactions on Signal Processing, February 2, 2009. V2: 30 pages, 1 column, 8 figures. Revised version submitted to IEEE Transactions on Signal Processing, July 23, 200

Comparing the Performance of Hyperbolic and Circular Rod Quadrupole Mass Spectrometers with Applied Higher Order Auxiliary Excitation

This work applies higher order auxiliary excitation techniques to two types of quadrupole mass spectrometers (QMSs): commercial systems and spaceborne instruments. The operational settings of a circular rod geometry commercial system and an engineering test-bed for a hyperbolic rod geometry spaceborne instrument were matched, with the relative performance of each sensor characterized with and without applied excitation using isotopic measurements of Kr+. Each instrument was operated at the limit of the test electronics to determine the effect of auxiliary excitation on extending instrument capabilities. For the circular rod sensor, with applied excitation, a doubling of the mass resolution at 1% of peak transmission resulted from the elimination of the low-mass side peak tail typical of such rod geometries. The mass peak stability and ion rejection efficiency were also increased by factors of 2 and 10, respectively, with voltage scan lines passing through the center of stability islands formed from auxiliary excitation. Auxiliary excitation also resulted in factors of 6 and 2 in peak stability and ion rejection efficiency, respectively, for the hyperbolic rod sensor. These results not only have significant implications for the use of circular rod quadrupoles with applied excitation as a suitable replacement for traditional hyperbolic rod sensors, but also for extending the capabilities of existing hyperbolic rod QMSs for the next generation of spaceborne instruments and low-mass commercial systems

Generation of optimal trajectories for Earth hybrid pole sitters

A pole-sitter orbit is a closed path that is constantly above one of the Earth's poles, by means of continuous low thrust. This work proposes to hybridize solar sail propulsion and solar electric propulsion (SEP) on the same spacecraft, to enable such a pole-sitter orbit. Locally-optimal control laws are found with a semi-analytical inverse method, starting from a trajectory that satisfies the pole-sitter condition in the Sun-Earth circular restricted three-body problem. These solutions are subsequently used as first guess to find optimal orbits, using a direct method based on pseudospectral transcription. The orbital dynamics of both the pure SEP case and the hybrid case are investigated and compared. It is found that the hybrid spacecraft allows savings on propellant mass fraction. Finally, it is shown that for sufficiently long missions, a hybrid pole-sitter, based on mid-term technology, enables a consistent reduction in the launch mass for a given payload, with respect to a pure SEP spacecraft

Motion of the MMS Spacecraft Relative to the Magnetic Reconnection Structure Observed on 16 Oct 2015 at 1307 UT

We analyze a magnetopause crossing by the Magnetospheric Multiscale (MMS) spacecraft at 1307 UT on 16 Oct 2016 that showed features of electron scale reconnection. For this event, we find orthonormal LMN coordinates from the magnetic field, with N and L varying respectively along the maximum gradient and maximum variance directions. We find the motion along N from the Spatio-Temporal Difference analysis and motion along L from measured particle velocities. We locate the position of the magnetic X point, finding that MMS-4 passed within about 1.4 km from the X point and that MMS-3 and MMS-2 passed within about 1.7 km and 2.4 km, respectively, from the position of maximum out of plane current

Differentiating EDRs from the Background Magnetopause Current Sheet: A Statistical Study

The solar wind is a continuous outflow of charged particles from the Sun's atmosphere into the solar system. At Earth, the solar wind's outward pressure is balanced by the Earth's magnetic field in a boundary layer known as the magnetopause. Plasma density and temperature differences across the boundary layer generate the Chapman-Ferraro current which supports the magnetopause. Along the dayside magnetopause, magnetic reconnection can occur in electron diffusion regions (EDRs) embedded into the larger ion diffusion regions (IDRs). These diffusion regions form when opposing magnetic field lines in the solar wind and Earth's magnetic field merge, releasing magnetic energy into the surrounding plasma. While previous studies have given us a general understanding of the structure of the diffusion regions, we still do not have a good grasp of how they are statistically differentiated from the non-diffusion region magnetopause. By investigating 251 magnetopause crossings from NASA's Magnetospheric Multiscale (MMS) Mission, we demonstrate that EDR magnetopause crossings show current densities an order of magnitude higher than regular magnetopause crossings - crossings that either passed through the reconnection exhausts or through the non-reconnecting magnetopause, providing a baseline for the magnetopause current sheet under a wide range of driving conditions. Significant current signatures parallel to the local magnetic field in EDR crossings are also identified, which is in contrast to the dominantly perpendicular current found in the regular magnetopause. Additionally, we show that the ion velocity along the magnetopause is highly correlated with a crossing's location, indicating the presence of magnetosheath flows inside the magnetopause