646 research outputs found
A FIRST-OCCUPANCY REPRESENTATION FOR REINFORCEMENT LEARNING
Both animals and artificial agents benefit from state representations that support rapid transfer of learning across tasks and which enable them to efficiently traverse their environments to reach rewarding states. The successor representation (SR), which measures the expected cumulative, discounted state occupancy under a fixed policy, enables efficient transfer to different reward structures in an otherwise constant Markovian environment and has been hypothesized to underlie aspects of biological behavior and neural activity. However, in the real world, rewards may only be available for consumption once, may shift location, or agents may simply aim to reach goal states as rapidly as possible without the constraint of artificially imposed task horizons. In such cases, the most behaviorally-relevant representation would carry information about when the agent was likely to first reach states of interest, rather than how often it should expect to visit them over a potentially infinite time span. To reflect such demands, we introduce the first-occupancy representation (FR), which measures the expected temporal discount to the first time a state is accessed. We demonstrate that the FR facilitates exploration, the selection of efficient paths to desired states, allows the agent, under certain conditions, to plan provably optimal trajectories defined by a sequence of subgoals, and induces similar behavior to animals avoiding threatening stimuli
Terrestrial Exoplanet Light Curves
The phase or orbital light curves of extrasolar terrestrial planets in
reflected or emitted light will contain information about their atmospheres and
surfaces complementary to data obtained by other techniques such as
spectrosopy. We show calculated light curves at optical and thermal infrared
wavelengths for a variety of Earth-like and Earth-unlike planets. We also show
that large satellites of Earth-sized planets are detectable, but may cause
aliasing effects if the lightcurve is insufficiently sampled.Comment: To appear in Proceedings of the IAU Colloquium 200, Direct Imaging of
Exoplanets; Science & Technology, Villefranche-sur-mer, France, October 2-7,
200
Amortised learning by wake-sleep
Models that employ latent variables to capture structure in observed data lie at the heart of many current unsupervised learning algorithms, but exact maximum-likelihood learning for powerful and flexible latent-variable models is almost always intractable. Thus, state-of-the-art approaches either abandon the maximum-likelihood framework entirely, or else rely on a variety of variational approximations to the posterior distribution over the latents. Here, we propose an alternative approach that we call amortised learning. Rather than computing an approximation to the posterior over latents, we use a wake-sleep Monte-Carlo strategy to learn a function that directly estimates the maximum-likelihood parameter updates. Amortised learning is possible whenever samples of latents and observations can be simulated from the generative model, treating the model as a “black box”. We demonstrate its effectiveness on a wide range of complex models, including those with latents that are discrete or supported on non-Euclidean spaces
A Unified Theory of Dual-Process Control
Dual-process theories play a central role in both psychology and
neuroscience, figuring prominently in fields ranging from executive control to
reward-based learning to judgment and decision making. In each of these
domains, two mechanisms appear to operate concurrently, one relatively high in
computational complexity, the other relatively simple. Why is neural
information processing organized in this way? We propose an answer to this
question based on the notion of compression. The key insight is that
dual-process structure can enhance adaptive behavior by allowing an agent to
minimize the description length of its own behavior. We apply a single model
based on this observation to findings from research on executive control,
reward-based learning, and judgment and decision making, showing that seemingly
diverse dual-process phenomena can be understood as domain-specific
consequences of a single underlying set of computational principles
Minimum Description Length Control
We propose a novel framework for multitask reinforcement learning based on
the minimum description length (MDL) principle. In this approach, which we term
MDL-control (MDL-C), the agent learns the common structure among the tasks with
which it is faced and then distills it into a simpler representation which
facilitates faster convergence and generalization to new tasks. In doing so,
MDL-C naturally balances adaptation to each task with epistemic uncertainty
about the task distribution. We motivate MDL-C via formal connections between
the MDL principle and Bayesian inference, derive theoretical performance
guarantees, and demonstrate MDL-C's empirical effectiveness on both discrete
and high-dimensional continuous control tasks
Physical map location of the peptide methionine sulfoxide reductase gene on the Escherichia coli chromosome
This is the publisher's version, also available electronically from "http://jb.asm.org".No abstract available
The astorb database at Lowell Observatory
The astorb database at Lowell Observatory is an actively curated catalog of all known asteroids in the Solar System. astorb has heritage dating back to the 1970s and has been publicly accessible since the 1990s. Work began in 2015 to modernize the underlying database infrastructure, operational software, and associated web applications. That effort involved the expansion of astorb to incorporate new data such as physical properties (e.g. albedo, colors, spectral types) from a variety of sources. The data in astorb are used to support a number of research tools hosted at https://asteroid.lowell.edu. Here we present a full description of the software tools, computational foundation, and data products upon which the astorb ecosystem has been built. (c) 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).Peer reviewe
The 2016 Reactivations of Main-Belt Comets 238P/Read and 288P/(300163) 2006 VW139
We report observations of the reactivations of main-belt comets 238P/Read and
288P/(300163) 2006 VW139, that also track the evolution of each object's
activity over several months in 2016 and 2017. We additionally identify and
analyze archival SDSS data showing 288P to be active in 2000, meaning that both
238P and 288P have now each been confirmed to be active near perihelion on
three separate occasions. From data obtained of 288P from 2012-2015 when it
appeared inactive, we find best-fit R-band H,G phase function parameters of
H_R=16.80+/-0.12 mag and G_R=0.18+/-0.11, corresponding to effective component
radii of r_c=0.80+/-0.04 km, assuming a binary system with equally-sized
components. Fitting linear functions to ejected dust masses inferred for 238P
and 288P soon after their observed reactivations in 2016, we find an initial
average net dust production rate of 0.7+/-0.3 kg/s and a best-fit start date of
2016 March 11 (when the object was at a true anomaly of -63 deg) for 238P, and
an initial average net dust production rate of 5.6+/-0.7 kg/s and a best-fit
start date of 2016 August 5 (when the object was at a true anomaly of -27 deg)
for 288P. Applying similar analyses to archival data, we find similar start
points for previous active episodes for both objects, suggesting that minimal
mantle growth or ice recession occurred between the active episodes in
question. Some changes in dust production rates between active episodes are
detected, however. More detailed dust modeling is suggested to further clarify
the process of activity evolution in main-belt comets.Comment: 21 pages, 9 figures, accepted by A
Methionine sulfoxide reductase regulates brain catechol-O-methyl transferase activity
This is the published version. Copyright 2014 Oxford University PressCatechol-O-methyl transferase (COMT) plays a key role in the degradation of brain dopamine (DA). Specifically,
low COMT activity results in higher DA levels in the prefrontal cortex (PFC), thereby reducing the vulnerability
for attentional and cognitive deficits in both psychotic and healthy individuals. COMT activity is markedly
reduced by a non-synonymous single-nucleotide polymorphism (SNP) that generates a valine-to-methionine
substitution on the residue 108/158, by means of as-yet incompletely understood post-translational mechanisms.
One post-translational modification is methionine sulfoxide, which can be reduced by the methionine sulfoxide
reductase (Msr) A and B enzymes. We used recombinant COMT proteins (Val/Met108) and mice (wild-type (WT)
and MsrA knockout) to determine the effect of methionine oxidation on COMT activity and COMT interaction
with Msr, through a combination of enzymatic activity and Western blot assays. Recombinant COMT activity is
positively regulated by MsrA, especially under oxidative conditions, whereas brains of MsrA knockout mice
exhibited lower COMT activity (as compared with their WT counterparts). These results suggest that COMT activity
may be reduced by methionine oxidation, and point to Msr as a key molecular determinant for the modulation
of COMT activity in the brain. The role of Msr in modulating cognitive functions in healthy individuals
and schizophrenia patients is yet to be determined
- …