95 research outputs found
Decision S4: Efficient Sequence-Based RL via State Spaces Layers
Recently, sequence learning methods have been applied to the problem of
off-policy Reinforcement Learning, including the seminal work on Decision
Transformers, which employs transformers for this task. Since transformers are
parameter-heavy, cannot benefit from history longer than a fixed window size,
and are not computed using recurrence, we set out to investigate the
suitability of the S4 family of models, which are based on state-space layers
and have been shown to outperform transformers, especially in modeling
long-range dependencies. In this work we present two main algorithms: (i) an
off-policy training procedure that works with trajectories, while still
maintaining the training efficiency of the S4 model. (ii) An on-policy training
procedure that is trained in a recurrent manner, benefits from long-range
dependencies, and is based on a novel stable actor-critic mechanism. Our
results indicate that our method outperforms multiple variants of decision
transformers, as well as the other baseline methods on most tasks, while
reducing the latency, number of parameters, and training time by several orders
of magnitude, making our approach more suitable for real-world RL.Comment: 21 pages,13 figure
Process Tomography for Systems in a Thermal State
We propose a new method for implementing process tomography that is based on
the information extracted from temporal correlations between observables,
rather than on state preparation and state tomography. As such, the approach is
applicable to systems that are in a mixed state, and in particular to thermal
states. We illustrate the method for an arbitrary evolution described by Kraus
operators, as well as for simpler cases such as a general Gaussian channels,
and qubit dynamics
Inter-areal coordination of columnar architectures during visual cortical development
The occurrence of a critical period of plasticity in the visual cortex has
long been established, yet its function in normal development is not fully
understood. Here we show that as the late phase of the critical period unfolds,
different areas of cat visual cortex develop in a coordinated manner.
Orientation columns in areas V1 and V2 become matched in size in regions that
are mutually connected. The same age trend is found for such regions in the
left and right brain hemisphere. Our results indicate that a function of
critical period plasticity is to progressively coordinate the functional
architectures of different cortical areas - even across hemispheres.Comment: 30 pages, 1 table, 6 figure
Coordinated optimization of visual cortical maps (I) Symmetry-based analysis
In the primary visual cortex of primates and carnivores, functional
architecture can be characterized by maps of various stimulus features such as
orientation preference (OP), ocular dominance (OD), and spatial frequency. It
is a long-standing question in theoretical neuroscience whether the observed
maps should be interpreted as optima of a specific energy functional that
summarizes the design principles of cortical functional architecture. A
rigorous evaluation of this optimization hypothesis is particularly demanded by
recent evidence that the functional architecture of OP columns precisely
follows species invariant quantitative laws. Because it would be desirable to
infer the form of such an optimization principle from the biological data, the
optimization approach to explain cortical functional architecture raises the
following questions: i) What are the genuine ground states of candidate energy
functionals and how can they be calculated with precision and rigor? ii) How do
differences in candidate optimization principles impact on the predicted map
structure and conversely what can be learned about an hypothetical underlying
optimization principle from observations on map structure? iii) Is there a way
to analyze the coordinated organization of cortical maps predicted by
optimization principles in general? To answer these questions we developed a
general dynamical systems approach to the combined optimization of visual
cortical maps of OP and another scalar feature such as OD or spatial frequency
preference.Comment: 90 pages, 16 figure
Coordinated optimization of visual cortical maps (II) Numerical studies
It is an attractive hypothesis that the spatial structure of visual cortical
architecture can be explained by the coordinated optimization of multiple
visual cortical maps representing orientation preference (OP), ocular dominance
(OD), spatial frequency, or direction preference. In part (I) of this study we
defined a class of analytically tractable coordinated optimization models and
solved representative examples in which a spatially complex organization of the
orientation preference map is induced by inter-map interactions. We found that
attractor solutions near symmetry breaking threshold predict a highly ordered
map layout and require a substantial OD bias for OP pinwheel stabilization.
Here we examine in numerical simulations whether such models exhibit
biologically more realistic spatially irregular solutions at a finite distance
from threshold and when transients towards attractor states are considered. We
also examine whether model behavior qualitatively changes when the spatial
periodicities of the two maps are detuned and when considering more than 2
feature dimensions. Our numerical results support the view that neither minimal
energy states nor intermediate transient states of our coordinated optimization
models successfully explain the spatially irregular architecture of the visual
cortex. We discuss several alternative scenarios and additional factors that
may improve the agreement between model solutions and biological observations.Comment: 55 pages, 11 figures. arXiv admin note: substantial text overlap with
arXiv:1102.335
Effect of Music Therapy on Parent-Infant Bonding Among Infants Born Preterm: A Randomized Clinical Trial
Importance Parent-infant bonding contributes to long-term infant health but may be disrupted by preterm birth. Objective To determine if parent-led, infant-directed singing, supported by a music therapist and initiated in the neonatal intensive care unit (NICU), improves parent-infant bonding at 6 and 12 months. Design, Setting, and Participants This randomized clinical trial was conducted in level III and IV NICUs in 5 countries between 2018 and 2022. Eligible participants were preterm infants (under 35 weeksβ gestation) and their parents. Follow-up was conducted across 12 months (as part of the LongSTEP study) at home or in clinics. Final follow-up was conducted at 12 monthsβ infant-corrected age. Data were analyzed from August 2022 to November 2022. Intervention Participants randomized to music therapy (MT) plus standard care or standard care alone during NICU admission, or to MT plus standard care or standard care alone postdischarge, using computer-generated randomization (ratio 1:1, block sizes of 2 or 4 varying randomly), stratified by site (51 allocated to MT NICU, 53 to MT postdischarge, 52 to both, and 50 to neither). MT consisted of parent-led, infant-directed singing tailored to infant responses and supported by a music therapist 3 times per week throughout hospitalization or 7 sessions across 6 monthsβ postdischarge. Main Outcome and Measure Primary outcome was mother-infant bonding at 6 monthsβ corrected age, measured by the Postpartum Bonding Questionnaire (PBQ), with follow-up at 12 monthsβ corrected age, and analyzed intention-to-treat as group differences. Results Of 206 enrolled infants with 206 mothers (mean [SD] age, 33 [6] years) and 194 fathers (mean [SD] age, 36 [6] years) randomized at discharge, 196 (95.1%) completed assessments at 6 months and were analyzed. Estimated group effects for PBQ at 6 monthsβ corrected age were 0.55 (95% CI, β2.20 to 3.30; Pβ=β.70) for MT in the NICU, 1.02 (95% CI, β1.72 to 3.76; Pβ=β.47) for MT postdischarge, and β0.20 (95% CI, β4.03 to 3.63; Pβ=β.92) for the interaction (12 months: MT in NICU, 0.17; 95% CI, β2.71 to 3.05; Pβ=β.91; MT postdischarge, 1.78; 95% CI, β1.13 to 4.70; Pβ=β.24; interaction, β1.68; 95% CI, β5.77 to 2.41; Pβ=β.42). There were no clinically important between-group differences for secondary variables. Conclusions and Relevance In this randomized clinical trial, parent-led, infant-directed singing did not have clinically important effects on mother-infant bonding, but was safe and well-accepted.publishedVersio
On the Origin of the Functional Architecture of the Cortex
The basic structure of receptive fields and functional maps in primary visual cortex is established without exposure to normal sensory experience and before the onset of the critical period. How the brain wires these circuits in the early stages of development remains unknown. Possible explanations include activity-dependent mechanisms driven by spontaneous activity in the retina and thalamus, and molecular guidance orchestrating thalamo-cortical connections on a fine spatial scale. Here I propose an alternative hypothesis: the blueprint for receptive fields, feature maps, and their inter-relationships may reside in the layout of the retinal ganglion cell mosaics along with a simple statistical connectivity scheme dictating the wiring between thalamus and cortex. The model is shown to account for a number of experimental findings, including the relationship between retinotopy, orientation maps, spatial frequency maps and cytochrome oxidase patches. The theory's simplicity, explanatory and predictive power makes it a serious candidate for the origin of the functional architecture of primary visual cortex
Recommended from our members
Protecting polyethylene irrigation pipes against damage caused by woodpeckers
Several methods were evaluated for protecting polrethylene irrigation pipes against pecking damage caused by the Syrian woodpecker (Dendrocopos syriacus). Only by burying the pipes in the ground damage was effectively prevented. Other methods studied, the use of the game repellent Arbinol, covering the pipes with polyethylene sheets, and growing a weed cover, though reducing the rate of the damage, proved not to be sufficiently effective as an economic solution of the problem
- β¦