918 research outputs found
Actor-Critic Reinforcement Learning for Control with Stability Guarantee
Reinforcement Learning (RL) and its integration with deep learning have
achieved impressive performance in various robotic control tasks, ranging from
motion planning and navigation to end-to-end visual manipulation. However,
stability is not guaranteed in model-free RL by solely using data. From a
control-theoretic perspective, stability is the most important property for any
control system, since it is closely related to safety, robustness, and
reliability of robotic systems. In this paper, we propose an actor-critic RL
framework for control which can guarantee closed-loop stability by employing
the classic Lyapunov's method in control theory. First of all, a data-based
stability theorem is proposed for stochastic nonlinear systems modeled by
Markov decision process. Then we show that the stability condition could be
exploited as the critic in the actor-critic RL to learn a controller/policy. At
last, the effectiveness of our approach is evaluated on several well-known
3-dimensional robot control tasks and a synthetic biology gene network tracking
task in three different popular physics simulation platforms. As an empirical
evaluation on the advantage of stability, we show that the learned policies can
enable the systems to recover to the equilibrium or way-points when interfered
by uncertainties such as system parametric variations and external disturbances
to a certain extent.Comment: IEEE RA-L + IROS 202
Recommended from our members
Complexity Theory and Language Development: In celebration of Diane Larsen-Freeman
This book presents a collective effort of scholars in applied linguistics to expand the horizon of the application of Complexity Theory (CT) in the research of second language development (SLD). Ever since the inception of this paradigmatic shift initiated by Diane Larsen-Freeman in 1997, SLD research has gradually transformed to acknowledge, value, and focus on the investigation of the variability and nonlinearity of the SLD process. The curation of this volume includes first a centerpiece by Larsen-Freeman that lays out the theoretical background of CT in SLD research and nine other chapters, both theoretical and empirical, that center around the discussion of SLD as a complex dynamic system. Topics covered include the use of terminology under the CT framework, complex brain structures and language development, research on individual differences, methodological techniques and norms under CT, CT and L2 pedagogy, the multilingual systems, and the ultimate outcome of SLD
The dependence of the structure of planet-opened gaps in protoplanetary disks on radiative cooling
Planets can excite density waves and open annular gas gaps in protoplanetary
disks. The depth of gaps is influenced by the evolving angular momentum carried
by density waves. While the impact of radiative cooling on the evolution of
density waves has been studied, a quantitative correlation to connect gap depth
with the cooling timescale is lacking. To address this gap in knowledge, we
employ the grid-based code Athena++ to simulate disk-planet interactions,
treating cooling as a thermal relaxation process. We establish quantitative
dependences of steady-state gap depth (Eq. 36) and width (Eq. 41) on planetary
mass, Shakura-Sunyaev viscosity, disk scale height, and thermal relaxation
timescale . We confirm previous results that gap opening is the
weakest when thermal relaxation timescale is comparable to local dynamical
timescale. Significant variations in gap depth, up to an order of magnitude,
are found with different . In terms of width, a gap is at its narrowest
around , approximately to narrower compared to the
isothermal case. When , it can be wider, and higher
viscosity enhances this effect. We derive possible masses of the gas
gap-opening planets in AS 209, HD 163296, MWC 480, and HL Tau, accounting for
the uncertainties in local thermal relaxation timescale.Comment: 19 pages, 16 figures, 4 tables, accepted for publication in Ap
Expressing metaphorically, writing creatively: Metaphor identification for creativity assessment
Metaphor, which can implicitly express profound meanings and emotions, is a unique writing technique frequently used in human language. In writing, meaningful metaphorical expressions can enhance the literariness and creativity of texts. Therefore, the usage of metaphor is a significant impact factor when assessing the creativity and literariness of writing. However, little to no automatic writing assessment system considers metaphorical expressions when giving the score of creativity. For improving the accuracy of automatic writing assessment, this paper proposes a novel creativity assessment model that imports a token-level metaphor identification method to extract metaphors as the indicators for creativity scoring. The experimental results show that our model can accurately assess the creativity of different texts with precise metaphor identification. To the best of our knowledge, we are the first to apply automatic metaphor identification to assess writing creativity. Moreover, identifying features (e.g., metaphors) that influence writing creativity using computational approaches can offer fair and reliable assessment methods for educational settings
- …