105 research outputs found
End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks
Reinforcement Learning (RL) algorithms have found limited success beyond
simulated applications, and one main reason is the absence of safety guarantees
during the learning process. Real world systems would realistically fail or
break before an optimal controller can be learned. To address this issue, we
propose a controller architecture that combines (1) a model-free RL-based
controller with (2) model-based controllers utilizing control barrier functions
(CBFs) and (3) on-line learning of the unknown system dynamics, in order to
ensure safety during learning. Our general framework leverages the success of
RL algorithms to learn high-performance controllers, while the CBF-based
controllers both guarantee safety and guide the learning process by
constraining the set of explorable polices. We utilize Gaussian Processes (GPs)
to model the system dynamics and its uncertainties.
Our novel controller synthesis algorithm, RL-CBF, guarantees safety with high
probability during the learning process, regardless of the RL algorithm used,
and demonstrates greater policy exploration efficiency. We test our algorithm
on (1) control of an inverted pendulum and (2) autonomous car-following with
wireless vehicle-to-vehicle communication, and show that our algorithm attains
much greater sample efficiency in learning than other state-of-the-art
algorithms and maintains safety during the entire learning process.Comment: Published in AAAI 201
Connected Cruise and Traffic Control for Pairs of Connected Automated Vehicles
This paper considers mixed traffic consisting of connected automated vehicles
equipped with vehicle-to-everything (V2X) connectivity and human-driven
vehicles. A control strategy is proposed for communicating pairs of connected
automated vehicles, where the two vehicles regulate their longitudinal motion
by responding to each other, and, at the same time, stabilize the human-driven
traffic between them. Stability analysis is conducted to find stabilizing
controllers, and simulations are used to show the efficacy of the proposed
approach. The impact of the penetration of connectivity and automation on the
string stability of traffic is quantified. It is shown that, even with moderate
penetration, connected automated vehicle pairs executing the proposed
controllers achieve significant benefits compared to when these vehicles are
disconnected and controlled independently.Comment: Accepted to the IEEE Transactions on Intelligent Transportation
Systems. 11 pages, 10 figure
Control Regularization for Reduced Variance Reinforcement Learning
Dealing with high variance is a significant challenge in model-free
reinforcement learning (RL). Existing methods are unreliable, exhibiting high
variance in performance from run to run using different initializations/seeds.
Focusing on problems arising in continuous control, we propose a functional
regularization approach to augmenting model-free RL. In particular, we
regularize the behavior of the deep policy to be similar to a policy prior,
i.e., we regularize in function space. We show that functional regularization
yields a bias-variance trade-off, and propose an adaptive tuning strategy to
optimize this trade-off. When the policy prior has control-theoretic stability
guarantees, we further show that this regularization approximately preserves
those stability guarantees throughout learning. We validate our approach
empirically on a range of settings, and demonstrate significantly reduced
variance, guaranteed dynamic stability, and more efficient learning than deep
RL alone.Comment: Appearing in ICML 201
On the Safety of Connected Cruise Control: Analysis and Synthesis with Control Barrier Functions
Connected automated vehicles have shown great potential to improve the
efficiency of transportation systems in terms of passenger comfort, fuel
economy, stability of driving behavior and mitigation of traffic congestions.
Yet, to deploy these vehicles and leverage their benefits, the underlying
algorithms must ensure their safe operation. In this paper, we address the
safety of connected cruise control strategies for longitudinal car following
using control barrier function (CBF) theory. In particular, we consider various
safety measures such as minimum distance, time headway and time to conflict,
and provide a formal analysis of these measures through the lens of CBFs.
Additionally, motivated by how stability charts facilitate stable controller
design, we derive safety charts for existing connected cruise controllers to
identify safe choices of controller parameters. Finally, we combine the
analysis of safety measures and the corresponding stability charts to
synthesize safety-critical connected cruise controllers using CBFs. We verify
our theoretical results by numerical simulations.Comment: Accepted to the 62nd IEEE Conference on Decision and Control. 6
pages, 5 figure
Implication of the period-magnitude relation for massive AGB stars and its astronomical applications
We present astrometric very long baseline interferometry (VLBI) studies of
AGB stars. To understand the properties and evolution of AGB stars, distances
are an important parameter. The distribution and kinematics of their
circumstellar matter are also revealed with the VLBI method. We used the VERA
array to observe 22\,GHz HO masers in various subclasses of AGB stars.
Parallaxes of the three OH/IR stars NSV17351, OH39.71.5, IRC30363, and
the Mira-type variable star AW~Tau were newly obtained. We present the
circumstellar distribution and kinematics of HO masers around NSV17351. The
absolute magnitudes in mid-infrared bands of OH/IR stars with very long
pulsation periods were investigated and a period-magnitude relation in the WISE
W3 band, , was found
for the Galactic AGB stars. The VLBI is still a powerful tool for parallax
measurements of the Galactic AGB stars surrounded by thick dust shells.Comment: 24 pages, 8 figures, Proceedings of the IAU symposium 376, At the
cross-roads of astrophysics and cosmology Period luminosity relations in the
2020
Too many swipes for today: The Development of the Problematic Tinder Use Scale (PTUS)
Background and aims Tinder is a very popular smartphone-based geolocated dating application. The goal of the present study was creating a short Problematic Tinder Use Scale (PTUS). Methods Griffiths’ (2005) six-component model was implemented for covering all components of problematic Tinder use. Confirmatory factor analyses were carried out on a Tinder user sample (N = 430). Results Both the 12- and the 6-item versions were tested. The 6-item unidimensional structure has appropriate reliability and factor structure. No salient demography-related differences were found. Users irrespectively to their relationship status have similar scores on PTUS. Discussion Tinder users deserve the attention of scientific examination considering their large proportion among smartphone users. It is especially true considering the emerging trend of geolocated online dating applications. Conclusions Before PTUS, no prior scale has been created to measure problematic Tinder use. The PTUS is a suitable and reliable measure to assess problematic Tinder use
Bow shocks in water fountain jets
We briefly introduce the VLBI maser astrometric analysis of IRAS 18043-2116
and IRAS 18113-2503, two remarkable and unusual water fountains with
spectacular bipolar bow shocks in their high-speed collimated jet-driven
outflows. The 22 GHz H2O maser structures and velocities clearly show that the
jets are formed in very short-lived, episodic outbursts, which may indicate
episodic accretion in an underlying binary system.Comment: To appear in the proceedings of the IAU Symposium 336: Astrophysical
Masers: Unlocking the Mysteries of the Universe (4-8 September 2017,
Cagliari, Italy) - IAU Proceedings Series, eds. A. Tarchi, M. J. Reid, and P.
Castangi
Coordination for Connected Automated Vehicles at Merging Roadways in Mixed Traffic Environment
In this paper, we present a two-level optimal control framework to address
motion coordination of connected automated vehicles (CAVs) in the presence of
human-driven vehicles (HDVs) in merging scenarios. Our framework combines an
unconstrained trajectory solution of a low-level energy-optimal control problem
with an upper-level optimization problem that yields the minimum travel time
for CAVs. We predict the future trajectories of the HDVs using Newell's
car-following model. To handle potential deviations of HDVs' actual behavior
from the one predicted, we provide a risk-triggered re-planning mechanism for
the CAVs based on time-to-conflict. The effectiveness of the proposed control
framework is demonstrated via simulations with heterogeneous human driving
behaviors and via experiments in a scaled environment.Comment: first manuscript, 7 page
- …