16 research outputs found
Active Perception in Adversarial Scenarios using Maximum Entropy Deep Reinforcement Learning
We pose an active perception problem where an autonomous agent actively
interacts with a second agent with potentially adversarial behaviors. Given the
uncertainty in the intent of the other agent, the objective is to collect
further evidence to help discriminate potential threats. The main technical
challenges are the partial observability of the agent intent, the adversary
modeling, and the corresponding uncertainty modeling. Note that an adversary
agent may act to mislead the autonomous agent by using a deceptive strategy
that is learned from past experiences. We propose an approach that combines
belief space planning, generative adversary modeling, and maximum entropy
reinforcement learning to obtain a stochastic belief space policy. By
accounting for various adversarial behaviors in the simulation framework and
minimizing the predictability of the autonomous agent's action, the resulting
policy is more robust to unmodeled adversarial strategies. This improved
robustness is empirically shown against an adversary that adapts to and
exploits the autonomous agent's policy when compared with a standard
Chance-Constraint Partially Observable Markov Decision Process robust approach
The Impact of Road Configuration on V2V-based Cooperative Localization
Cooperative localization with map matching has been shown to reduce Global
Navigation Satellite System (GNSS) localization error from several meters to
sub-meter level by fusing the GNSS measurements of four vehicles in our
previous work. While further error reduction is expected to be achievable by
increasing the number of vehicles, the quantitative relationship between the
estimation error and the number of connected vehicles has neither been
systematically investigated nor analytically proved. In this work, a
theoretical study is presented that analytically proves the correlation between
the localization error and the number of connected vehicles in two cases of
practical interest. More specifically, it is shown that, under the assumption
of small non-common error, the expected square error of the GNSS common error
correction is inversely proportional to the number of vehicles, if the road
directions obey a uniform distribution, or inversely proportional to logarithm
of the number of vehicles, if the road directions obey a Bernoulli
distribution. Numerical simulations are conducted to justify these analytic
results. Moreover, the simulation results show that the aforementioned error
decrement rates hold even when the assumption of small non-common error is
violated
Transferable Pedestrian Motion Prediction Models at Intersections
One desirable capability of autonomous cars is to accurately predict the
pedestrian motion near intersections for safe and efficient trajectory
planning. We are interested in developing transfer learning algorithms that can
be trained on the pedestrian trajectories collected at one intersection and yet
still provide accurate predictions of the trajectories at another, previously
unseen intersection. We first discussed the feature selection for transferable
pedestrian motion models in general. Following this discussion, we developed
one transferable pedestrian motion prediction algorithm based on Inverse
Reinforcement Learning (IRL) that infers pedestrian intentions and predicts
future trajectories based on observed trajectory. We evaluated our algorithm on
a dataset collected at two intersections, trained at one intersection and
tested at the other intersection. We used the accuracy of augmented
semi-nonnegative sparse coding (ASNSC), trained and tested at the same
intersection as a baseline. The result shows that the proposed algorithm
improves the baseline accuracy by 40% in the non-transfer task, and 16% in the
transfer task
Scaling Up Multiagent Reinforcement Learning for Robotic Systems: Learn an Adaptive Sparse Communication Graph
The complexity of multiagent reinforcement learning (MARL) in multiagent
systems increases exponentially with respect to the agent number. This
scalability issue prevents MARL from being applied in large-scale multiagent
systems. However, one critical feature in MARL that is often neglected is that
the interactions between agents are quite sparse. Without exploiting this
sparsity structure, existing works aggregate information from all of the agents
and thus have a high sample complexity. To address this issue, we propose an
adaptive sparse attention mechanism by generalizing a sparsity-inducing
activation function. Then a sparse communication graph in MARL is learned by
graph neural networks based on this new attention mechanism. Through this
sparsity structure, the agents can communicate in an effective as well as
efficient way via only selectively attending to agents that matter the most and
thus the scale of the MARL problem is reduced with little optimality
compromised. Comparative results show that our algorithm can learn an
interpretable sparse structure and outperforms previous works by a significant
margin on applications involving a large-scale multiagent system