1,601 research outputs found
Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability
A trustworthy reinforcement learning algorithm should be competent in solving
challenging real-world problems, including {robustly} handling uncertainties,
satisfying {safety} constraints to avoid catastrophic failures, and
{generalizing} to unseen scenarios during deployments. This study aims to
overview these main perspectives of trustworthy reinforcement learning
considering its intrinsic vulnerabilities on robustness, safety, and
generalizability. In particular, we give rigorous formulations, categorize
corresponding methodologies, and discuss benchmarks for each perspective.
Moreover, we provide an outlook section to spur promising future directions
with a brief discussion on extrinsic vulnerabilities considering human
feedback. We hope this survey could bring together separate threads of studies
together in a unified framework and promote the trustworthiness of
reinforcement learning.Comment: 36 pages, 5 figure
Human Motion Trajectory Prediction: A Survey
With growing numbers of intelligent autonomous systems in human environments,
the ability of such systems to perceive, understand and anticipate human
behavior becomes increasingly important. Specifically, predicting future
positions of dynamic agents and planning considering such predictions are key
tasks for self-driving vehicles, service robots and advanced surveillance
systems. This paper provides a survey of human motion trajectory prediction. We
review, analyze and structure a large selection of work from different
communities and propose a taxonomy that categorizes existing methods based on
the motion modeling approach and level of contextual information used. We
provide an overview of the existing datasets and performance metrics. We
discuss limitations of the state of the art and outline directions for further
research.Comment: Submitted to the International Journal of Robotics Research (IJRR),
37 page
Neurosymbolic Reinforcement Learning and Planning: A Survey
The area of Neurosymbolic Artificial Intelligence (Neurosymbolic AI) is
rapidly developing and has become a popular research topic, encompassing
sub-fields such as Neurosymbolic Deep Learning (Neurosymbolic DL) and
Neurosymbolic Reinforcement Learning (Neurosymbolic RL). Compared to
traditional learning methods, Neurosymbolic AI offers significant advantages by
simplifying complexity and providing transparency and explainability.
Reinforcement Learning(RL), a long-standing Artificial Intelligence(AI) concept
that mimics human behavior using rewards and punishment, is a fundamental
component of Neurosymbolic RL, a recent integration of the two fields that has
yielded promising results. The aim of this paper is to contribute to the
emerging field of Neurosymbolic RL by conducting a literature survey. Our
evaluation focuses on the three components that constitute Neurosymbolic RL:
neural, symbolic, and RL. We categorize works based on the role played by the
neural and symbolic parts in RL, into three taxonomies:Learning for Reasoning,
Reasoning for Learning and Learning-Reasoning. These categories are further
divided into sub-categories based on their applications. Furthermore, we
analyze the RL components of each research work, including the state space,
action space, policy module, and RL algorithm. Additionally, we identify
research opportunities and challenges in various applications within this
dynamic field.Comment: 16 pages, 9 figures, IEEE Transactions on Artificial Intelligenc
Core Challenges in Embodied Vision-Language Planning
Recent advances in the areas of multimodal machine learning and artificial
intelligence (AI) have led to the development of challenging tasks at the
intersection of Computer Vision, Natural Language Processing, and Embodied AI.
Whereas many approaches and previous survey pursuits have characterised one or
two of these dimensions, there has not been a holistic analysis at the center
of all three. Moreover, even when combinations of these topics are considered,
more focus is placed on describing, e.g., current architectural methods, as
opposed to also illustrating high-level challenges and opportunities for the
field. In this survey paper, we discuss Embodied Vision-Language Planning
(EVLP) tasks, a family of prominent embodied navigation and manipulation
problems that jointly use computer vision and natural language. We propose a
taxonomy to unify these tasks and provide an in-depth analysis and comparison
of the new and current algorithmic approaches, metrics, simulated environments,
as well as the datasets used for EVLP tasks. Finally, we present the core
challenges that we believe new EVLP works should seek to address, and we
advocate for task construction that enables model generalizability and furthers
real-world deployment.Comment: 35 page
Assessment of a Cognitive-Motor Training Program in Adults at Increased Risk for Developing Dementia
With the prevalence of dementia increasing each year, preclinically implemented therapeutic interventions are critically needed. It has been suggested that cascading neural network failures may bring on behavioural deficits associated with Alzheimers disease. Previously we have shown that cognitive-motor integration (CMI) training in adults with mild cognitive impairments generalized to improved global cognitive and activities of daily living scores. Here we employ a novel movement-control based training approach involving CMI rather than traditional cognition-only brain training. We hypothesized that such training would stimulate widespread neural networks and enhance rule-based visuomotor ability in at-risk individuals. We observed a significant improvement in bimanual coordination in the at-risk training group. We also observed significant decreases in movement variability for the most complex CMI condition in the at-risk and healthy training groups. These data suggest that integrating cognition into action in a training intervention may be effective at strengthening vulnerable brain networks in asymptomatic adults at risk for developing dementia
Human-Guided Learning for Probabilistic Logic Models
Advice-giving has been long explored in the artificial intelligence community to build robust learning algorithms when the data is noisy, incorrect or even insufficient. While logic based systems were effectively used in building expert systems, the role of the human has been restricted to being a “mere labeler” in recent times. We hypothesize and demonstrate that probabilistic logic can provide an effective and natural way for the expert to specify domain advice. Specifically, we consider different types of advice-giving in relational domains where noise could arise due to systematic errors or class-imbalance inherent in the domains. The advice is provided as logical statements or privileged features that are thenexplicitly considered by an iterative learning algorithm at every update. Our empirical evidence shows that human advice can effectively accelerate learning in noisy, structured domains where so far humans have been merely used as labelers or as designers of the (initial or final) structure of the model
An Approach to Analyze Tradeoffs for Aerospace System Design and Operation
There are important tradeoffs that need to be considered for the design and operation of aerospace systems. In addition to tradeoffs, there may also be multiple stakeholders of interest to the system and each may have different preferences as to the balance amongst the tradeoffs under consideration. A tradeoff hyperspace is created when there are three or more tradeoff dimensions and this increases the challenge associated with resolving the hyperspace in order to determine the best design and operation of a system. The corresponding objectives of this research are to develop a framework to analyze tradeoff hyperspaces and to account for the preferences of multiple stakeholders in this framework.This work was supported by the National Aeronautics and Space Administration (NASA) under grant NRA- #NNX10AN92A (NASA Ames). The authors are grateful to Dr. Neil Y. Chen and Dr. Banavar Sridhar in the Aviation Systems Division at NASA Ames for their valuable guidance and feedback in managing this project
On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learning
Recently, unsupervised representation learning (URL) has improved the sample
efficiency of Reinforcement Learning (RL) by pretraining a model from a large
unlabeled dataset. The underlying principle of these methods is to learn
temporally predictive representations by predicting future states in the latent
space. However, an important challenge of this approach is the representational
collapse, where the subspace of the latent representations collapses into a
low-dimensional manifold. To address this issue, we propose a novel URL
framework that causally predicts future states while increasing the dimension
of the latent manifold by decorrelating the features in the latent space.
Through extensive empirical studies, we demonstrate that our framework
effectively learns predictive representations without collapse, which
significantly improves the sample efficiency of state-of-the-art URL methods on
the Atari 100k benchmark. The code is available at
https://github.com/dojeon-ai/SimTPR.Comment: Accepted to ICML 202
- …