551 research outputs found
Learning non-Markovian Decision-Making from State-only Sequences
Conventional imitation learning assumes access to the actions of
demonstrators, but these motor signals are often non-observable in naturalistic
settings. Additionally, sequential decision-making behaviors in these settings
can deviate from the assumptions of a standard Markov Decision Process (MDP).
To address these challenges, we explore deep generative modeling of state-only
sequences with non-Markov Decision Process (nMDP), where the policy is an
energy-based prior in the latent space of the state transition generator. We
develop maximum likelihood estimation to achieve model-based imitation, which
involves short-run MCMC sampling from the prior and importance sampling for the
posterior. The learned model enables \textit{decision-making as inference}:
model-free policy execution is equivalent to prior sampling, model-based
planning is posterior sampling initialized from the policy. We demonstrate the
efficacy of the proposed method in a prototypical path planning task with
non-Markovian constraints and show that the learned model exhibits strong
performances in challenging domains from the MuJoCo suite
UV R-CNN: Stable and Efficient Dense Human Pose Estimation
Dense pose estimation is a dense 3D prediction task for instance-level human
analysis, aiming to map human pixels from an RGB image to a 3D surface of the
human body. Due to a large amount of surface point regression, the training
process appears to be easy to collapse compared to other region-based human
instance analyzing tasks. By analyzing the loss formulation of the existing
dense pose estimation model, we introduce a novel point regression loss
function, named Dense Points} loss to stable the training progress, and a new
balanced loss weighting strategy to handle the multi-task losses. With the
above novelties, we propose a brand new architecture, named UV R-CNN. Without
auxiliary supervision and external knowledge from other tasks, UV R-CNN can
handle many complicated issues in dense pose model training progress, achieving
65.0% and 66.1% on the DensePose-COCO validation subset
with ResNet-50-FPN feature extractor, competitive among the state-of-the-art
dense human pose estimation methods.Comment: 9pages, 4 figure
SMART: A Situation Model for Algebra Story Problems via Attributed Grammar
Solving algebra story problems remains a challenging task in artificial
intelligence, which requires a detailed understanding of real-world situations
and a strong mathematical reasoning capability. Previous neural solvers of math
word problems directly translate problem texts into equations, lacking an
explicit interpretation of the situations, and often fail to handle more
sophisticated situations. To address such limits of neural solvers, we
introduce the concept of a \emph{situation model}, which originates from
psychology studies to represent the mental states of humans in problem-solving,
and propose \emph{SMART}, which adopts attributed grammar as the representation
of situation models for algebra story problems. Specifically, we first train an
information extraction module to extract nodes, attributes, and relations from
problem texts and then generate a parse graph based on a pre-defined attributed
grammar. An iterative learning strategy is also proposed to improve the
performance of SMART further. To rigorously study this task, we carefully
curate a new dataset named \emph{ASP6.6k}. Experimental results on ASP6.6k show
that the proposed model outperforms all previous neural solvers by a large
margin while preserving much better interpretability. To test these models'
generalization capability, we also design an out-of-distribution (OOD)
evaluation, in which problems are more complex than those in the training set.
Our model exceeds state-of-the-art models by 17\% in the OOD evaluation,
demonstrating its superior generalization ability
A HINT from Arithmetic: On Systematic Generalization of Perception, Syntax, and Semantics
Inspired by humans' remarkable ability to master arithmetic and generalize to
unseen problems, we present a new dataset, HINT, to study machines' capability
of learning generalizable concepts at three different levels: perception,
syntax, and semantics. In particular, concepts in HINT, including both digits
and operators, are required to learn in a weakly-supervised fashion: Only the
final results of handwriting expressions are provided as supervision. Learning
agents need to reckon how concepts are perceived from raw signals such as
images (i.e., perception), how multiple concepts are structurally combined to
form a valid expression (i.e., syntax), and how concepts are realized to afford
various reasoning tasks (i.e., semantics). With a focus on systematic
generalization, we carefully design a five-fold test set to evaluate both the
interpolation and the extrapolation of learned concepts. To tackle this
challenging problem, we propose a neural-symbolic system by integrating neural
networks with grammar parsing and program synthesis, learned by a novel
deduction--abduction strategy. In experiments, the proposed neural-symbolic
system demonstrates strong generalization capability and significantly
outperforms end-to-end neural methods like RNN and Transformer. The results
also indicate the significance of recursive priors for extrapolation on syntax
and semantics.Comment: Preliminary wor
Neural-Symbolic Recursive Machine for Systematic Generalization
Despite the tremendous success, existing machine learning models still fall
short of human-like systematic generalization -- learning compositional rules
from limited data and applying them to unseen combinations in various domains.
We propose Neural-Symbolic Recursive Machine (NSR) to tackle this deficiency.
The core representation of NSR is a Grounded Symbol System (GSS) with
combinatorial syntax and semantics, which entirely emerges from training data.
Akin to the neuroscience studies suggesting separate brain systems for
perceptual, syntactic, and semantic processing, NSR implements analogous
separate modules of neural perception, syntactic parsing, and semantic
reasoning, which are jointly learned by a deduction-abduction algorithm. We
prove that NSR is expressive enough to model various sequence-to-sequence
tasks. Superior systematic generalization is achieved via the inductive biases
of equivariance and recursiveness embedded in NSR. In experiments, NSR achieves
state-of-the-art performance in three benchmarks from different domains: SCAN
for semantic parsing, PCFG for string manipulation, and HINT for arithmetic
reasoning. Specifically, NSR achieves 100% generalization accuracy on SCAN and
PCFG and outperforms state-of-the-art models on HINT by about 23%. Our NSR
demonstrates stronger generalization than pure neural networks due to its
symbolic representation and inductive biases. NSR also demonstrates better
transferability than existing neural-symbolic approaches due to less
domain-specific knowledge required
- …