176 research outputs found
Learning Logistic Circuits
This paper proposes a new classification model called logistic circuits. On
MNIST and Fashion datasets, our learning algorithm outperforms neural networks
that have an order of magnitude more parameters. Yet, logistic circuits have a
distinct origin in symbolic AI, forming a discriminative counterpart to
probabilistic-logical circuits such as ACs, SPNs, and PSDDs. We show that
parameter learning for logistic circuits is convex optimization, and that a
simple local search algorithm can induce strong model structures from data.Comment: Published in the Proceedings of the Thirty-Third AAAI Conference on
Artificial Intelligence (AAAI19
RulE: Neural-Symbolic Knowledge Graph Reasoning with Rule Embedding
Knowledge graph (KG) reasoning is an important problem for knowledge graphs.
It predicts missing links by reasoning on existing facts. Knowledge graph
embedding (KGE) is one of the most popular methods to address this problem. It
embeds entities and relations into low-dimensional vectors and uses the learned
entity/relation embeddings to predict missing facts. However, KGE only uses
zeroth-order (propositional) logic to encode existing triplets (e.g., ``Alice
is Bob's wife."); it is unable to leverage first-order (predicate) logic to
represent generally applicable logical \textbf{rules} (e.g., ``''). On the other hand, traditional rule-based KG reasoning methods
usually rely on hard logical rule inference, making it brittle and hardly
competitive with KGE. In this paper, we propose RulE, a novel and principled
framework to represent and model logical rules and triplets. RulE jointly
represents entities, relations and logical rules in a unified embedding space.
By learning an embedding for each logical rule, RulE can perform logical rule
inference in a soft way and give a confidence score to each grounded rule,
similar to how KGE gives each triplet a confidence score. Compared to KGE
alone, RulE allows injecting prior logical rule information into the embedding
space, which improves the generalization of knowledge graph embedding. Besides,
the learned confidence scores of rules improve the logical rule inference
process by softly controlling the contribution of each rule, which alleviates
the brittleness of logic. We evaluate our method with link prediction tasks.
Experimental results on multiple benchmark KGs demonstrate the effectiveness of
RulE
Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction
We study the problem of learning goal-conditioned policies in Minecraft, a
popular, widely accessible yet challenging open-ended environment for
developing human-level multi-task agents. We first identify two main challenges
of learning such policies: 1) the indistinguishability of tasks from the state
distribution, due to the vast scene diversity, and 2) the non-stationary nature
of environment dynamics caused by partial observability. To tackle the first
challenge, we propose Goal-Sensitive Backbone (GSB) for the policy to encourage
the emergence of goal-relevant visual state representations. To tackle the
second challenge, the policy is further fueled by an adaptive horizon
prediction module that helps alleviate the learning uncertainty brought by the
non-stationary dynamics. Experiments on 20 Minecraft tasks show that our method
significantly outperforms the best baseline so far; in many of them, we double
the performance. Our ablation and exploratory studies then explain how our
approach beat the counterparts and also unveil the surprising bonus of
zero-shot generalization to new scenes (biomes). We hope our agent could help
shed some light on learning goal-conditioned, multi-task agents in challenging,
open-ended environments like Minecraft.Comment: This paper is accepted by CVPR202
Efficient Meta Reinforcement Learning for Preference-based Fast Adaptation
Learning new task-specific skills from a few trials is a fundamental
challenge for artificial intelligence. Meta reinforcement learning (meta-RL)
tackles this problem by learning transferable policies that support few-shot
adaptation to unseen tasks. Despite recent advances in meta-RL, most existing
methods require the access to the environmental reward function of new tasks to
infer the task objective, which is not realistic in many practical
applications. To bridge this gap, we study the problem of few-shot adaptation
in the context of human-in-the-loop reinforcement learning. We develop a
meta-RL algorithm that enables fast policy adaptation with preference-based
feedback. The agent can adapt to new tasks by querying human's preference
between behavior trajectories instead of using per-step numeric rewards. By
extending techniques from information theory, our approach can design query
sequences to maximize the information gain from human interactions while
tolerating the inherent error of non-expert human oracle. In experiments, we
extensively evaluate our method, Adaptation with Noisy OracLE (ANOLE), on a
variety of meta-RL benchmark tasks and demonstrate substantial improvement over
baseline algorithms in terms of both feedback efficiency and error tolerance.Comment: Thirty-sixth Conference on Neural Information Processing Systems
(NeurIPS 2022
- …