12,506 research outputs found
TensorLog: A Differentiable Deductive Database
Large knowledge bases (KBs) are useful in many tasks, but it is unclear how
to integrate this sort of knowledge into "deep" gradient-based learning
systems. To address this problem, we describe a probabilistic deductive
database, called TensorLog, in which reasoning uses a differentiable process.
In TensorLog, each clause in a logical theory is first converted into certain
type of factor graph. Then, for each type of query to the factor graph, the
message-passing steps required to perform belief propagation (BP) are
"unrolled" into a function, which is differentiable. We show that these
functions can be composed recursively to perform inference in non-trivial
logical theories containing multiple interrelated clauses and predicates. Both
compilation and inference in TensorLog are efficient: compilation is linear in
theory size and proof depth, and inference is linear in database size and the
number of message-passing steps used in BP. We also present experimental
results with TensorLog and discuss its relationship to other first-order
probabilistic logics
Uncertainty Aware AI ML: Why and How
This paper argues the need for research to realize uncertainty-aware
artificial intelligence and machine learning (AI\&ML) systems for decision
support by describing a number of motivating scenarios. Furthermore, the paper
defines uncertainty-awareness and lays out the challenges along with surveying
some promising research directions. A theoretical demonstration illustrates how
two emerging uncertainty-aware ML and AI technologies could be integrated and
be of value for a route planning operation.Comment: Presented at AAAI FSS-18: Artificial Intelligence in Government and
Public Sector, Arlington, Virginia, US
Learning Relational Representations with Auto-encoding Logic Programs
Deep learning methods capable of handling relational data have proliferated
over the last years. In contrast to traditional relational learning methods
that leverage first-order logic for representing such data, these deep learning
methods aim at re-representing symbolic relational data in Euclidean spaces.
They offer better scalability, but can only numerically approximate relational
structures and are less flexible in terms of reasoning tasks supported. This
paper introduces a novel framework for relational representation learning that
combines the best of both worlds. This framework, inspired by the auto-encoding
principle, uses first-order logic as a data representation language, and the
mapping between the original and latent representation is done by means of
logic programs instead of neural networks. We show how learning can be cast as
a constraint optimisation problem for which existing solvers can be used. The
use of logic as a representation language makes the proposed framework more
accurate (as the representation is exact, rather than approximate), more
flexible, and more interpretable than deep learning methods. We experimentally
show that these latent representations are indeed beneficial in relational
learning tasks.Comment: 8 pages,4 figures, paper + supplement, published at IJCA
Simple, Distributed, and Accelerated Probabilistic Programming
We describe a simple, low-level approach for embedding probabilistic
programming in a deep learning ecosystem. In particular, we distill
probabilistic programming down to a single abstraction---the random variable.
Our lightweight implementation in TensorFlow enables numerous applications: a
model-parallel variational auto-encoder (VAE) with 2nd-generation tensor
processing units (TPUv2s); a data-parallel autoregressive model (Image
Transformer) with TPUv2s; and multi-GPU No-U-Turn Sampler (NUTS). For both a
state-of-the-art VAE on 64x64 ImageNet and Image Transformer on 256x256
CelebA-HQ, our approach achieves an optimal linear speedup from 1 to 256 TPUv2
chips. With NUTS, we see a 100x speedup on GPUs over Stan and 37x over PyMC3.Comment: Appears in Neural Information Processing Systems, 2018. Code
available at http://bit.ly/2JpFip
Neural Logic Machines
We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for
both inductive learning and logic reasoning. NLMs exploit the power of both
neural networks---as function approximators, and logic programming---as a
symbolic processor for objects with properties, relations, logic connectives,
and quantifiers. After being trained on small-scale tasks (such as sorting
short arrays), NLMs can recover lifted rules, and generalize to large-scale
tasks (such as sorting longer arrays). In our experiments, NLMs achieve perfect
generalization in a number of tasks, from relational reasoning tasks on the
family tree and general graphs, to decision making tasks including sorting
arrays, finding shortest paths, and playing the blocks world. Most of these
tasks are hard to accomplish for neural networks or inductive logic programming
alone.Comment: ICLR 2019. Project page:
https://sites.google.com/view/neural-logic-machine
SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver
Integrating logical reasoning within deep learning architectures has been a
major goal of modern AI systems. In this paper, we propose a new direction
toward this goal by introducing a differentiable (smoothed) maximum
satisfiability (MAXSAT) solver that can be integrated into the loop of larger
deep learning systems. Our (approximate) solver is based upon a fast coordinate
descent approach to solving the semidefinite program (SDP) associated with the
MAXSAT problem. We show how to analytically differentiate through the solution
to this SDP and efficiently solve the associated backward pass. We demonstrate
that by integrating this solver into end-to-end learning systems, we can learn
the logical structure of challenging problems in a minimally supervised
fashion. In particular, we show that we can learn the parity function using
single-bit supervision (a traditionally hard task for deep networks) and learn
how to play 9x9 Sudoku solely from examples. We also solve a "visual Sudok"
problem that maps images of Sudoku puzzles to their associated logical
solutions by combining our MAXSAT solver with a traditional convolutional
architecture. Our approach thus shows promise in integrating logical structures
within deep learning.Comment: Accepted at ICML'19. The code can be found at
https://github.com/locuslab/satne
Hinge-Loss Markov Random Fields and Probabilistic Soft Logic
A fundamental challenge in developing high-impact machine learning
technologies is balancing the need to model rich, structured domains with the
ability to scale to big data. Many important problem areas are both richly
structured and large scale, from social and biological networks, to knowledge
graphs and the Web, to images, video, and natural language. In this paper, we
introduce two new formalisms for modeling structured data, and show that they
can both capture rich structure and scale to big data. The first, hinge-loss
Markov random fields (HL-MRFs), is a new kind of probabilistic graphical model
that generalizes different approaches to convex inference. We unite three
approaches from the randomized algorithms, probabilistic graphical models, and
fuzzy logic communities, showing that all three lead to the same inference
objective. We then define HL-MRFs by generalizing this unified objective. The
second new formalism, probabilistic soft logic (PSL), is a probabilistic
programming language that makes HL-MRFs easy to define using a syntax based on
first-order logic. We introduce an algorithm for inferring most-probable
variable assignments (MAP inference) that is much more scalable than
general-purpose convex optimization methods, because it uses message passing to
take advantage of sparse dependency structures. We then show how to learn the
parameters of HL-MRFs. The learned HL-MRFs are as accurate as analogous
discrete models, but much more scalable. Together, these algorithms enable
HL-MRFs and PSL to model rich, structured data at scales not previously
possible
Incremental Knowledge Base Construction Using DeepDive
Populating a database with unstructured information is a long-standing
problem in industry and research that encompasses problems of extraction,
cleaning, and integration. Recent names used for this problem include dealing
with dark data and knowledge base construction (KBC). In this work, we describe
DeepDive, a system that combines database and machine learning ideas to help
develop KBC systems, and we present techniques to make the KBC process more
efficient. We observe that the KBC process is iterative, and we develop
techniques to incrementally produce inference results for KBC systems. We
propose two methods for incremental inference, based respectively on sampling
and variational techniques. We also study the tradeoff space of these methods
and develop a simple rule-based optimizer. DeepDive includes all of these
contributions, and we evaluate DeepDive on five KBC systems, showing that it
can speed up KBC inference tasks by up to two orders of magnitude with
negligible impact on quality
Probabilistic Logic Programming with Beta-Distributed Random Variables
We enable aProbLog---a probabilistic logical programming approach---to reason
in presence of uncertain probabilities represented as Beta-distributed random
variables. We achieve the same performance of state-of-the-art algorithms for
highly specified and engineered domains, while simultaneously we maintain the
flexibility offered by aProbLog in handling complex relational domains. Our
motivation is that faithfully capturing the distribution of probabilities is
necessary to compute an expected utility for effective decision making under
uncertainty: unfortunately, these probability distributions can be highly
uncertain due to sparse data. To understand and accurately manipulate such
probability distributions we need a well-defined theoretical framework that is
provided by the Beta distribution, which specifies a distribution of
probabilities representing all the possible values of a probability when the
exact value is unknown.Comment: Accepted for presentation at AAAI 201
Exploiting Errors for Efficiency: A Survey from Circuits to Algorithms
When a computational task tolerates a relaxation of its specification or when
an algorithm tolerates the effects of noise in its execution, hardware,
programming languages, and system software can trade deviations from correct
behavior for lower resource usage. We present, for the first time, a synthesis
of research results on computing systems that only make as many errors as their
users can tolerate, from across the disciplines of computer aided design of
circuits, digital system design, computer architecture, programming languages,
operating systems, and information theory.
Rather than over-provisioning resources at each layer to avoid errors, it can
be more efficient to exploit the masking of errors occurring at one layer which
can prevent them from propagating to a higher layer. We survey tradeoffs for
individual layers of computing systems from the circuit level to the operating
system level and illustrate the potential benefits of end-to-end approaches
using two illustrative examples. To tie together the survey, we present a
consistent formalization of terminology, across the layers, which does not
significantly deviate from the terminology traditionally used by research
communities in their layer of focus.Comment: 35 page
- …