10 research outputs found
Integrating Learning and Reasoning with Deep Logic Models
Deep learning is very effective at jointly learning feature representations
and classification models, especially when dealing with high dimensional input
patterns. Probabilistic logic reasoning, on the other hand, is capable to take
consistent and robust decisions in complex environments. The integration of
deep learning and logic reasoning is still an open-research problem and it is
considered to be the key for the development of real intelligent agents. This
paper presents Deep Logic Models, which are deep graphical models integrating
deep learning and logic reasoning both for learning and inference. Deep Logic
Models create an end-to-end differentiable architecture, where deep learners
are embedded into a network implementing a continuous relaxation of the logic
knowledge. The learning process allows to jointly learn the weights of the deep
learners and the meta-parameters controlling the high-level reasoning. The
experimental results show that the proposed methodology overtakes the
limitations of the other approaches that have been proposed to bridge deep
learning and reasoning
Teaching the old dog new tricks: Supervised learning with constraints
Methods for taking into account external knowledge in Machine Learning models have the potential to address outstanding issues in data-driven AI methods, such as improving safety and fairness, and can simplify training in the presence of scarce data. We propose a simple, but effective, method for injecting constraints at training time in supervised learning, based on decomposition and bi-level optimization: a master step is in charge of enforcing the constraints, while a learner step takes care of training the model. The process leads to approximate constraint satisfaction. The method is applicable to any ML approach for which the concept of label (or target) is well defined (most regression and classification scenarios), and allows to reuse existing training algorithms with no modifications. We require no assumption on the constraints, although their properties affect the shape and complexity of the master problem. Convergence guarantees are hard to provide, but we found that the approach performs well on ML tasks with fairness constraints and on classical datasets with synthetic constraints
T-Norms Driven Loss Functions for Machine Learning
Neural-symbolic approaches have recently gained popularity to inject prior
knowledge into a learner without requiring it to induce this knowledge from
data. These approaches can potentially learn competitive solutions with a
significant reduction of the amount of supervised data. A large class of
neural-symbolic approaches is based on First-Order Logic to represent prior
knowledge, relaxed to a differentiable form using fuzzy logic. This paper shows
that the loss function expressing these neural-symbolic learning tasks can be
unambiguously determined given the selection of a t-norm generator. When
restricted to supervised learning, the presented theoretical apparatus provides
a clean justification to the popular cross-entropy loss, which has been shown
to provide faster convergence and to reduce the vanishing gradient problem in
very deep structures. However, the proposed learning formulation extends the
advantages of the cross-entropy loss to the general knowledge that can be
represented by a neural-symbolic method. Therefore, the methodology allows the
development of a novel class of loss functions, which are shown in the
experimental results to lead to faster convergence rates than the approaches
previously proposed in the literature
From Statistical Relational to Neurosymbolic Artificial Intelligence: a Survey
This survey explores the integration of learning and reasoning in two
different fields of artificial intelligence: neurosymbolic and statistical
relational artificial intelligence. Neurosymbolic artificial intelligence
(NeSy) studies the integration of symbolic reasoning and neural networks, while
statistical relational artificial intelligence (StarAI) focuses on integrating
logic with probabilistic graphical models. This survey identifies seven shared
dimensions between these two subfields of AI. These dimensions can be used to
characterize different NeSy and StarAI systems. They are concerned with (1) the
approach to logical inference, whether model or proof-based; (2) the syntax of
the used logical theories; (3) the logical semantics of the systems and their
extensions to facilitate learning; (4) the scope of learning, encompassing
either parameter or structure learning; (5) the presence of symbolic and
subsymbolic representations; (6) the degree to which systems capture the
original logic, probabilistic, and neural paradigms; and (7) the classes of
learning tasks the systems are applied to. By positioning various NeSy and
StarAI systems along these dimensions and pointing out similarities and
differences between them, this survey contributes fundamental concepts for
understanding the integration of learning and reasoning.Comment: To appear in Artificial Intelligence. Shorter version at IJCAI 2020
survey track, https://www.ijcai.org/proceedings/2020/0688.pd