1,545 research outputs found
Analyzing Differentiable Fuzzy Implications
Combining symbolic and neural approaches has gained considerable attention in
the AI community, as it is often argued that the strengths and weaknesses of
these approaches are complementary. One such trend in the literature are weakly
supervised learning techniques that employ operators from fuzzy logics. In
particular, they use prior background knowledge described in such logics to
help the training of a neural network from unlabeled and noisy data. By
interpreting logical symbols using neural networks (or grounding them), this
background knowledge can be added to regular loss functions, hence making
reasoning a part of learning.
In this paper, we investigate how implications from the fuzzy logic
literature behave in a differentiable setting. In such a setting, we analyze
the differences between the formal properties of these fuzzy implications. It
turns out that various fuzzy implications, including some of the most
well-known, are highly unsuitable for use in a differentiable learning setting.
A further finding shows a strong imbalance between gradients driven by the
antecedent and the consequent of the implication. Furthermore, we introduce a
new family of fuzzy implications (called sigmoidal implications) to tackle this
phenomenon. Finally, we empirically show that it is possible to use
Differentiable Fuzzy Logics for semi-supervised learning, and show that
sigmoidal implications outperform other choices of fuzzy implications.Comment: 10 pages, 10 figures, accepted to 17th International Conference on
Principles of Knowledge Representation and Reasoning (KR 2020). arXiv admin
note: substantial text overlap with arXiv:2002.0610
Reduced Implication-bias Logic Loss for Neuro-Symbolic Learning
Integrating logical reasoning and machine learning by approximating logical
inference with differentiable operators is a widely used technique in
Neuro-Symbolic systems.
However, some differentiable operators could bring a significant bias during
backpropagation and degrade the performance of Neuro-Symbolic learning.
In this paper, we reveal that this bias, named \textit{Implication Bias} is
common in loss functions derived from fuzzy logic operators.
Furthermore, we propose a simple yet effective method to transform the biased
loss functions into \textit{Reduced Implication-bias Logic Loss (RILL)} to
address the above problem.
Empirical study shows that RILL can achieve significant improvements compared
with the biased logic loss functions, especially when the knowledge base is
incomplete, and keeps more robust than the compared methods when labelled data
is insufficient.Comment: ACML'2023 Journal Track(Accepted by Machine Learning Journal
Differentiable Logics for Neural Network Training and Verification
The rising popularity of neural networks (NNs) in recent years and their
increasing prevalence in real-world applications have drawn attention to the
importance of their verification. While verification is known to be
computationally difficult theoretically, many techniques have been proposed for
solving it in practice. It has been observed in the literature that by default
neural networks rarely satisfy logical constraints that we want to verify. A
good course of action is to train the given NN to satisfy said constraint prior
to verifying them. This idea is sometimes referred to as continuous
verification, referring to the loop between training and verification. Usually
training with constraints is implemented by specifying a translation for a
given formal logic language into loss functions. These loss functions are then
used to train neural networks. Because for training purposes these functions
need to be differentiable, these translations are called differentiable logics
(DL). This raises several research questions. What kind of differentiable
logics are possible? What difference does a specific choice of DL make in the
context of continuous verification? What are the desirable criteria for a DL
viewed from the point of view of the resulting loss function? In this extended
abstract we will discuss and answer these questions.Comment: FOMLAS'22 pape
Grounding LTLf specifications in images
A critical challenge in neurosymbolic approaches is to handle the symbol grounding problem without direct supervision. That is mapping high-dimensional raw data into an interpretation over a finite set of abstract concepts with a known meaning, without using labels. In this work, we ground symbols into sequences of images by exploiting symbolic logical knowledge in the form of Linear Temporal Logic over finite traces (LTLf) formulas, and sequence-level labels expressing if a sequence of images is compliant or not with the given formula. Our approach is based on translating the LTLf formula into an equivalent
deterministic finite automaton (DFA) and interpreting the latter in fuzzy logic. Experiments show that our system outperforms recurrent neural networks in sequence classification and can reach high image classification accuracy without being trained with any single-image label
Deep Learning with Logical Constraints
In recent years, there has been an increasing interest in exploiting
logically specified background knowledge in order to obtain neural models (i)
with a better performance, (ii) able to learn from less data, and/or (iii)
guaranteed to be compliant with the background knowledge itself, e.g., for
safety-critical applications. In this survey, we retrace such works and
categorize them based on (i) the logical language that they use to express the
background knowledge and (ii) the goals that they achieve.Comment: Survey paper. IJCAI 202
- …