362 research outputs found
Neural disjunctive normal form: Vertically integrating logic with deep learning for classification
Inspired by the limitations of pure deep learning and symbolic logic-based models, in this thesis we consider a specific type of neuro-symbolic integration called vertical integration to bridge logic reasoning and deep learning and address their limitations. The motivation of vertical integration is to combine perception and reasoning as two separate stages of computation, while still being able to utilize simple and efficient end-to-end learning. It uses a perceptive deep neural network (DNN) to learn abstract concepts from raw sensory data and uses a symbolic model that operates on these abstract concepts to make interpretable predictions. As a preliminary step towards this direction, we tackle the task of binary classification and propose the Neural Disjunctive Normal Form (Neural DNF). Specifically, we utilize a per- ceptive DNN module to extract features from data, then after binarization (0 or 1), feed them into a Disjunctive Normal Form (DNF) module to perform logical rule-based classi- fication. We introduce the BOAT algorithm to optimize these two normally-incompatible modules in an end-to-end manner. Compared to standard DNF, Neural DNF can handle prediction tasks from raw sensory data (such as images) thanks to the neurally-extracted concepts. Compared to standard DNN, Neural DNF offers improved interpretability via an explicit symbolic representation while being able to achieve comparable accuracy despite the reduction of model flexibility, and is particularly suited for certain classification tasks that require some logical composition. Our experiments show that BOAT can optimize Neural DNF in an end-to-end manner, i.e. jointly learn the logical rules and concepts from scratch, and that in certain cases the rules and the meanings of concepts are aligned with human understanding. We view Neural DNF as an important first step towards more sophisticated vertical inte- gration models, which use symbolic models of more powerful rule languages for advanced prediction and algorithmic tasks, beyond using DNF (propositional logic) for classification tasks. The BOAT algorithm introduced in this thesis can potentially be applied to such advanced hybrid models
Multi-Modal Answer Validation for Knowledge-Based VQA
The problem of knowledge-based visual question answering involves answering
questions that require external knowledge in addition to the content of the
image. Such knowledge typically comes in various forms, including visual,
textual, and commonsense knowledge. Using more knowledge sources increases the
chance of retrieving more irrelevant or noisy facts, making it challenging to
comprehend the facts and find the answer. To address this challenge, we propose
Multi-modal Answer Validation using External knowledge (MAVEx), where the idea
is to validate a set of promising answer candidates based on answer-specific
knowledge retrieval. Instead of searching for the answer in a vast collection
of often irrelevant facts as most existing approaches do, MAVEx aims to learn
how to extract relevant knowledge from noisy sources, which knowledge source to
trust for each answer candidate, and how to validate the candidate using that
source. Our multi-modal setting is the first to leverage external visual
knowledge (images searched using Google), in addition to textual knowledge in
the form of Wikipedia sentences and ConceptNet concepts. Our experiments with
OK-VQA, a challenging knowledge-based VQA dataset, demonstrate that MAVEx
achieves new state-of-the-art results. Our code is available at
https://github.com/jialinwu17/MAVEXComment: AAAI 202
Subgroup Discovery in Unstructured Data
Subgroup discovery is a descriptive and exploratory data mining technique to
identify subgroups in a population that exhibit interesting behavior with
respect to a variable of interest. Subgroup discovery has numerous applications
in knowledge discovery and hypothesis generation, yet it remains inapplicable
for unstructured, high-dimensional data such as images. This is because
subgroup discovery algorithms rely on defining descriptive rules based on
(attribute, value) pairs, however, in unstructured data, an attribute is not
well defined. Even in cases where the notion of attribute intuitively exists in
the data, such as a pixel in an image, due to the high dimensionality of the
data, these attributes are not informative enough to be used in a rule. In this
paper, we introduce the subgroup-aware variational autoencoder, a novel
variational autoencoder that learns a representation of unstructured data which
leads to subgroups with higher quality. Our experimental results demonstrate
the effectiveness of the method at learning subgroups with high quality while
supporting the interpretability of the concepts
Human activities accelerated the degradation of saline seepweed red beaches by amplifying topâdown and bottomâup forces
Salt marshes dominated by saline seepweed (Suaeda heteroptera) provide important ecosystem services such as sequestering carbon (blue carbon), maintaining healthy fisheries, and protecting shorelines. These salt marshes also constitute stunning red beach landscapes, and the resulting tourism significantly contributes to the local economy. However, land use change and degradation have led to a substantial loss of the red beach area. It remains unclear how human activities influence the topâdown and bottomâup forces that regulate the distribution and succession of these salt marshes and lead to the degradation of the red beaches. We examined how bottomâup forces influenced the germination, emergence, and colonization of saline seepweed with field measurements and a laboratory experiment. We also examined whether topâdown forces affected the red beach distribution by conducting a field survey for crab burrows and density, laboratory feeding trials, and waterbird investigations. The higher sediment accretion rate induced by human activities limited the establishment of new red beaches. The construction of tourism facilities and the frequent presence of tourists reduced the density of waterbirds, which in turn increased the density of crabs, intensifying the topâdown forces such as predators and herbivores that drive the degradation of the coastal red beaches. Our results show that sediment accretion and plantâherbivory changes induced by human activities were likely the two primary ecological processes leading to the degradation of the red beaches. Human activities significantly shaped the abundance and distribution of the red beaches by altering both topâdown and bottomâup ecological processes. Our findings can help us better understand the dynamics of salt marshes and have implications for the management and restoration of coastal wetlands
A manometric feature descriptor with linear-SVM to distinguish esophageal contraction vigor
n clinical, if a patient presents with nonmechanical obstructive dysphagia,
esophageal chest pain, and gastro esophageal reflux symptoms, the physician
will usually assess the esophageal dynamic function. High-resolution manometry
(HRM) is a clinically commonly used technique for detection of esophageal
dynamic function comprehensively and objectively. However, after the results of
HRM are obtained, doctors still need to evaluate by a variety of parameters.
This work is burdensome, and the process is complex. We conducted image
processing of HRM to predict the esophageal contraction vigor for assisting the
evaluation of esophageal dynamic function. Firstly, we used Feature-Extraction
and Histogram of Gradients (FE-HOG) to analyses feature of proposal of swallow
(PoS) to further extract higher-order features. Then we determine the
classification of esophageal contraction vigor normal, weak and failed by using
linear-SVM according to these features. Our data set includes 3000 training
sets, 500 validation sets and 411 test sets. After verification our accuracy
reaches 86.83%, which is higher than other common machine learning methods
- âŠ