DeepLogic: Towards end-to-end differentiable logical reasoning

Combining machine learning with logic-based expert systems in order to get the best of both worlds are becoming increasingly popular. However, to what extent machine learning can already learn to reason over rule-based knowledge is still an open problem. In this paper, we explore how symbolic logic, defined as logic programs at a character level, is learned to be represented in a high-dimensional vector space using RNN-based iterative neural networks to perform reasoning. We create a new dataset that defines 12 classes of logic programs exemplifying increased level of complexity of logical reasoning and train the networks in an end-to-end fashion to learn whether a logic program entails a given query. We analyse how learning the inference algorithm gives rise to representations of atoms, literals and rules within logic programs and evaluate against increasing lengths of predicate and constant symbols as well as increasing steps of multi-hop reasoning

Neuro-symbolic Rule Learning in Real-world Classification Tasks

Neuro-symbolic rule learning has attracted lots of attention as it offers better interpretability than pure neural models and scales better than symbolic rule learning. A recent approach named pix2rule proposes a neural Disjunctive Normal Form (neural DNF) module to learn symbolic rules with feed-forward layers. Although proved to be effective in synthetic binary classification, pix2rule has not been applied to more challenging tasks such as multi-label and multi-class classifications over real-world data. In this paper, we address this limitation by extending the neural DNF module to (i) support rule learning in real-world multi-class and multi-label classification tasks, (ii) enforce the symbolic property of mutual exclusivity (i.e. predicting exactly one class) in multi-class classification, and (iii) explore its scalability over large inputs and outputs. We train a vanilla neural DNF model similar to pix2rule's neural DNF module for multi-label classification, and we propose a novel extended model called neural DNF-EO (Exactly One) which enforces mutual exclusivity in multi-class classification. We evaluate the classification performance, scalability and interpretability of our neural DNF-based models, and compare them against pure neural models and a state-of-the-art symbolic rule learner named FastLAS. We demonstrate that our neural DNF-based models perform similarly to neural networks, but provide better interpretability by enabling the extraction of logical rules. Our models also scale well when the rule search space grows in size, in contrast to FastLAS, which fails to learn in multi-class classification tasks with 200 classes and in all multi-label settings.Comment: Accepted at AAAI-MAKE 202

End-to-end neuro-symbolic learning of logic-based inference

Artificial Intelligence has long taken the human mind as a point of inspiration and research. One remarkable feat of the human brain is its ability to seamlessly reconcile low-level sensory inputs such as vision with high-level abstract reasoning using symbols related to objects and rules. Inspired by this, neuro-symbolic computing attempts to bring together advances in connectionist architectures like artificial neural networks with principled symbolic inference of logic-based systems. How this integration between the two branches of research can be achieved remains an open question. In this thesis, we tackle neuro-symbolic inference in an end-to-end differentiable fashion from three different aspects: learning to perform symbolic deduction and manipulation over logic programs, the ability to learn and leverage variables through unification across data points and finally the ability to induce symbolic rules directly from non-symbolic inputs such as images. We first start by proposing a novel neural network model, Iterative Memory Attention (IMA), to ascertain the level of symbolic deduction and manipulation neural networks can achieve over logic programs of increased complexity. We demonstrate that our approach outperforms existing neural network models and analyse the vector representations learnt by our model. We observe that the principal components of the continuous real-valued embedding space align with the constructs of logic programs such as arity of predicates and types of rules. We then focus on a key component of symbolic inference: variables. Humans leverage variables in everyday reasoning to construct high level abstract rules such as “if someone went somewhere then they are there” instead of mentioning specific people or places. We present a novel end-to-end differentiable neural network architecture called Unification Network that is capable of recognising which symbols can act as variables through the application of soft unification. The by-products of the model are invariants that capture some common underlying principle present in the dataset. Unification Networks exhibit better data efficiency and generalisation to unseen examples compared to models that do not utilise soft unification. Finally, we redirect our attention to the question: How can a neural network learn symbolic rules directly from visual inputs in a coherent manner? We bridge the gap between continuous vector representations and discrete symbolic reasoning by presenting a fully differentiable layer in a deep learning architecture called the Semi-symbolic Layer. When stacked, the Semi-symbolic Layers within a larger model are able to learn complete logic programs along with continuous representations of image patches directly from pixel level input in an end-to-end fashion. The resulting model holistically learns objects, relations between them and logical rules. By pruning and thresholding the weights of the Semi-symbolic Layers, we can extract out the exact symbolic relations and rules used to reason about the tasks and verify them using symbolic inference engines. Using two datasets, we demonstrate that our approach scales better than existing state-of-the-art symbolic rule learning systems and outperforms previous deep relational neural network architectures.Open Acces

Tumor-free distance from outermost layer of cervix is of prognostic value in surgically treated cervical cancer patients: A multicenter study

PubMedID: 24435497Objective: This study aimed at determining if tumor-free distance (TFD) from outermost layer of cervix predicts surgicopathologic factors and outcome in surgically treated cervical cancer patients. Materials and methods: One hundred sixteen surgically treated cervical squamous cell carcinomas between 1991 and 2010 with FIGO stage IB/2A were identified and reevaluated histologically regarding the TFD. TFD was defined as the distance between outermost layer of cervix and deepest cervical stromal invasion. Depth of invasion (DOI) and TFD were expressed as continuous variables and compared with traditional surgicopathologic variables and survival to determine their prognostic significance. Results: The mean DOI was 10.3 mm and the mean TFD was 4.2 mm. The most common stage was IB1 (60 patients, 51.7%). The mean number of removed pelvic lymph nodes was 32.2 (median 30; range 8-78). Positive pelvic lymph nodes were found in 27 (23%) of the patients. Sixty-eight patients had lymphovascular space involvement (LVSI). Sixty-eight patients (59%) received postoperative radiotherapy where the following items were present: tumor diameter >4 cm, positive lymph nodes, LVSI and positive surgical margins. With the median fol-low- up of 53 months (3-219 months); 14 patients had local and 13 patients had distant metastases (5 of the patients had both at the time of recurrence). With logistic regression analysis, TFD was a predictor of pelvic lymph involvement ( p=0.028) and LVSI (p=0.008) while DOI was a predictor of LVSI (p=0.044). In Cox regression analysis, increased TFD was associated with improved disease-free survival (DFS) (p=0.007). DFS curves (for TFD cut off value 2.5 mm) according to Kaplan-Meier were found to be statistically significant (log rank test=0.002). Conclusion: The results indicate that TFD is predictive of pelvic lymph node involvement, LVSI and patient outcome in surgically treated cervical cancer patients. However, prospective measurement of TFD is still necessary to determine its value in clinical practice. © Springer-Verlag 2014

The value of frozen section evaluation in the management of borderline ovarian tumors

WOS: 000299473200007PubMed ID: 22269402Purpose: To evaluate the accuracy of a frozen section and to analyze the factors affecting frozen section results in cases of borderline ovarian tumors (BOTs). Materials and Methods: The files and pathological reports of 82 cases diagnosed with BOT at our clinic, between January 1994 and June 2009, have been retrospectively evaluated. The frozen section results were compared to the permanent paraffin section results. Accuracy, overdiagnosis, and underdiagnosis rates were estimated. The factors affecting the diagnosis were also evaluated using logistic regression analysis. Results: The mean age was 40.16 14.01 years. Of the patients, 47.6 had serous and 42.7 had mucinous histology. About 90 of the cases were in stage I. The rate of correct diagnosis with frozen section was 69.5. The rates of overdiagnosis and underdiagnosis were 1.2 and 29.3; respectively. The factors affecting the diagnosis were determined as, the dimension of the ovarian mass (P = 0.005), presence of a solid component (P = 0.000), preoperative CA 125 value (P = 0.016), and intraoperative rupture of the ovarian cyst (P = 0.049). Conclusion: In the frozen section evaluation of BOTs, the underdiagnosis that restricts the diagnostic performance of the method seems to be a major problem. A more careful approach is therefore needed, while choosing a proper surgical technique during laparotomy for ovarian masses. In order to reduce the false diagnosis and surgical morbidity, the frozen section analysis should be applied by experienced pathologists and the possible predictive factors affecting a false diagnosis should carefully be taken into consideration

The value of frozen section evaluation in the management of borderline ovarian tumors

Purpose: To evaluate the accuracy of a frozen section and to analyze the factors affecting frozen section results in cases of borderline ovarian tumors (BOTs). Materials and Methods: The files and pathological reports of 82 cases diagnosed with BOT at our clinic, between January 1994 and June 2009, have been retrospectively evaluated. The frozen section results were compared to the permanent paraffin section results. Accuracy, overdiagnosis, and underdiagnosis rates were estimated. The factors affecting the diagnosis were also evaluated using logistic regression analysis. Results: The mean age was 40.16 ± 14.01 years. Of the patients, 47.6% had serous and 42.7% had mucinous histology. About 90% of the cases were in stage I. The rate of correct diagnosis with frozen section was 69.5%. The rates of overdiagnosis and underdiagnosis were 1.2 and 29.3%; respectively. The factors affecting the diagnosis were determined as, the dimension of the ovarian mass (P = 0.005), presence of a solid component (P = 0.000), preoperative CA 125 value (P = 0.016), and intraoperative rupture of the ovarian cyst (P = 0.049). Conclusion: In the frozen section evaluation of BOTs, the underdiagnosis that restricts the diagnostic performance of the method seems to be a major problem. A more careful approach is therefore needed, while choosing a proper surgical technique during laparotomy for ovarian masses. In order to reduce the false diagnosis and surgical morbidity, the frozen section analysis should be applied by experienced pathologists and the possible predictive factors affecting a false diagnosis should carefully be taken into consideration