Symbolic knowledge extraction from trained neural networks: A sound approach

Although neural networks have shown very good performance in many application domains, one of their main drawbacks lies in the incapacity to provide an explanation for the underlying reasoning mechanisms. The “explanation capability” of neural networks can be achieved by the extraction of symbolic knowledge. In this paper, we present a new method of extraction that captures nonmonotonic rules encoded in the network, and prove that such a method is sound. We start by discussing some of the main problems of knowledge extraction methods. We then discuss how these problems may be ameliorated. To this end, a partial ordering on the set of input vectors of a network is defined, as well as a number of pruning and simplification rules. The pruning rules are then used to reduce the search space of the extraction algorithm during a pedagogical extraction, whereas the simplification rules are used to reduce the size of the extracted set of rules. We show that, in the case of regular networks, the extraction algorithm is sound and complete. We proceed to extend the extraction algorithm to the class of non-regular networks, the general case. We show that non-regular networks always contain regularities in their subnetworks. As a result, the underlying extraction method for regular networks can be applied, but now in a decompositional fashion. In order to combine the sets of rules extracted from each subnetwork into the final set of rules, we use a method whereby we are able to keep the soundness of the extraction algorithm. Finally, we present the results of an empirical analysis of the extraction system, using traditional examples and real-world application problems. The results have shown that a very high fidelity between the extracted set of rules and the network can be achieved

Neural-symbolic cognitive agents : Architecture and theory

In real-world applications, the effective integration of learning and reasoning in a cognitive agent model is a difficult task. However, such integration may lead to a better understanding, use and construction of more realistic models. Unfortunately, existing models are either oversimplified or require much processing time, which is unsuitable for online learning and reasoning. Currently, controlled environments like training simulators do not effectively integrate learning and reasoning. In particular, higher-order concepts and cognitive abilities have many unknown temporal relations with the data, making it impossible to represent such relationships by hand. We introduce a novel cognitive agent model and architecture for online learning and reasoning that seeks to effectively represent, learn and reason in complex real-world applications. The agent architecture of the model combines neural learning with symbolic knowledge representation. It is capable of learning new hypotheses from observed data, and infer new beliefs based on these hypotheses. Furthermore, it deals with uncertainty and errors in the data using a Bayesian inference model. The model has successfully been applied in real-time simulation and visual intelligence systems

An integrated neural-symbolic cognitive agent architecture for training and assessment in simulators

Training and assessment of complex tasks has always been a complex task in itself. Training simulators can be used for training and assessment of low-order skills. High-order skills (e.g. safe driving, leadership, tactical manoeuvring, etc.) are generally trained and assessed by human experts, due to its complex nature (i.e. many temporal relations, biased behaviour and poorly documented). This paper proposes a new cognitive agent architecture that is able to model this complex behaviour and use it for the assessment and training of both low- and high-order skills. Therefore the agent integrates learning from observation, temporal logic and probalistic reasoning in a unified architecture that is based on Neural-Symbolic Learning and Reasoning. This so-called Neural Symbolic Cognitive Agent (NSCA) architecture combines encoding temporal logic based expert knowledge and learning new knowledge by observing experts and trainees during task execution in a simulator. The learned knowledge can be extracted in temporal logic rules for validation. Learning and reasoning is done using a Recurrent Temporal Restricted Boltzmann Machine (RTRBM). For training organizations, this provides a quicker, cost-saving and more objective evaluation of the trainee in simulation-based training. A prototype NSCA has been developed and tested as part of a three-year research project on assessment in driving simulators for training and certification, and will be tested in various other domains, such as jetfighter pilot training and strategic command and control training

Formal considerations on Gabbay's restart rule in natural deduction and goal-directed reasoning

NO ABSTRAC

alpha-logic

NO ABSTRAC