Dimensions of Neural-symbolic Integration - A Structured Survey

Research on integrated neural-symbolic systems has made significant progress in the recent past. In particular the understanding of ways to deal with symbolic knowledge within connectionist systems (also called artificial neural networks) has reached a critical mass which enables the community to strive for applicable implementations and use cases. Recent work has covered a great variety of logics used in artificial intelligence and provides a multitude of techniques for dealing with them within the context of artificial neural networks. We present a comprehensive survey of the field of neural-symbolic integration, including a new classification of system according to their architectures and abilities.Comment: 28 page

KAJIAN PEMANFAATAN MODEL PENGETAHUAN UNTUK MEMBANGUN SUATU MODEL ARSITEKTUR JARINGAN SYARAF TIRUAN (JST BERBASIS-ATURAN)

Jaringan syaraf tiruan (JST) banyak mendapat kritikan sebagai model kotak hitam yang walaupun dapat mengambil suatu kesimpulan dengan baik namun tidak mampu menjelaskan secara logis mengapa kesimpulan tersebut masuk akal untuk diambil. Pada makalah ini ditunjukkan bahwa pernyataan ini tidak sepenuhnya benar dengan cara memperkenalkan JST yang dapat melakukan penalaran mundur pada klausa definite logika proposisi maupun logika orde pertama. JST yang digunakan bukan merupakan JST konvensional seperti propagasi-balik (backpropagation) yang fitur utamanya adalah pembelajaran statistikal. JST yang digunakan adalah JST nonstatistik yang dibuat sedemikian rupa sehingga cocok digunakan untuk melakukan penalaran secara logis dengan kemampuan untuk memberikan argumentasi terhadap kesimpulan yang diperolehnya. JST pada makalah ini merupakan model inferensi yang massively parallel sehingga dapat digunakan untuk melakukan penalaran dengan lebih efisien jika dijalankan pada sistem komputer paralel.Kata Kunci: jaringan syaraf tiruan, argumentasi, penalaran, inferensi, klausa definite, massively parallel

Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning

Current advances in Artificial Intelligence and machine learning in general, and deep learning in particular have reached unprecedented impact not only across research communities, but also over popular media channels. However, concerns about interpretability and accountability of AI have been raised by influential thinkers. In spite of the recent impact of AI, several works have identified the need for principled knowledge representation and reasoning mechanisms integrated with deep learning-based systems to provide sound and explainable models for such systems. Neural-symbolic computing aims at integrating, as foreseen by Valiant, two most fundamental cognitive abilities: the ability to learn from the environment, and the ability to reason from what has been learned. Neural-symbolic computing has been an active topic of research for many years, reconciling the advantages of robust learning in neural networks and reasoning and interpretability of symbolic representation. In this paper, we survey recent accomplishments of neural-symbolic computing as a principled methodology for integrated machine learning and reasoning. We illustrate the effectiveness of the approach by outlining the main characteristics of the methodology: principled integration of neural learning with symbolic knowledge representation and reasoning allowing for the construction of explainable AI systems. The insights provided by neural-symbolic computing shed new light on the increasingly prominent need for interpretable and accountable AI systems

Acausality and the Machian Mind

In this paper we propose a mechanism in the brain for supporting consciousness. We leave open the question of the origin of consciousness itself, although an acausal origin is suggested since it should mesh with the proposed quasi-acausal network dynamics.  In particular, we propose simply that fixed-point attractors, such as exemplified by the simple deterministic Hopfield network, correspond to conscious moments.  In a sort of dual to Tononi's Integrated Information Theory, we suggest that the "main experience" corresponds to a dominant fixed point that incorporates sub-networks that span the brain and maximizes "relatedness." The dynamics around the dominant fixed point correspond in some parts of the system to associative memory dynamics, and to more binding constraint satisfaction dynamics in other areas. Since the memories that we are familiar with appear to have a conscious origin, it makes sense that a conscious moment itself corresponds in effect to what amounts to memory recollection.  Furthermore, since Hopfield-like networks are generative, a conscious moment can in effect be seen as a living, partially predicted memory. Another primary motivation for this approach is that alternative states can be naturally sensed, or contrasted, at the fixed points

Evoked Patterns of Oscillatory Activity in Mean-Field Neuronal Networks

Oscillatory behaviors in populations of neurons are oberved in diverse contexts. In tasks involving working memory, a form of short-term memory, oscillations in different frequency bands have been shown to increase across varying spatial scales using recording methods such as EEG (electroencephalogram) and MEG (magnetoencephalogram). Such oscillatory activity has also been observed in the context of neural binding, where different features of objects that are perceived or recalled are associated with one another. These sets of data suggest that oscillatory dynamics may also play a key role in the maintenance and manipulation of items in working memory. Using similar recording techniques, including EEG and MEG, oscillatory neuronal activity has also been seen to occur when certain images that cause aversion and headaches in healthy human subjects or seizures in those with pattern-sensitive epilepsy are presented. The images most likely to cause such responses are those with dominant spatial frequencies near 3--5 cycles per degree, the same band of wavenumbers to which normal human vision exhibits the greatest contrast sensitivity. We model these oscillatory behaviors using mean-field, Wilson-Cowan-type neuronal networks. In the case of working memory and binding, we find that including the activity of certain long-lasting excitatory synapses in addition to the usual inhibitory and shorter-term excitatory synaptic activity allows for bistability between a low steady state and a high oscillatory state. By coupling several such populations together, both in-phase and out-of-phase oscillations arise, corresponding to distinct and bound items in working memory, respectively. We analyze the network's dynamics and dependence on biophysically relevant parameters using a combination of techniques, including numerical bifurcation analysis and weak coupling theory. In the case of spatially resonant responses to static simtuli, we employ Wilson-Cowan networks extended in one and two spatial dimensions. By placing the networks near Turing-Hopf bifurcations, we find they exhibit spatial resonances that compare well with empirical results. Using simulations, numerical bifurcation analysis, and perturbation theory, we characterize the observed dynamics and gain mathematical insight into the mechanisms that lead to these dynamics

Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review

Advocates for Neuro-Symbolic Artificial Intelligence (NeSy) assert that combining deep learning with symbolic reasoning will lead to stronger AI than either paradigm on its own. As successful as deep learning has been, it is generally accepted that even our best deep learning systems are not very good at abstract reasoning. And since reasoning is inextricably linked to language, it makes intuitive sense that Natural Language Processing (NLP), would be a particularly well-suited candidate for NeSy. We conduct a structured review of studies implementing NeSy for NLP, with the aim of answering the question of whether NeSy is indeed meeting its promises: reasoning, out-of-distribution generalization, interpretability, learning and reasoning from small data, and transferability to new domains. We examine the impact of knowledge representation, such as rules and semantic networks, language structure and relational structure, and whether implicit or explicit reasoning contributes to higher promise scores. We find that systems where logic is compiled into the neural network lead to the most NeSy goals being satisfied, while other factors such as knowledge representation, or type of neural architecture do not exhibit a clear correlation with goals being met. We find many discrepancies in how reasoning is defined, specifically in relation to human level reasoning, which impact decisions about model architectures and drive conclusions which are not always consistent across studies. Hence we advocate for a more methodical approach to the application of theories of human reasoning as well as the development of appropriate benchmarks, which we hope can lead to a better understanding of progress in the field. We make our data and code available on github for further analysis.Comment: Surve

Restorative Justice and the Rule of Law: Rethinking Due Process through a Relational Theory of Rights

Restorative approaches to criminal justice can be reconciled with fundamental notions of the rule of law through a relational understanding of rights. Firstly, the paper demonstrates how theories of rights have evolved from a liberal understanding in representative democracies, where individual rights holders can trump the interests of others, to a relational theory where rights embody values which structure appropriate relationships among citizens. Second, the paper shows that relational theory can explain how formal criminal justice and restorative justice in a deliberate democracy interrelate, while embodying different, though compatible, rights, duties and remedies among wrongdoers, victims, communities and justice system authorities. Third, the paper invokes a relational understanding of administrative law to chart an approach to judicial review of restorative justice processes, which can reinforce their deliberative and participatory nature through vindication of relational rights and remedies, without simply returning cases to criminal courts. Finally, the paper details the substantive and procedural administrative law standards to be applied in reviewing restorative justice. The conclusion asserts that a relational understanding of the role and rule of law in relation to restorative justice promotes relationships of equality based on mutual concern, respect and dignity in ways that can enhance justice and social solidarity in a deliberative democracy

Integration of Event Processing with Service-oriented Architectures and Business Processes

Data sources like the Internet of Things or Cyber-physical Systems provide enormous amounts of real-time information in form of streams of events. The use of such event streams enables reactive software components as building blocks in a new generation of systems. Businesses, for example, can benefit from the integration of event streams; new services can be provided to customers, or existing business processes can be improved. The development of reactive systems and the integration with existing application landscapes, however, is challenging. While traditional system components follow a pull-based request/reply interaction style, event-based systems follow a push-based interaction scheme; events arrive continuously and application logic is triggered implicitly. To benefit from push-based and pull-based interactions together, an intuitive software abstraction is necessary to integrate push-based application logic with existing systems. In this work we introduce such an abstraction: we present Event Stream Processing Units (SPUs) - a container model for the encapsulation of event-processing application logic at the technical layer as well as at the business process layer. At the technical layer SPUs provide a service-like abstraction and simplify the development of scalable reactive applications. At the business process layer SPUs make event processing explicitly representable. SPUs have a managed lifecycle and are instantiated implicitly - upon arrival of appropriate events - or explicitly upon request. At the business process layer SPUs encapsulate application logic for event stream processing and enable a seamless transition between process models, executable process representations, and components at the IT layer. Throughout this work, we focus on different aspects of the SPU container model: we first introduce the SPU container model and its execution semantics. Since SPUs rely on a publish/subscribe system for event dissemination, we discuss quality of service requirements in the context of event processing. SPUs rely on input in form of events; in event-based systems, however, event production is logically decoupled, i.e., event producers are not aware of the event consumers. This influences the system development process and requires an appropriate methodology. Fur this purpose we present a requirements engineering approach that takes the specifics of event-based applications into account. The integration of events with business processes leads to new business opportunities. SPUs can encapsulate event processing at the abstraction level of business functions and enable a seamless integration with business processes. For this integration, we introduce extensions to the business process modeling notations BPMN and EPCs to model SPUs. We also present a model-to-execute workflow for SPU-containing process models and implementation with business process modeling software. The SPU container model itself is language-agnostic; thus, we present Eventlets as SPU implementation based on Java Enterprise technology. Eventlets are executed inside a distributed middleware and follow a lifecycle. They reduce the development effort of scalable event processing applications as we show in our evaluation. Since the SPU container model introduces an additional layer of abstraction we analyze the overhead in terms of performance and show that Eventlets can compete with traditional event processing approaches in terms of performance. SPUs can be used to process sensitive data, e.g., in health care environments. Thus, privacy protection is an important requirement for certain use cases and we sketch the application of a privacy-preserving event dissemination scheme to protect event consumers and producers from curious brokers. We also quantify the resulting overhead introduced by a privacy-preserving brokering scheme in an evaluation

Towards Comprehensive Foundations of Computational Intelligence

Abstract. Although computational intelligence (CI) covers a vast variety of different methods it still lacks an integrative theory. Several proposals for CI foundations are discussed: computing and cognition as compression, meta-learning as search in the space of data models, (dis)similarity based methods providing a framework for such meta-learning, and a more general approach based on chains of transformations. Many useful transformations that extract information from features are discussed. Heterogeneous adaptive systems are presented as particular example of transformation-based systems, and the goal of learning is redefined to facilitate creation of simpler data models. The need to understand data structures leads to techniques for logical and prototype-based rule extraction, and to generation of multiple alternative models, while the need to increase predictive power of adaptive models leads to committees of competent models. Learning from partial observations is a natural extension towards reasoning based on perceptions, and an approach to intuitive solving of such problems is presented. Throughout the paper neurocognitive inspirations are frequently used and are especially important in modeling of the higher cognitive functions. Promising directions such as liquid and laminar computing are identified and many open problems presented.