193,840 research outputs found
Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective
On metrics of density and power efficiency, neuromorphic technologies have
the potential to surpass mainstream computing technologies in tasks where
real-time functionality, adaptability, and autonomy are essential. While
algorithmic advances in neuromorphic computing are proceeding successfully, the
potential of memristors to improve neuromorphic computing have not yet born
fruit, primarily because they are often used as a drop-in replacement to
conventional memory. However, interdisciplinary approaches anchored in machine
learning theory suggest that multifactor plasticity rules matching neural and
synaptic dynamics to the device capabilities can take better advantage of
memristor dynamics and its stochasticity. Furthermore, such plasticity rules
generally show much higher performance than that of classical Spike Time
Dependent Plasticity (STDP) rules. This chapter reviews the recent development
in learning with spiking neural network models and their possible
implementation with memristor-based hardware
Attentive Tensor Product Learning
This paper proposes a new architecture - Attentive Tensor Product Learning
(ATPL) - to represent grammatical structures in deep learning models. ATPL is a
new architecture to bridge this gap by exploiting Tensor Product
Representations (TPR), a structured neural-symbolic model developed in
cognitive science, aiming to integrate deep learning with explicit language
structures and rules. The key ideas of ATPL are: 1) unsupervised learning of
role-unbinding vectors of words via TPR-based deep neural network; 2) employing
attention modules to compute TPR; and 3) integration of TPR with typical deep
learning architectures including Long Short-Term Memory (LSTM) and Feedforward
Neural Network (FFNN). The novelty of our approach lies in its ability to
extract the grammatical structure of a sentence by using role-unbinding
vectors, which are obtained in an unsupervised manner. This ATPL approach is
applied to 1) image captioning, 2) part of speech (POS) tagging, and 3)
constituency parsing of a sentence. Experimental results demonstrate the
effectiveness of the proposed approach
Identification of Evolving Rule-based Models.
An approach to identification of evolving fuzzy rule-based (eR) models is proposed. eR models implement a method for the noniterative update of both the rule-base structure and parameters by incremental unsupervised learning. The rule-base evolves by adding more informative rules than those that previously formed the model. In addition, existing rules can be replaced with new rules based on ranking using the informative potential of the data. In this way, the rule-base structure is inherited and updated when new informative data become available, rather than being completely retrained. The adaptive nature of these evolving rule-based models, in combination with the highly transparent and compact form of fuzzy rules, makes them a promising candidate for modeling and control of complex processes, competitive to neural networks. The approach has been tested on a benchmark problem and on an air-conditioning component modeling application using data from an installation serving a real building. The results illustrate the viability and efficiency of the approach. (c) IEEE Transactions on Fuzzy System
Generating by Understanding: Neural Visual Generation with Logical Symbol Groundings
Despite the great success of neural visual generative models in recent years,
integrating them with strong symbolic knowledge reasoning systems remains a
challenging task. The main challenges are two-fold: one is symbol assignment,
i.e. bonding latent factors of neural visual generators with meaningful symbols
from knowledge reasoning systems. Another is rule learning, i.e. learning new
rules, which govern the generative process of the data, to augment the
knowledge reasoning systems. To deal with these symbol grounding problems, we
propose a neural-symbolic learning approach, Abductive Visual Generation
(AbdGen), for integrating logic programming systems with neural visual
generative models based on the abductive learning framework. To achieve
reliable and efficient symbol assignment, the quantized abduction method is
introduced for generating abduction proposals by the nearest-neighbor lookups
within semantic codebooks. To achieve precise rule learning, the contrastive
meta-abduction method is proposed to eliminate wrong rules with positive cases
and avoid less-informative rules with negative cases simultaneously.
Experimental results on various benchmark datasets show that compared to the
baselines, AbdGen requires significantly fewer instance-level labeling
information for symbol assignment. Furthermore, our approach can effectively
learn underlying logical generative rules from data, which is out of the
capability of existing approaches
- …