7,143 research outputs found
Selective Decoding in Associative Memories Based on Sparse-Clustered Networks
Associative memories are structures that can retrieve previously stored
information given a partial input pattern instead of an explicit address as in
indexed memories. A few hardware approaches have recently been introduced for a
new family of associative memories based on Sparse-Clustered Networks (SCN)
that show attractive features. These architectures are suitable for
implementations with low retrieval latency, but are limited to small networks
that store a few hundred data entries. In this paper, a new hardware
architecture of SCNs is proposed that features a new data-storage technique as
well as a method we refer to as Selective Decoding (SD-SCN). The SD-SCN has
been implemented using a similar FPGA used in the previous efforts and achieves
two orders of magnitude higher capacity, with no error-performance penalty but
with the cost of few extra clock cycles per data access.Comment: 4 pages, Accepted in IEEE Global SIP 2013 conferenc
The complexity of approximating conservative counting CSPs
We study the complexity of approximately solving the weighted counting
constraint satisfaction problem #CSP(F). In the conservative case, where F
contains all unary functions, there is a classification known for the case in
which the domain of functions in F is Boolean. In this paper, we give a
classification for the more general problem where functions in F have an
arbitrary finite domain. We define the notions of weak log-modularity and weak
log-supermodularity. We show that if F is weakly log-modular, then #CSP(F)is in
FP. Otherwise, it is at least as difficult to approximate as #BIS, the problem
of counting independent sets in bipartite graphs. #BIS is complete with respect
to approximation-preserving reductions for a logically-defined complexity class
#RHPi1, and is believed to be intractable. We further sub-divide the #BIS-hard
case. If F is weakly log-supermodular, then we show that #CSP(F) is as easy as
a (Boolean) log-supermodular weighted #CSP. Otherwise, we show that it is
NP-hard to approximate. Finally, we give a full trichotomy for the arity-2
case, where #CSP(F) is in FP, or is #BIS-equivalent, or is equivalent in
difficulty to #SAT, the problem of approximately counting the satisfying
assignments of a Boolean formula in conjunctive normal form. We also discuss
the algorithmic aspects of our classification.Comment: Minor revisio
A Digital Neuromorphic Architecture Efficiently Facilitating Complex Synaptic Response Functions Applied to Liquid State Machines
Information in neural networks is represented as weighted connections, or
synapses, between neurons. This poses a problem as the primary computational
bottleneck for neural networks is the vector-matrix multiply when inputs are
multiplied by the neural network weights. Conventional processing architectures
are not well suited for simulating neural networks, often requiring large
amounts of energy and time. Additionally, synapses in biological neural
networks are not binary connections, but exhibit a nonlinear response function
as neurotransmitters are emitted and diffuse between neurons. Inspired by
neuroscience principles, we present a digital neuromorphic architecture, the
Spiking Temporal Processing Unit (STPU), capable of modeling arbitrary complex
synaptic response functions without requiring additional hardware components.
We consider the paradigm of spiking neurons with temporally coded information
as opposed to non-spiking rate coded neurons used in most neural networks. In
this paradigm we examine liquid state machines applied to speech recognition
and show how a liquid state machine with temporal dynamics maps onto the
STPU-demonstrating the flexibility and efficiency of the STPU for instantiating
neural algorithms.Comment: 8 pages, 4 Figures, Preprint of 2017 IJCN
Liquid State Machine with Dendritically Enhanced Readout for Low-power, Neuromorphic VLSI Implementations
In this paper, we describe a new neuro-inspired, hardware-friendly readout
stage for the liquid state machine (LSM), a popular model for reservoir
computing. Compared to the parallel perceptron architecture trained by the
p-delta algorithm, which is the state of the art in terms of performance of
readout stages, our readout architecture and learning algorithm can attain
better performance with significantly less synaptic resources making it
attractive for VLSI implementation. Inspired by the nonlinear properties of
dendrites in biological neurons, our readout stage incorporates neurons having
multiple dendrites with a lumped nonlinearity. The number of synaptic
connections on each branch is significantly lower than the total number of
connections from the liquid neurons and the learning algorithm tries to find
the best 'combination' of input connections on each branch to reduce the error.
Hence, the learning involves network rewiring (NRW) of the readout network
similar to structural plasticity observed in its biological counterparts. We
show that compared to a single perceptron using analog weights, this
architecture for the readout can attain, even by using the same number of
binary valued synapses, up to 3.3 times less error for a two-class spike train
classification problem and 2.4 times less error for an input rate approximation
task. Even with 60 times larger synapses, a group of 60 parallel perceptrons
cannot attain the performance of the proposed dendritically enhanced readout.
An additional advantage of this method for hardware implementations is that the
'choice' of connectivity can be easily implemented exploiting address event
representation (AER) protocols commonly used in current neuromorphic systems
where the connection matrix is stored in memory. Also, due to the use of binary
synapses, our proposed method is more robust against statistical variations.Comment: 14 pages, 19 figures, Journa
Multi-task CNN Model for Attribute Prediction
This paper proposes a joint multi-task learning algorithm to better predict
attributes in images using deep convolutional neural networks (CNN). We
consider learning binary semantic attributes through a multi-task CNN model,
where each CNN will predict one binary attribute. The multi-task learning
allows CNN models to simultaneously share visual knowledge among different
attribute categories. Each CNN will generate attribute-specific feature
representations, and then we apply multi-task learning on the features to
predict their attributes. In our multi-task framework, we propose a method to
decompose the overall model's parameters into a latent task matrix and
combination matrix. Furthermore, under-sampled classifiers can leverage shared
statistics from other classifiers to improve their performance. Natural
grouping of attributes is applied such that attributes in the same group are
encouraged to share more knowledge. Meanwhile, attributes in different groups
will generally compete with each other, and consequently share less knowledge.
We show the effectiveness of our method on two popular attribute datasets.Comment: 11 pages, 3 figures, ieee transaction pape
- …