1,157 research outputs found
On Neural Associative Memory Structures: Storage and Retrieval of Sequences in a Chain of Tournaments
Associative memories enjoy many interesting properties in terms of error correction capabilities, robustness to noise, storage capacity, and retrieval performance, and their usage spans over a large set of applications. In this letter, we investigate and extend tournament-based neural networks, originally proposed by Jiang, Gripon, Berrou, and Rabbat (2016), a novel sequence storage associative memory architecture with high memory efficiency and accurate sequence retrieval. We propose a more general method for learning the sequences, which we call feedback tournament-based neural networks. The retrieval process is also extended to both directions: forward and backward—in other words, any large-enough segment of a sequence can produce the whole sequence. Furthermore, two retrieval algorithms, cache-winner and explore-winner, are introduced to increase the retrieval performance. Through simulation results, we shed light on the strengths and weaknesses of each algorithm.publishedVersio
The sparse Blume-Emery-Griffiths model of associative memories
We analyze the Blume-Emery-Griffiths (BEG) associative memory with sparse
patterns and at zero temperature. We give bounds on its storage capacity
provided that we want the stored patterns to be fixed points of the retrieval
dynamics. We compare our results to that of other models of sparse neural
networks and show that the BEG model has a superior performance compared to
them.Comment: 23 p
A Comparative Study of Sparse Associative Memories
We study various models of associative memories with sparse information, i.e.
a pattern to be stored is a random string of s and s with about
s, only. We compare different synaptic weights, architectures and retrieval
mechanisms to shed light on the influence of the various parameters on the
storage capacity.Comment: 28 pages, 2 figure
DebateKG: Automatic Policy Debate Case Creation with Semantic Knowledge Graphs
Recent work within the Argument Mining community has shown the applicability
of Natural Language Processing systems for solving problems found within
competitive debate. One of the most important tasks within competitive debate
is for debaters to create high quality debate cases. We show that effective
debate cases can be constructed using constrained shortest path traversals on
Argumentative Semantic Knowledge Graphs. We study this potential in the context
of a type of American Competitive Debate, called Policy Debate, which already
has a large scale dataset targeting it called DebateSum. We significantly
improve upon DebateSum by introducing 53180 new examples, as well as further
useful metadata for every example, to the dataset. We leverage the txtai
semantic search and knowledge graph toolchain to produce and contribute 9
semantic knowledge graphs built on this dataset. We create a unique method for
evaluating which knowledge graphs are better in the context of producing policy
debate cases. A demo which automatically generates debate cases, along with all
other code and the Knowledge Graphs, are open-sourced and made available to the
public here: https://github.com/Hellisotherpeople/DebateKGComment: 8 pages, knife-edge reject from EACL 2023 and workshops, System
Demonstration pape
Recommended from our members
Discovering gated recurrent neural network architectures
Reinforcement Learning agent networks with memory are a key component in solving POMDP tasks.
Gated recurrent networks such as those composed of Long Short-Term
Memory (LSTM) nodes have recently been used to improve
state of the art in many supervised sequential processing tasks such as speech
recognition and machine translation. However, scaling them to deep
memory tasks in reinforcement learning domain is challenging because of sparse and deceptive
reward function. To address this challenge first, a new secondary optimization objective is introduced
that maximizes the information (Info-max) stored in
the LSTM network. Results indicate that when combined with neuroevolution, Info-max can discover powerful
LSTM-based memory solutions that outperform traditional
RNNs. Next, for the supervised learning tasks, neuroevolution techniques are employed
to design new LSTM architectures. Such architectural variations include
discovering new pathways between the recurrent layers as well as designing new gated
recurrent nodes. This dissertation proposes evolution of a tree-based
encoding of the gated memory nodes, and shows that it makes
it possible to explore new variations more effectively than other
methods. The method discovers nodes with multiple recurrent paths
and multiple memory cells, which lead to significant improvement
in the standard language modeling benchmark task. The dissertation also
shows how the search process can be speeded up by training an
LSTM network to estimate performance of candidate structures, and
by encouraging exploration of novel solutions. Thus, evolutionary
design of complex neural network structures promises to improve
performance of deep learning architectures beyond human ability
to do so.Computer Science
ダイナミックバイナリーニューラルネットの学習と安定化
A dynamic binary neural network is a simple two-layer network with a delayed feedback and is able to generate various binary periodic orbits. The network is characterized by the signum activationfunction, ternary connection parameters, and integer threshold parameters. The ternary connection brings benefits to network hardware and to computation costs in numerical analysis.The dynamics is simplified into a digital return map on a set of lattice points. We investigate the relation between sparsity of network connection and stability of a target periodic orbit. In order to stabilize a desired binary periodic orbit, we introdece some methods algorithm uses Each individual is evaluated by some feature quantities that characterize the stability of the periodic orbit. Applying the algorithm to a class of periodic orbits that are applicable to control signals of switching power converters, the usefulness of sparsification in stabilization of desired periodicorbit is confirmed.Key Words : Dynamic binary neural networks, Stabilization, Feature quantitie
Advances in Artificial Intelligence: Models, Optimization, and Machine Learning
The present book contains all the articles accepted and published in the Special Issue “Advances in Artificial Intelligence: Models, Optimization, and Machine Learning” of the MDPI Mathematics journal, which covers a wide range of topics connected to the theory and applications of artificial intelligence and its subfields. These topics include, among others, deep learning and classic machine learning algorithms, neural modelling, architectures and learning algorithms, biologically inspired optimization algorithms, algorithms for autonomous driving, probabilistic models and Bayesian reasoning, intelligent agents and multiagent systems. We hope that the scientific results presented in this book will serve as valuable sources of documentation and inspiration for anyone willing to pursue research in artificial intelligence, machine learning and their widespread applications
- …