371,801 research outputs found
Information theory, complexity and neural networks
Some of the main results in the mathematical evaluation of neural networks as information processing systems are discussed. The basic operation of feedback and feed-forward neural networks is described. Their memory capacity and computing power are considered. The concept of learning by example as it applies to neural networks is examined
Characterizing Self-Developing Biological Neural Networks: A First Step Towards their Application To Computing Systems
Carbon nanotubes are often seen as the only alternative technology to silicon
transistors. While they are the most likely short-term one, other longer-term
alternatives should be studied as well. While contemplating biological neurons
as an alternative component may seem preposterous at first sight, significant
recent progress in CMOS-neuron interface suggests this direction may not be
unrealistic; moreover, biological neurons are known to self-assemble into very
large networks capable of complex information processing tasks, something that
has yet to be achieved with other emerging technologies. The first step to
designing computing systems on top of biological neurons is to build an
abstract model of self-assembled biological neural networks, much like computer
architects manipulate abstract models of transistors and circuits. In this
article, we propose a first model of the structure of biological neural
networks. We provide empirical evidence that this model matches the biological
neural networks found in living organisms, and exhibits the small-world graph
structure properties commonly found in many large and self-organized systems,
including biological neural networks. More importantly, we extract the simple
local rules and characteristics governing the growth of such networks, enabling
the development of potentially large but realistic biological neural networks,
as would be needed for complex information processing/computing tasks. Based on
this model, future work will be targeted to understanding the evolution and
learning properties of such networks, and how they can be used to build
computing systems
Integrating Evolutionary Computation with Neural Networks
There is a tremendous interest in the development of the evolutionary computation techniques as they are well suited to deal with optimization of functions containing a large number of variables. This paper presents a brief review of evolutionary computing techniques. It also discusses briefly the hybridization of evolutionary computation and neural networks and presents a solution of a classical problem using neural computing and evolutionary computing technique
Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective
Neural-symbolic computing has now become the subject of interest of both
academic and industry research laboratories. Graph Neural Networks (GNN) have
been widely used in relational and symbolic domains, with widespread
application of GNNs in combinatorial optimization, constraint satisfaction,
relational reasoning and other scientific domains. The need for improved
explainability, interpretability and trust of AI systems in general demands
principled methodologies, as suggested by neural-symbolic computing. In this
paper, we review the state-of-the-art on the use of GNNs as a model of
neural-symbolic computing. This includes the application of GNNs in several
domains as well as its relationship to current developments in neural-symbolic
computing.Comment: Updated version, draft of accepted IJCAI2020 Survey Pape
Optimal modularity and memory capacity of neural reservoirs
The neural network is a powerful computing framework that has been exploited
by biological evolution and by humans for solving diverse problems. Although
the computational capabilities of neural networks are determined by their
structure, the current understanding of the relationships between a neural
network's architecture and function is still primitive. Here we reveal that
neural network's modular architecture plays a vital role in determining the
neural dynamics and memory performance of the network of threshold neurons. In
particular, we demonstrate that there exists an optimal modularity for memory
performance, where a balance between local cohesion and global connectivity is
established, allowing optimally modular networks to remember longer. Our
results suggest that insights from dynamical analysis of neural networks and
information spreading processes can be leveraged to better design neural
networks and may shed light on the brain's modular organization
- âŠ