15 research outputs found
Cellular Automata Can Reduce Memory Requirements of Collective-State Computing
Various non-classical approaches of distributed information processing, such
as neural networks, computation with Ising models, reservoir computing, vector
symbolic architectures, and others, employ the principle of collective-state
computing. In this type of computing, the variables relevant in a computation
are superimposed into a single high-dimensional state vector, the
collective-state. The variable encoding uses a fixed set of random patterns,
which has to be stored and kept available during the computation. Here we show
that an elementary cellular automaton with rule 90 (CA90) enables space-time
tradeoff for collective-state computing models that use random dense binary
representations, i.e., memory requirements can be traded off with computation
running CA90. We investigate the randomization behavior of CA90, in particular,
the relation between the length of the randomization period and the size of the
grid, and how CA90 preserves similarity in the presence of the initialization
noise. Based on these analyses we discuss how to optimize a collective-state
computing model, in which CA90 expands representations on the fly from short
seed patterns - rather than storing the full set of random patterns. The CA90
expansion is applied and tested in concrete scenarios using reservoir computing
and vector symbolic architectures. Our experimental results show that
collective-state computing with CA90 expansion performs similarly compared to
traditional collective-state models, in which random patterns are generated
initially by a pseudo-random number generator and then stored in a large
memory.Comment: 13 pages, 11 figure
On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks
A change of the prevalent supervised learning techniques is foreseeable in
the near future: from the complex, computational expensive algorithms to more
flexible and elementary training ones. The strong revitalization of randomized
algorithms can be framed in this prospect steering. We recently proposed a
model for distributed classification based on randomized neural networks and
hyperdimensional computing, which takes into account cost of information
exchange between agents using compression. The use of compression is important
as it addresses the issues related to the communication bottleneck, however,
the original approach is rigid in the way the compression is used. Therefore,
in this work, we propose a more flexible approach to compression and compare it
to conventional compression algorithms, dimensionality reduction, and
quantization techniques.Comment: 12 pages, 3 figure
Perceptron theory can predict the accuracy of neural networks
Multilayer neural networks set the current state of
the art for many technical classification problems. But, these
networks are still, essentially, black boxes in terms of analyzing
them and predicting their performance. Here, we develop a
statistical theory for the one-layer perceptron and show that
it can predict performances of a surprisingly large variety of
neural networks with different architectures. A general theory
of classification with perceptrons is developed by generalizing
an existing theory for analyzing reservoir computing models
and connectionist models for symbolic reasoning known as
vector symbolic architectures. Our statistical theory offers three
formulas leveraging the signal statistics with increasing detail.
The formulas are analytically intractable, but can be evaluated
numerically. The description level that captures maximum details
requires stochastic sampling methods. Depending on the network
model, the simpler formulas already yield high prediction accuracy.
The quality of the theory predictions is assessed in three
experimental settings, a memorization task for echo state networks
(ESNs) from reservoir computing literature, a collection of
classification datasets for shallow randomly connected networks,
and the ImageNet dataset for deep convolutional neural networks.
We find that the second description level of the perceptron theory
can predict the performance of types of ESNs, which could not
be described previously. Furthermore, the theory can predict
deep multilayer neural networks by being applied to their output
layer. While other methods for prediction of neural networks
performance commonly require to train an estimator model,
the proposed theory requires only the first two moments of
the distribution of the postsynaptic sums in the output neurons.
Moreover, the perceptron theory compares favorably to other
methods that do not rely on training an estimator model
Hardware-Aware Static Optimization of Hyperdimensional Computations
Binary spatter code (BSC)-based hyperdimensional computing (HDC) is a highly
error-resilient approximate computational paradigm suited for error-prone,
emerging hardware platforms. In BSC HDC, the basic datatype is a hypervector, a
typically large binary vector, where the size of the hypervector has a
significant impact on the fidelity and resource usage of the computation.
Typically, the hypervector size is dynamically tuned to deliver the desired
accuracy; this process is time-consuming and often produces hypervector sizes
that lack accuracy guarantees and produce poor results when reused for very
similar workloads. We present Heim, a hardware-aware static analysis and
optimization framework for BSC HD computations. Heim analytically derives the
minimum hypervector size that minimizes resource usage and meets the target
accuracy requirement. Heim guarantees the optimized computation converges to
the user-provided accuracy target on expectation, even in the presence of
hardware error. Heim deploys a novel static analysis procedure that unifies
theoretical results from the neuroscience community to systematically optimize
HD computations.
We evaluate Heim against dynamic tuning-based optimization on 25 benchmark
data structures. Given a 99% accuracy requirement, Heim-optimized computations
achieve a 99.2%-100.0% median accuracy, up to 49.5% higher than dynamic
tuning-based optimization, while achieving 1.15x-7.14x reductions in
hypervector size compared to HD computations that achieve comparable query
accuracy and finding parametrizations 30.0x-100167.4x faster than dynamic
tuning-based approaches. We also use Heim to systematically evaluate the
performance benefits of using analog CAMs and multiple-bit-per-cell ReRAM over
conventional hardware, while maintaining iso-accuracy -- for both emerging
technologies, we find usages where the emerging hardware imparts significant
benefits
Brain-Inspired Computational Intelligence via Predictive Coding
Artificial intelligence (AI) is rapidly becoming one of the key technologies
of this century. The majority of results in AI thus far have been achieved
using deep neural networks trained with the error backpropagation learning
algorithm. However, the ubiquitous adoption of this approach has highlighted
some important limitations such as substantial computational cost, difficulty
in quantifying uncertainty, lack of robustness, unreliability, and biological
implausibility. It is possible that addressing these limitations may require
schemes that are inspired and guided by neuroscience theories. One such theory,
called predictive coding (PC), has shown promising performance in machine
intelligence tasks, exhibiting exciting properties that make it potentially
valuable for the machine learning community: PC can model information
processing in different brain areas, can be used in cognitive control and
robotics, and has a solid mathematical grounding in variational inference,
offering a powerful inversion scheme for a specific class of continuous-state
generative models. With the hope of foregrounding research in this direction,
we survey the literature that has contributed to this perspective, highlighting
the many ways that PC might play a role in the future of machine learning and
computational intelligence at large.Comment: 37 Pages, 9 Figure
Toward a formal theory for computing machines made out of whatever physics offers: extended version
Approaching limitations of digital computing technologies have spurred
research in neuromorphic and other unconventional approaches to computing. Here
we argue that if we want to systematically engineer computing systems that are
based on unconventional physical effects, we need guidance from a formal theory
that is different from the symbolic-algorithmic theory of today's computer
science textbooks. We propose a general strategy for developing such a theory,
and within that general view, a specific approach that we call "fluent
computing". In contrast to Turing, who modeled computing processes from a
top-down perspective as symbolic reasoning, we adopt the scientific paradigm of
physics and model physical computing systems bottom-up by formalizing what can
ultimately be measured in any physical substrate. This leads to an
understanding of computing as the structuring of processes, while classical
models of computing systems describe the processing of structures.Comment: 76 pages. This is an extended version of a perspective article with
the same title that will appear in Nature Communications soon after this
manuscript goes public on arxi
A Review of Findings from Neuroscience and Cognitive Psychology as Possible Inspiration for the Path to Artificial General Intelligence
This review aims to contribute to the quest for artificial general
intelligence by examining neuroscience and cognitive psychology methods for
potential inspiration. Despite the impressive advancements achieved by deep
learning models in various domains, they still have shortcomings in abstract
reasoning and causal understanding. Such capabilities should be ultimately
integrated into artificial intelligence systems in order to surpass data-driven
limitations and support decision making in a way more similar to human
intelligence. This work is a vertical review that attempts a wide-ranging
exploration of brain function, spanning from lower-level biological neurons,
spiking neural networks, and neuronal ensembles to higher-level concepts such
as brain anatomy, vector symbolic architectures, cognitive and categorization
models, and cognitive architectures. The hope is that these concepts may offer
insights for solutions in artificial general intelligence.Comment: 143 pages, 49 figures, 244 reference
Mortal Computation: A Foundation for Biomimetic Intelligence
This review motivates and synthesizes research efforts in
neuroscience-inspired artificial intelligence and biomimetic computing in terms
of mortal computation. Specifically, we characterize the notion of mortality by
recasting ideas in biophysics, cybernetics, and cognitive science in terms of a
theoretical foundation for sentient behavior. We frame the mortal computation
thesis through the Markov blanket formalism and the circular causality entailed
by inference, learning, and selection. The ensuing framework -- underwritten by
the free energy principle -- could prove useful for guiding the construction of
unconventional connectionist computational systems, neuromorphic intelligence,
and chimeric agents, including sentient organoids, which stand to revolutionize
the long-term future of embodied, enactive artificial intelligence and
cognition research.Comment: Several revisions applied, corrected error in Jarzynski equality
equation (w/ new citaion); references and citations now correctly aligne
Artificial intelligence and architectural design : an introduction
Descripció del recurs: 27 juliol 2022The aim of this book on artificial intelligence for architects and designers is to guide future designers, in general, and architects, in particular, to support the social and cultural wellbeing of the humanity in a digital and global environment. This objective is today essential but also extremely large, interdisciplinary and interartistic, so we have done just a brief introduction of the subject. We will start with the argument fixed by the Professor Jonas Langer in his web some years ago, that we have defined as: “The Langer’s Tree”.Primera edició
An Adjectival Interface for procedural content generation
Includes abstract.Includes bibliographical references.In this thesis, a new interface for the generation of procedural content is proposed, in which the user describes the content that they wish to create by using adjectives. Procedural models are typically controlled by complex parameters and often require expert technical knowledge. Since people communicate with each other using language, an adjectival interface to the creation of procedural content is a natural step towards addressing the needs of non-technical and non-expert users. The key problem addressed is that of establishing a mapping between adjectival descriptors, and the parameters employed by procedural models. We show how this can be represented as a mapping between two multi-dimensional spaces, adjective space and parameter space, and approximate the mapping by applying novel function approximation techniques to points of correspondence between the two spaces. These corresponding point pairs are established through a training phase, in which random procedural content is generated and then described, allowing one to map from parameter space to adjective space. Since we ultimately seek a means of mapping from adjective space to parameter space, particle swarm optimisation is employed to select a point in parameter space that best matches any given point in adjective space. The overall result, is a system in which the user can specify adjectives that are then used to create appropriate procedural content, by mapping the adjectives to a suitable set of procedural parameters and employing the standard procedural technique using those parameters as inputs. In this way, none of the control offered by procedural modelling is sacrificed â although the adjectival interface is simpler, it can at any point be stripped away to reveal the standard procedural model and give users access to the full set of procedural parameters. As such, the adjectival interface can be used for rapid prototyping to create an approximation of the content desired, after which the procedural parameters can be used to fine-tune the result. The adjectival interface also serves as a means of intermediate bridging, affording users a more comfortable interface until they are fully conversant with the technicalities of the underlying procedural parameters. Finally, the adjectival interface is compared and contrasted to an interface that allows for direct specification of the procedural parameters. Through user experiments, it is found that the adjectival interface presented in this thesis is not only easier to use and understand, but also that it produces content which more accurately reflects usersâ intentions