4,639 research outputs found
Stochastic Digital Circuits for Probabilistic Inference
We introduce combinational stochastic logic, an abstraction that generalizes deterministic digital circuit design (based on Boolean logic gates) to the probabilistic setting. We show how this logic can be combined with techniques from contemporary digital design to generate stateless and stateful circuits for exact and approximate sampling from a range of probability distributions. We focus on Markov chain Monte Carlo algorithms for Markov random fields, using massively parallel circuits. We implement these circuits on commodity reconfigurable logic and estimate the resulting performance in time, space and price. Using our approach, these simple and general algorithms could be affordably run for thousands of iterations on models with hundreds of thousands of variables in real time
Stochastic thermodynamics of computation
One of the major resource requirements of computers - ranging from biological
cells to human brains to high-performance (engineered) computers - is the
energy used to run them. Those costs of performing a computation have long been
a focus of research in physics, going back to the early work of Landauer. One
of the most prominent aspects of computers is that they are inherently
nonequilibrium systems. However, the early research was done when
nonequilibrium statistical physics was in its infancy, which meant the work was
formulated in terms of equilibrium statistical physics. Since then there have
been major breakthroughs in nonequilibrium statistical physics, which are
allowing us to investigate the myriad aspects of the relationship between
statistical physics and computation, extending well beyond the issue of how
much work is required to erase a bit. In this paper I review some of this
recent work on the `stochastic thermodynamics of computation'. After reviewing
the salient parts of information theory, computer science theory, and
stochastic thermodynamics, I summarize what has been learned about the entropic
costs of performing a broad range of computations, extending from bit erasure
to loop-free circuits to logically reversible circuits to information ratchets
to Turing machines. These results reveal new, challenging engineering problems
for how to design computers to have minimal thermodynamic costs. They also
allow us to start to combine computer science theory and stochastic
thermodynamics at a foundational level, thereby expanding both.Comment: 111 pages, no figures. arXiv admin note: text overlap with
arXiv:1901.0038
Automated Synthesis of Unconventional Computing Systems
Despite decades of advancements, modern computing systems which are based on the von Neumann architecture still carry its shortcomings. Moore\u27s law, which had substantially masked the effects of the inherent memory-processor bottleneck of the von Neumann architecture, has slowed down due to transistor dimensions nearing atomic sizes. On the other hand, modern computational requirements, driven by machine learning, pattern recognition, artificial intelligence, data mining, and IoT, are growing at the fastest pace ever. By their inherent nature, these applications are particularly affected by communication-bottlenecks, because processing them requires a large number of simple operations involving data retrieval and storage. The need to address the problems associated with conventional computing systems at the fundamental level has given rise to several unconventional computing paradigms. In this dissertation, we have made advancements for automated syntheses of two types of unconventional computing paradigms: in-memory computing and stochastic computing. In-memory computing circumvents the problem of limited communication bandwidth by unifying processing and storage at the same physical locations. The advent of nanoelectronic devices in the last decade has made in-memory computing an energy-, area-, and cost-effective alternative to conventional computing. We have used Binary Decision Diagrams (BDDs) for in-memory computing on memristor crossbars. Specifically, we have used Free-BDDs, a special class of binary decision diagrams, for synthesizing crossbars for flow-based in-memory computing. Stochastic computing is a re-emerging discipline with several times smaller area/power requirements as compared to conventional computing systems. It is especially suited for fault-tolerant applications like image processing, artificial intelligence, pattern recognition, etc. We have proposed a decision procedures-based iterative algorithm to synthesize Linear Finite State Machines (LFSM) for stochastically computing non-linear functions such as polynomials, exponentials, and hyperbolic functions
Accelerating Stochastic Random Projection Neural Networks
Artificial Neural Network (ANN), a computational model based on the biological neural networks, has a recent resurgence in machine intelligence with breakthrough results in pattern recognition, speech recognition, and mapping. This has led to a growing interest in designing dedicated hardware substrates for ANNs with a goal of achieving energy efficiency, high network connectivity and better computational capabilities that are typically not optimized in software ANN stack. Using stochastic computing is a natural choice to reduce the total system energy, where a signal is expressed through the statistical distribution of the logical values as a random bit stream. Generally, the accuracy of these systems is correlated with the stochastic bit stream length and requires long compute times.
In this work, a framework is proposed to accelerate the long compute times in stochastic ANNs. A GPU acceleration framework has been developed to validate two random projection networks to test the efficacy of these networks prior to custom hardware design. The networks are stochastic extreme learning machine, a supervised feed-forward neural network and stochastic echo state network, a recurrent neural network with online learning. The framework also provisions identifying optimal values for various network parameters like learning rate, number of hidden layers and stochastic number length. The proposed stochastic extreme learning machine design
is validated for two standardized datasets, MNIST dataset and orthopedic dataset. The proposed stochastic echo state network is validated on the time series EEG dataset. The CPU models were developed for each of these networks to calculate the relative performance boost. The design knobs for performance boost include stochastic bit stream generation, activation function, reservoir layer and training unit of the networks. Proposed stochastic extreme learning machine and stochastic echo state network achieved a performance boost of 60.61x for Orthopedic dataset and 42.03x for EEG dataset with 2^12 bit stream length when tested on an Nvidia GeForce1050 Ti
Toward a formal theory for computing machines made out of whatever physics offers: extended version
Approaching limitations of digital computing technologies have spurred
research in neuromorphic and other unconventional approaches to computing. Here
we argue that if we want to systematically engineer computing systems that are
based on unconventional physical effects, we need guidance from a formal theory
that is different from the symbolic-algorithmic theory of today's computer
science textbooks. We propose a general strategy for developing such a theory,
and within that general view, a specific approach that we call "fluent
computing". In contrast to Turing, who modeled computing processes from a
top-down perspective as symbolic reasoning, we adopt the scientific paradigm of
physics and model physical computing systems bottom-up by formalizing what can
ultimately be measured in any physical substrate. This leads to an
understanding of computing as the structuring of processes, while classical
models of computing systems describe the processing of structures.Comment: 76 pages. This is an extended version of a perspective article with
the same title that will appear in Nature Communications soon after this
manuscript goes public on arxi
- …