A reconfigurable stochastic architecture for highly reliable computing

Mounting concerns over variability, defects and noise motivate a new approach for integrated circuits: the design of stochastic logic, that is to say, digital circuitry that operates on probabilistic signals, and so can cope with errors and uncertainty. Techniques for prob-abilistic analysis are well established. We advocate a strategy for synthesis. In this paper, we present a reconfigurable architecture that implements the computation of arbitrary continuous functions with stochastic logic. We analyze the sources of error: approxima-tion, quantization, and random fluctuations. We demonstrate the ef-fectiveness of our method on a collection of benchmarks for image processing. Synthesis trials show that our stochastic architecture requires less area than conventional hardware implementations. It achieves a large speed up compared to software conventional im-plementations. Most importantly, it is much more tolerant of soft errors (bit flips) than these deterministic implementations

Projeto e treinamento de redes neurais recorrentes utilizando computação estocástica para síntese em FPGA

Trabalho de Conclusão de Curso (graduação)—Universidade de Brasília, Instituto de Ciências Exatas, Departamento de Ciência da Computação, 2019.Este trabalho apresenta o treinamento e implementação de redes neurais recorrentes uti- lizando computação estocástica. Um estudo teórico acerca de redes neurais foi feito, a fim de que fosse possível compreender os algoritmos de treinamento e utilização das redes adotadas no projeto. Foram implementadas redes neurais tradicionais e redes neurais recorrentes, além de redes neurais recorrentes utilizando computação estocástica em soft- ware, com a finalidade de validar a utilização de tal paradigma na construção destas redes para então sintetizá-las em hardware, através de chips FPGA, de modo que sejam avaliadas questões como precisão da saída, tamanho do circuito digital final e comparação com implementações utilizando ponto fixo e ponto flutuante. A utilização de computação estocástica permite a construção de circuitos digitais mais simples para realizar operações necessárias em redes neurais, possibilitando que grandes redes possam ser sintetizadas em FPGA e problemas mais complexos possam ser resolvidos através de hardware.This paper presents the implementation of recurrent neural networks using stochastic computation for training. A theoretical study about neural networks was made, so that it was possible to understand the training algorithms and use of networks adopted in the project. Traditional neural networks and recurrent neural networks were implemented, as well as recurrent neural networks using stochastic computation in software, in order to validate the use of such a paradigm in the construction of these networks and then syn- thesize them in hardware, using FPGA chips, so that issues such as output accuracy, final digital circuit size, and comparison with fixed point and floating point implementations are evaluated. The use of stochastic computing allows the construction of simpler digital circuits to perform necessary operations in neural networks, allowing large networks to be synthesized in FPGA and more complex problems to be solved through hardware

Resistive-RAM for Data Storage Applications.

Mainstream non-volatile memory technology, dominated by the floating gate transistor, has historically improved in density, performance and cost primarily by means of process scaling. This simple geometrical scaling now faces significant challenges due to constraints of electrostatics and reliability. Thus, novel non-transistor based memory paradigms are being widely explored. Among the various contenders for next generation storage technology, RRAM devices have got immense attention due to their high-speed, multilevel capability, scalability, simple structure, low voltage operation and high endurance. In this thesis, electrical and material characterization is carried out on a MIM device system and formation / annihilation of nanoscale filaments is shown to be the reason behind the resistance switching. The MIM system is optimized to include an in-cell resistor which is shown to improve device endurance and reduce stuck-at-one faults. For highest density, the devices were arranged in a crossbar geometry and vertically integrated on CMOS decoders to demonstrate the feasibility of practical data storage applications. Next, we show that these binary RRAM devices exhibit native stochastic nature of resistive switching. Even for a fixed voltage on the same device, the wait time associated with programming is not fixed and is random and broadly distributed. However, the probability of switching can be predicted and controlled by the programming pulse. These binary devices have been used to generate random bit-streams with predicable bias ratios in time and space domains. The ability to produce random bit-streams using binary resistive switching devices based on the native stochastic switching principle may potentially lead to novel non-von-Neumann computing paradigms. Further, sub-1nA operating current devices have been developed. This ultra-low current provides energy savings by minimizing programming, erase and read currents. Despite having such low currents, excellent retention, on/off ratio and endurance have been demonstrated. Finally a scalable approach to simple 3D stacking is discussed. By implementation of a vertical sidewall-based architecture, the number of critical lithography steps can be reduced. A vertical device structure based on a W / WOx / Pd material system is developed. This scalable architecture is well suited for development of analog memory and neuromorphic systems.PhDElectrical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/110461/1/sidgaba_1.pd

Designing Accurate and Low-Cost Stochastic Circuits.

Stochastic computing (SC) is an unconventional computing approach that processes data represented by pseudo-random bit-streams called stochastic numbers (SNs). It enables arithmetic functions to be implemented by tiny, low-power logic circuits, and is highly error-tolerant. These properties make SC practical for applications that need massive parallelism or operate in noisy environments where conventional binary designs are too costly or too unreliable. SC has recently come to be seen as an attractive choice for tasks such as biomedical image processing and decoding complex error-correcting codes. Despite its desirable properties, SC has features that limit its usefulness, including insufficient accuracy and an inadequate design theory. Accuracy is especially vulnerable to correlation among interacting SNs and to the random fluctuations inherent in SC’s data representation. This dissertation examines the major factors affecting accuracy using analytical and experimental approaches based on probability theory and circuit simulation, respectively. We devise methods to quantify the error effects in stochastic circuits by means of probabilistic transfer matrices and Bernouilli processes. These methods make it possible to compare the impact of errors on conventional and stochastic circuits under various conditions. We then analyze correlation in detail and show that correlation-induced errors can be reduced by the careful insertion of delay elements, a de-correlation technique called isolation. Noting that different logic functions can have the same stochastic behavior when constant SNs are applied to their inputs, we show how to partition logic functions into stochastic equivalence classes (SECs). We derive a procedure for identifying SECs, and apply SEC concepts to the synthesis and optimization of stochastic circuits. While addition, subtraction and multiplication have well-known and simple SC implementations, this is not true for division. We study stochastic division methods and propose a new type of stochastic divider that combines low cost with high accuracy. Finally, we turn to the design of general stochastic circuits and investigate a desirable property of SNs called monotonic progressive precision (MPP) whereby accuracy increases steadily with bit-stream length. We develop an SC design technique which produces results that are accurate and have good MPP. The dissertation concludes with some ideas for future research.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/133255/1/tehsuan_1.pd