Design Space Exploration of Neural Network Activation Function Circuits

The widespread application of artificial neural networks has prompted researchers to experiment with FPGA and customized ASIC designs to speed up their computation. These implementation efforts have generally focused on weight multiplication and signal summation operations, and less on activation functions used in these applications. Yet, efficient hardware implementations of nonlinear activation functions like Exponential Linear Units (ELU), Scaled Exponential Linear Units (SELU), and Hyperbolic Tangent (tanh), are central to designing effective neural network accelerators, since these functions require lots of resources. In this paper, we explore efficient hardware implementations of activation functions using purely combinational circuits, with a focus on two widely used nonlinear activation functions, i.e., SELU and tanh. Our experiments demonstrate that neural networks are generally insensitive to the precision of the activation function. The results also prove that the proposed combinational circuit-based approach is very efficient in terms of speed and area, with negligible accuracy loss on the MNIST, CIFAR-10 and IMAGENET benchmarks. Synopsys Design Compiler synthesis results show that circuit designs for tanh and SELU can save between 3.13-7.69 and 4.45-8:45 area compared to the LUT/memory-based implementations, and can operate at 5.14GHz and 4.52GHz using the 28nm SVT library, respectively. The implementation is available at: https://github.com/ThomasMrY/ActivationFunctionDemo.Comment: 5 pages, 5 figures, 16 conferenc

Emulating Digital Logic using Transputer Networks (Very High Parallelism = Simplicity = Performance)

Modern VLSI technology has changed the economic rules by which the balance between processing power, memory and communications is decided in computing systems. This will have a profound impact on the design rules for the controlling software. In particular, the criteria for judging efficiency of the algorithms will be somewhat different. This paper explores some of these implications through the development of highly parallel and highly distributable algorithms based on occam and transputer networks. The major results reported are a new simplicity for software designs, a corresponding ability to reason (formally and informally) about their properties, the reusability of their components and some real performance figures which demonstrate their practicality. Some guidelines to assist in these designs are also given. As a vehicle for discussion, an interactive simulator is developed for checking the functional and timing characteristics of digital logic circuits of arbitrary complexity

Automated performance evaluation of skew-tolerant clocking schemes

In this paper the authors evaluate the timing and power performance of three skew-tolerant clocking schemes. These schemes are the well known master–slave clocking scheme (MS) and two schemes developed by the authors: Parallel alternating latches clocking scheme (PALACS) and four-phase parallel alternating latches clocking scheme (four-phase PALACS). In order to evaluate the timing performance, the authors introduce algorithms to obtain the clock waveforms required by a synchronous sequential circuit. Separated algorithms were developed for every clocking scheme. From these waveforms it is possible to get parameters such as the non-overlapping time and the clock period. They have been implemented in a tool and have been used to compare the timing performance of the clocking schemes applied to a simple circuit. To analyse the power consumption the authors have electrically simulated a simple circuit for several operation frequencies. The most remarkable conclusion is that it is possible to save about 50% of the power consumption of the clock distribution network by using PALACS.Ministerio de Ciencia y Tecnología TEC 2004-00840/MI

Analog-digital simulation of transient-induced logic errors and upset susceptibility of an advanced control system

A simulation study is described which predicts the susceptibility of an advanced control system to electrical transients resulting in logic errors, latched errors, error propagation, and digital upset. The system is based on a custom-designed microprocessor and it incorporates fault-tolerant techniques. The system under test and the method to perform the transient injection experiment are described. Results for 2100 transient injections are analyzed and classified according to charge level, type of error, and location of injection

Philosophical foundations of the Death and Anti-Death discussion

Perhaps there has been no greater opportunity than in this “VOLUME FIFTEEN of our Death And Anti-Death set of anthologies” to write about how might think about life and how to avoid death. There are two reasons to discuss “life”, the first being enhancing our understanding of who we are and why we may be here in the Universe. The second is more practical: how humans meet the physical challenges brought about by the way they have interacted with their environment. Many persons discussing “life” beg the question about what “life” is. Surely, when one discusses how to overcome its opposite, death, they are not referring to another “living” thing such as a plant. There seems to be a commonality, though, and it is this commonality is one needing elaboration. It ostensibly seems to be the boundary condition separating what is completely passive (inert) from what attempts to maintain its integrity, as well as fulfilling other conditions we think “life” has. In our present discussion, there will be a reminder that it by no means has been unequivocally established what life really is by placing quotes around the word, namely, “life”. Consider it a tag representing a bundle of philosophical ideas that will be unpacked in this paper

Apollo experience report: Guidance and control systems: Command and service module entry monitor subsystem

The conceptual aspects of the command and service module entry monitor subsystem, together with an interpretation of the displays and their associated relationship to entry trajectory control, are presented. The entry monitor subsystem is described, and the problems encountered during the developmental phase and the first five manned Apollo flights are discussed in conjunction with the design improvements implemented

Applying Formal Methods to Networking: Theory, Techniques and Applications

Despite its great importance, modern network infrastructure is remarkable for the lack of rigor in its engineering. The Internet which began as a research experiment was never designed to handle the users and applications it hosts today. The lack of formalization of the Internet architecture meant limited abstractions and modularity, especially for the control and management planes, thus requiring for every new need a new protocol built from scratch. This led to an unwieldy ossified Internet architecture resistant to any attempts at formal verification, and an Internet culture where expediency and pragmatism are favored over formal correctness. Fortunately, recent work in the space of clean slate Internet design---especially, the software defined networking (SDN) paradigm---offers the Internet community another chance to develop the right kind of architecture and abstractions. This has also led to a great resurgence in interest of applying formal methods to specification, verification, and synthesis of networking protocols and applications. In this paper, we present a self-contained tutorial of the formidable amount of work that has been done in formal methods, and present a survey of its applications to networking.Comment: 30 pages, submitted to IEEE Communications Surveys and Tutorial