Successive log quantization for cost-efficient neural networks using stochastic computing

Despite the multifaceted benefits of stochastic computing (SC) such as low cost, low power, and flexible precision, SC-based deep neural networks (DNNs) still suffer from the long-latency problem, especially for those with high precision requirements. While log quantization can be of help, it has its own accuracy-saturation problem due to uneven precision distribution. In this paper we propose successive log quantization (SLQ), which extends log quantization with significant improvements in precision and accuracy, and apply it to state-of-the-art SC-DNNs. SLQ reuses the existing datapath of log quantization, and thus retains its advantages such as simple multiplier hardware. Our experimental results demonstrate that our SLQ can significantly extend both the accuracy and efficiency of SCDNNs over the state-of-the-art solutions, including linear-quantized and log-quantized SC-DNNs, achieving less than 1???1.5%p accuracy drop for AlexNet, SqueezeNet, and VGG-S at mere 4???5-bit weight resolution. ?? 2019 Copyright held by the owner/author(s)

Scalable stochastic-computing accelerator for convolutional neural networks

Stochastic Computing (SC) is an alternative design paradigm particularly useful for applications where cost is critical. SC has been applied to neural networks, as neural networks are known for their high computational complexity. However previous work in this area has critical limitations such as the fully-parallel architecture assumption, which prevent them from being applicable to recent ones such as convolutional neural networks, or ConvNets. This paper presents the first SC architecture for ConvNets, shows its feasibility, with detailed analyses of implementation overheads. Our SC-ConvNet is a hybrid between SC and conventional binary design, which is a marked difference from earlier SC-based neural networks. Though this might seem like a compromise, it is a novel feature driven by the need to support modern ConvNets at scale, which commonly have many, large layers. Our proposed architecture also features hybrid layer composition, which helps achieve very high recognition accuracy. Our detailed evaluation results involving functional simulation and RTL synthesis suggest that SC-ConvNets are indeed competitive with conventional binary designs, even without considering inherent error resilience of SC

MLogNet: A Logarithmic Quantization-Based Accelerator for Depthwise Separable Convolution

In this paper we propose a novel logarithmic quantization-based DNN (Deep Neural Network) architecture for depthwise separable convolution (DSC) networks. Our architecture is based on selective two-word logarithmic quantization (STLQ), which improves accuracy greatly over logarithmic-scale quantization while retaining the speed and area advantage of logarithmic quantization. On the other hand, it also comes with the synchronization problem due to variable-latency PEs (processing elements), which we address through a novel architecture and a compile-time optimization technique. Our architecture is dynamically reconfigurable to support various combinations of depthwise vs. pointwise convolution layers efficiently. Our experimental results using layers from MobileNetV2 and ShuffleNetV2 demonstrate that our architecture is significantly faster and more area-efficient than previous DSC accelerator architectures as well as previous accelerators utilizing logarithmic quantization

FPGA Implementation of Convolutional Neural Network Based on Stochastic Computing

There has been a body of research to use stochastic computing (SC) for the implementation of neural networks, in the hope that it will reduce the area cost and energy consumption. However, no working neural network system based on stochastic computing has been demonstrated to support the viability of SC-based deep neural networks in terms of both recognition accuracy and cost/energy efficiency. In this demonstration we present an SC-based deep nenural network system that is highly accurate and efficient. Our system takes an input image and processes it with a convolutional neural network implemented on an FPGA using stochastic computing to recognize the input image, with nearly the same accuracy as conventional binary implementations

Enhanced electrocatalytic full water-splitting reaction by interfacial electric field in 2D/2D heterojunction

To increase the productivity of hydrogen and oxygen generation, the carbon-based electrocatalyst with 2D/2D structure, graphitic carbon nitride with reduced graphene oxide (g-C3N4/rGO), was synthesized via a facile electrostatic self-assembly method and was used as electrocatalyst for full water-splitting. In the alkaline electrolyte, g-C3N4/rGO showed lower overpotential and Tafel slope of hydrogen evolution reaction (HER) and oxygen evolution reaction (OER), respectively, than the other 2D structured materials. The DFT calculation simulates the spontaneous electron transfer, low charge transfer resistance, and internal electric field of g-C3N4/rGO electrocatalysts. Especially, the internal electric field between the g-C3N4 and rGO is beneficial in solving the local pH reduction problem and electron transport. Consequentially, our study elucidates that 2D/2D structure, inducing the internal electric field, is an effective way to enhance the performance of the full water-splitting system

Molecular Design and Property Prediction of Sterically Confined Polyimides for Thermally Stable and Transparent Materials

To meet the demand for next-generation flexible optoelectronic devices, it is crucial to accurately establish the chemical structure-property relationships of new optical polymer films from a theoretical point of view, prior to production. In the current study, computer-aided simulations of newly designed poly(ester imide)s (PEsIs) with various side groups (–H, –CH3, and –CF3) and substituted positions were employed to study substituent-derived steric effects on their optical and thermal properties. From calculations of the dihedral angle distribution of the model compounds, it was found that the torsion angle of the C–N imide bonds was effectively constrained by the judicious introduction of di-, tetra-, and hexa-substituted aromatic diamines with –CF3 groups. A high degree of fluorination of the PEsI repeating units resulted in weaker intra- and intermolecular conjugations. Their behavior was consistent with the molecular orbital energies obtained using density functional theory (DFT). In addition, various potential energy components of the PEsIs were investigated, and their role in glass-transition behavior was studied. The van der Waals energy (EvdW) played a crucial role in the segmental chain motion, which had an abrupt change near glass-transition temperature (Tg). The more effective steric effect caused by –CF3 substituents at the 3-position of the 4-aminophenyl group significantly improved the chain rigidity, and showed high thermal stability (Tg > 731 K) when compared with the –CH3 substituent at the same position, by highly distorting (89.7°) the conformation of the main chain

In-situ Deposition of Graphene Oxide Catalyst for Efficient Photoelectrochemical Hydrogen Evolution Reaction Using Atmospheric Plasma

The vacuum deposition method requires high energy and temperature. Hydrophobic reduced graphene oxide (rGO) can be obtained by plasma-enhanced chemical vapor deposition under atmospheric pressure, which shows the hydrophobic surface property. Further, to compare the effect of hydrophobic and the hydrophilic nature of catalysts in the photoelectrochemical cell (PEC), the prepared rGO was additionally treated with plasma that attaches oxygen functional groups effectively to obtain hydrophilic graphene oxide (GO). The hydrogen evolution reaction (HER) electrocatalytic activity of the hydrophobic rGO and hydrophilic GO deposited on the p-type Si wafer was analyzed. Herein, we have proposed a facile way to directly deposit the surface property engineered GO

Structure, Dynamics, Receptor Binding, and Antibody Binding of the Fully Glycosylated Full-Length SARS-CoV-2 Spike Protein in a Viral Membrane.

The spike (S) protein of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) mediates host cell entry by binding to angiotensin-converting enzyme 2 (ACE2) and is considered the major target for drug and vaccine development. We previously built fully glycosylated full-length SARS-CoV-2 S protein models in a viral membrane including both open and closed conformations of the receptor-binding domain (RBD) and different templates for the stalk region. In this work, multiple μs-long all-atom molecular dynamics simulations were performed to provide deeper insights into the structure and dynamics of S protein and glycan functions. Our simulations reveal that the highly flexible stalk is composed of two independent joints and most probable S protein orientations are competent for ACE2 binding. We identify multiple glycans stabilizing the open and/or closed states of the RBD and demonstrate that the exposure of antibody epitopes can be captured by detailed antibody-glycan clash analysis instead of commonly used accessible surface area analysis that tends to overestimate the impact of glycan shielding and neglect possible detailed interactions between glycan and antibodies. Overall, our observations offer structural and dynamic insights into the SARS-CoV-2 S protein and potentialize for guiding the design of effective antiviral therapeutics