Distributive thermometer: A new unary encoding for weightless neural networks

The binary encoding of real valued inputs is a crucial part of Weightless Neural Networks. The Linear Thermometer and its variations are the most prominent methods to determine binary encoding for input data but, as they make assumptions about the input distribution, the resulting encoding is sub-optimal and possibly wasteful when the assumption is incorrect. We propose a new thermometer approach that doesn’t require such assumptions. Our results show that it achieves similar or better accuracy when compared to a thermometer that correctly assumes the distribution, and accuracy gains up to 26.3% when other thermometer representations assume an unsound distribution.info:eu-repo/semantics/publishedVersio

Pruning weightless neural networks

Weightless neural networks (WNNs) are a type of machine learning model which perform prediction using lookup tables (LUTs) instead of arithmetic operations. Recent advancements in WNNs have reduced model sizes and improved accuracies, reducing the gap in accuracy with deep neural networks (DNNs). Modern DNNs leverage “pruning” techniques to reduce model size, but this has not previously been explored for WNNs. We propose a WNN pruning strategy based on identifying and culling the LUTs which contribute least to overall model accuracy. We demonstrate an average 40% reduction in model size with at most 1% reduction in accuracy.info:eu-repo/semantics/publishedVersio

ULEEN: A Novel Architecture for Ultra Low-Energy Edge Neural Networks

The deployment of AI models on low-power, real-time edge devices requires accelerators for which energy, latency, and area are all first-order concerns. There are many approaches to enabling deep neural networks (DNNs) in this domain, including pruning, quantization, compression, and binary neural networks (BNNs), but with the emergence of the "extreme edge", there is now a demand for even more efficient models. In order to meet the constraints of ultra-low-energy devices, we propose ULEEN, a model architecture based on weightless neural networks. Weightless neural networks (WNNs) are a class of neural model which use table lookups, not arithmetic, to perform computation. The elimination of energy-intensive arithmetic operations makes WNNs theoretically well suited for edge inference; however, they have historically suffered from poor accuracy and excessive memory usage. ULEEN incorporates algorithmic improvements and a novel training strategy inspired by BNNs to make significant strides in improving accuracy and reducing model size. We compare FPGA and ASIC implementations of an inference accelerator for ULEEN against edge-optimized DNN and BNN devices. On a Xilinx Zynq Z-7045 FPGA, we demonstrate classification on the MNIST dataset at 14.3 million inferences per second (13 million inferences/Joule) with 0.21 $\mu$s latency and 96.2% accuracy, while Xilinx FINN achieves 12.3 million inferences per second (1.69 million inferences/Joule) with 0.31 $\mu$s latency and 95.83% accuracy. In a 45nm ASIC, we achieve 5.1 million inferences/Joule and 38.5 million inferences/second at 98.46% accuracy, while a quantized Bit Fusion model achieves 9230 inferences/Joule and 19,100 inferences/second at 99.35% accuracy. In our search for ever more efficient edge devices, ULEEN shows that WNNs are deserving of consideration.Comment: 14 pages, 14 figures Portions of this article draw heavily from arXiv:2203.01479, most notably sections 5E and 5F.

Heme doming in ferric cytochrome c: Femtosecond xray absorption and x-ray emission studies

The photoinduced dynamics of ferric Cytochrome c was investigated by ultrafast non-resonant X-ray emission (XES) and X-Ray Absorption (XAS) spectroscopies, and a cascade through high spin states accompanied by heme doming are observed for the first time

Ultrafast Energy Transfer from Photoexcited Tryptophan to the Haem in Cytochrome c

We report femtosecond Fe K-edge absorption (XAS) and nonresonant X-ray emission (XES) spectra of ferric cytochrome C (Cyt c) upon excitation of the haem (>300 nm) or mixed excitation of the haem and tryptophan (<300 nm). The XAS and XES transients obtained in both excitation energy ranges show no evidence for electron transfer processes between photoexcited tryptophan (Trp) and the haem, but rather an ultrafast energy transfer, in agreement with previous ultrafast optical fluorescence and transient absorption studies. The reported (J. Phys. Chem. B 2011, 115 (46), 13723-13730) decay times of Trp fluorescence in ferrous (∼350 fs) and ferric (∼700 fs) Cyt c are among the shortest ever reported for Trp in a protein. The observed time scales cannot be rationalized in terms of Förster or Dexter energy transfer mechanisms and call for a more thorough theoretical investigation

A WiSARD-based conditional branch predictor

Conditional branch prediction is a technique used to speculatively execute instructions before knowing the direction of conditional branch statements. Perceptron-based predictors have been extensively studied, however, they need large input sizes for the data to be linearly separable. To learn nonlinear functions from the inputs, we propose a conditional branch predictor based on the WiSARD model and compare it with two state-of-the-art predictors, the TAGE-SC-L and the Multiperspective Perceptron. We show that the WiSARD-based predictor with a smaller input size outperforms the perceptron-based predictor by about 0.09% and achieves similar accuracy to that of TAGE-SC-L.info:eu-repo/semantics/publishedVersio