Distributive thermometer: A new unary encoding for weightless neural networks

The binary encoding of real valued inputs is a crucial part of Weightless Neural Networks. The Linear Thermometer and its variations are the most prominent methods to determine binary encoding for input data but, as they make assumptions about the input distribution, the resulting encoding is sub-optimal and possibly wasteful when the assumption is incorrect. We propose a new thermometer approach that doesn’t require such assumptions. Our results show that it achieves similar or better accuracy when compared to a thermometer that correctly assumes the distribution, and accuracy gains up to 26.3% when other thermometer representations assume an unsound distribution.info:eu-repo/semantics/publishedVersio

LogicWiSARD: Memoryless synthesis of weightless neural networks

Weightless neural networks (WNNs) are an alternative pattern recognition technique where RAM nodes function as neurons. As both training and inference require mostly table lookups, few additions, and no multiplications, WNNs are suitable for high-performance and low-power embedded applications. This work introduces a novel approach to implement WiSARD, the leading WNN state-of-the-art architecture, completely eliminating memories and arithmetic circuits and utilizing only logic functions. The approach creates compressed minimized implementations by converting trained WNN nodes from lookup tables to logic functions. The proposed LogicWiSARD is implemented in FPGA and ASIC technologies to illustrate its suitability for edge inference. Experimental results show more than 80% reduction in energy consumption when the proposed LogicWiSARD model is compared with a multilayer perceptron network (MLP) of equivalent accuracy. Compared to previous work on FPGA implementations for WNNs, convolutional neural networks, and binary neural networks, the energy savings of LogicWiSARD range between 32.2% and 99.6%.info:eu-repo/semantics/acceptedVersio

ULEEN: A Novel Architecture for Ultra Low-Energy Edge Neural Networks

The deployment of AI models on low-power, real-time edge devices requires accelerators for which energy, latency, and area are all first-order concerns. There are many approaches to enabling deep neural networks (DNNs) in this domain, including pruning, quantization, compression, and binary neural networks (BNNs), but with the emergence of the "extreme edge", there is now a demand for even more efficient models. In order to meet the constraints of ultra-low-energy devices, we propose ULEEN, a model architecture based on weightless neural networks. Weightless neural networks (WNNs) are a class of neural model which use table lookups, not arithmetic, to perform computation. The elimination of energy-intensive arithmetic operations makes WNNs theoretically well suited for edge inference; however, they have historically suffered from poor accuracy and excessive memory usage. ULEEN incorporates algorithmic improvements and a novel training strategy inspired by BNNs to make significant strides in improving accuracy and reducing model size. We compare FPGA and ASIC implementations of an inference accelerator for ULEEN against edge-optimized DNN and BNN devices. On a Xilinx Zynq Z-7045 FPGA, we demonstrate classification on the MNIST dataset at 14.3 million inferences per second (13 million inferences/Joule) with 0.21 $\mu$s latency and 96.2% accuracy, while Xilinx FINN achieves 12.3 million inferences per second (1.69 million inferences/Joule) with 0.31 $\mu$s latency and 95.83% accuracy. In a 45nm ASIC, we achieve 5.1 million inferences/Joule and 38.5 million inferences/second at 98.46% accuracy, while a quantized Bit Fusion model achieves 9230 inferences/Joule and 19,100 inferences/second at 99.35% accuracy. In our search for ever more efficient edge devices, ULEEN shows that WNNs are deserving of consideration.Comment: 14 pages, 14 figures Portions of this article draw heavily from arXiv:2203.01479, most notably sections 5E and 5F.

Bus line trajectories classification using weightless neural networks

Geo-enabled devices are ubiquitous nowadays. Within a diversity of possible applications using the huge of amount data generated by this technology, our work focuses on a chronic problem of Rio de Janeiro city: its public bus system. This text presents a framework for GPS trajectories classification, whose focus is the identification of bus routes of a public bus system. In order to do that, it was used the lightweight and versatile WiSARD, a weightless neural network classifier. Different binarization methods were used to adapt raw data to WiSARD’s binary input, making use of a set of rules defined by the application domain. Yet, it is evaluated a way of combining WiSARD through decision directed acyclic graphs. All these approachs result in different flavors of a neuro-symbolic learning system. The framework was tested against a vast data set created from open access and real-time data acquired from the current bus system of Rio de Janeiro city. Results obtained suggest the applicability of the proposed solution in a classification problem with more than 500 classes. Comparisons made also indicate an equivalent performance of WiSARD and other state-of-art and widely used machine learning methods. In addition, the framework described here is believed to be adaptable to other application domains.Dispositivos com localização espacial estão em toda parte hoje em dia. Dentre várias possíveis aplicações com a grande quantidade de dados gerada por esse tipo de equipamento, nosso trabalho foca em um problema crônico da cidade do Rio de Janeiro: seu sistema público de ônibus. Apresenta-se neste texto uma arquitetura para classificação de trajetórias GPS, cujo foco é a identificação de rotas de ônibus do sistema público. Para isso, utilizamos o leve e versátil classificador baseado em redes neurais sem peso WiSARD. Para a geração da entrada da rede, experimentamos diferentes formas de binarização, fazendo uso de regras definidas pelo problema. Ainda, avaliamos uma forma de combinação das redes WiSARD com o uso de um grafo acíclico de decisões. Todas essas propostas resultam em diferentes sabores de um sistema de aprendizado neurossimbólico. Tal arquitetura foi testada contra um vasto conjunto de dados construído a partir de dados fornecido em tempo real e de forma pública pelo sistema corrente da cidade do Rio de Janeiro. Os resultados obtidos indicam a aplicabilidade da solução proposta em um problema de classificação envolvendo mais de 500 classes. As comparações efetuadas indicam uma equiparação do modelo WiSARD com outros modelos em estado da arte. No mais, acreditamos que a metodologia aqui descrita possa ser utilizada com sucesso em outros domínios