Mixed-precision weights network for field-programmable gate array

In this study, we introduced a mixed-precision weights network (MPWN), which is a quantization neural network that jointly utilizes three different weight spaces: binary {−1,1}, ternary {−1,0,1}, and 32-bit floating-point. We further developed the MPWN from both software and hardware aspects. From the software aspect, we evaluated the MPWN on the Fashion-MNIST and CIFAR10 datasets. We systematized the accuracy sparsity bit score, which is a linear combination of accuracy, sparsity, and number of bits. This score allows Bayesian optimization to be used efficiently to search for MPWN weight space combinations. From the hardware aspect, we proposed XOR signed-bits to explore floating-point and binary weight spaces in the MPWN. XOR signed-bits is an efficient implementation equivalent to multiplication of floating-point and binary weight spaces. Using the concept from XOR signed bits, we also provide a ternary bitwise operation that is an efficient implementation equivalent to the multiplication of floating-point and ternary weight space. To demonstrate the compatibility of the MPWN with hardware implementation, we synthesized and implemented the MPWN in a field-programmable gate array using high-level synthesis. Our proposed MPWN implementation utilized up to 1.68-4.89 times less hardware resources depending on the type of resources than a conventional 32-bit floating-point model. In addition, our implementation reduced the latency up to 31.55 times compared to 32-bit floating-point model without optimizations

FPGA Implementation of a Binarized Dual Stream Convolutional Neural Network for Service Robots

In this study, with the aim of installing an object recognition algorithm on the hardware device of a service robot, we propose a Binarized Dual Stream VGG-16 (BDS-VGG16) network model to realize high-speed computations and low power consumption. The BDS-VGG16 model has improved in terms of the object recognition accuracy by using not only RGB images but also depth images. It achieved a 99.3% accuracy in tests using an RGB-D Object Dataset. We have also confirmed that the proposed model can be installed in a field-programmable gate array (FPGA). We have further installed BDS-VGG16 Tiny, a small BDS-VGG16 model in XCZU9EG, a system on a chip with a CPU and a middle-scale FPGA on a single chip that can be installed in robots. We have also integrated the BDS-VGG16 Tiny with a robot operating system. As a result, the BDS-VGG16 Tiny installed in the XCZU9EG FPGA realizes approximately 1.9-times more computations than the one installed in the graphics processing unit (GPU) with a power efficiency approximately 8-times higher than that installed in the GPU

Object Recognition System using Deep Learning with Depth Images for Service Robots

In an aging society with fewer children, service robots are expected to play an increasingly important role in people’s lives. To realize a future with service robots, a generic object recognition system is necessary to recognize a wide variety of objects with a high degree of accuracy. Therefore, this study employs deep convolutional neural networks for the generic object recognition system. To improve the accuracy of object recognition, both RGB images and depth images can be used effectively. In this paper, we propose a new architecture “Dual Stream - VGG16 (DS-VGG16)” for a deep convolutional neural network to train both the RGB images and depth images, and we also present a new training method for the proposed architecture. The experimental results indicate that the proposed architecture and training method are effective. Finally, we develop an object recognition system based on the proposed method that has an interface of robot operating system for integrating the system into service robots.2018 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS2018), 27 - 30, November 2018, Okinawa, Japa

A hardware-oriented echo state network and its FPGA implementation

This paper proposes implementation of an Echo State Network (ESN) to Field Programmable Gate Array (FPGA). The proposed method is able to reduce hardware resources by using fixed-point operation, quantization of weights, which includes accumulate operations and efficient dataflow modules. The performance of the designed circuit is verified via experiments including prediction of sine and cosine waves. Experimental result shows that the proposed circuit supports to 200 MHz of operation frequency and facilitates faster computing of the ESN algorithm compared with a central processing unit

Semi-automatic Dataset Generation for Object Detection and Recognition and its Evaluation on Domestic Service Robots

This paper proposes a method for the semi-automatic generation of a dataset for deep neural networks to perform end-to-end object detection and classification from images, which is expected to be applied to domestic service robots. In the proposed method, the background image of the floor or furniture is first captured. Subsequently, objects are captured from various viewpoints. Then, the background image and the object images are composited by the system (software) to generate images of the virtual scenes expected to be encountered by the robot. At this point, the annotation files, which will be used as teaching signals by the deep neural network, are automatically generated, as the region and category of the object composited with the background image are known. This reduces the human workload for dataset generation. Experiment results showed that the proposed method reduced the time taken to generate a data unit from 167 s, when performed manually, to 0.58 s, i.e., by a factor of approximately 1/287. The dataset generated using the proposed method was used to train a deep neural network, which was then applied to a domestic service robot for evaluation. The robot was entered into the World Robot Challenge, in which, out of ten trials, it succeeded in touching the target object eight times and grasping it four times

Training autoencoder using three different reversed color models for anomaly detection

Autoencoders (AEs) have been applied in several applications such as anomaly detectors and object recognition systems. However, although the recent neural networks have relatively high accuracy but sometimes false detection may occur. This paper introduces AE as an anomaly detector. The proposed AE is trained using both normal and anomalous data based on convolutional neural network with three different color models Hue Saturation Value (HSV), Red Green Blue (RGB), and our own model (TUV). As a result, the trained AE reconstruct the normal images without change, whereas the anomalous image would be reconstructed reversely. The training and testing of the AE in case of RGB, HSV, and TUV color models were demonstrated and Cifar-10 dataset had been used for the evaluation process. It can be noticed that HSV color model has been more effective and achievable as an anomaly detector rather than other color models based on Z- and F-test analyses

Synchronization of Pulse-Coupled Phase Oscillators over Multi-FPGA Communication Links

This study proposes an implementation of pulse-coupled phase oscillators over multiple field-programmable-gate-array (FPGA) communication links. Two FPGAs are connected by a gigabit transceiver and a First-In First-Out interface. To verify the effect of communication delay between FPGAs on oscillator synchronization, we implement four oscillators on the multi-FPGA platform. We have successfully observed synchronization over two FPGAs correctly, despite of a 0.1 μs communication delay. The measurement results show that first spike synchronization requires 12.47 μs with a 3.2 Gbps communication throughput

Open Set Recognition Using the Feature Space of Deep Neural Networks

Image classification assumes that all classes used in testing are known. Therefore, when an unknown class data is input, it cannot be recognized correctly. A method that enables unknown classes to be identified is called open set recognition. In this paper, we propose a method of open set recognition focusing on the feature space of the classifier and a Mahalanobis-based threshold. The experimental results show that the proposed method surpasses state-of-the-art methods on some datasets, demonstrating the potential of a method focusing on the feature space.2021 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2021), November 16-19, 2021, Hualien, Taiwa