An FPGA-Based On-Device Reinforcement Learning Approach using Online Sequential Learning

DQN (Deep Q-Network) is a method to perform Q-learning for reinforcement learning using deep neural networks. DQNs require a large buffer and batch processing for an experience replay and rely on a backpropagation based iterative optimization, making them difficult to be implemented on resource-limited edge devices. In this paper, we propose a lightweight on-device reinforcement learning approach for low-cost FPGA devices. It exploits a recently proposed neural-network based on-device learning approach that does not rely on the backpropagation method but uses OS-ELM (Online Sequential Extreme Learning Machine) based training algorithm. In addition, we propose a combination of L2 regularization and spectral normalization for the on-device reinforcement learning so that output values of the neural network can be fit into a certain range and the reinforcement learning becomes stable. The proposed reinforcement learning approach is designed for PYNQ-Z1 board as a low-cost FPGA platform. The evaluation results using OpenAI Gym demonstrate that the proposed algorithm and its FPGA implementation complete a CartPole-v0 task 29.77x and 89.40x faster than a conventional DQN-based approach when the number of hidden-layer nodes is 64

An On-Device Federated Learning Approach for Cooperative Anomaly Detection

Most edge AI focuses on prediction tasks on resource-limited edge devices while the training is done at server machines. However, retraining or customizing a model is required at edge devices as the model is becoming outdated due to environmental changes over time. To follow such a concept drift, a neural-network based on-device learning approach is recently proposed, so that edge devices train incoming data at runtime to update their model. In this case, since a training is done at distributed edge devices, the issue is that only a limited amount of training data can be used for each edge device. To address this issue, one approach is a cooperative learning or federated learning, where edge devices exchange their trained results and update their model by using those collected from the other devices. In this paper, as an on-device learning algorithm, we focus on OS-ELM (Online Sequential Extreme Learning Machine) to sequentially train a model based on recent samples and combine it with autoencoder for anomaly detection. We extend it for an on-device federated learning so that edge devices can exchange their trained results and update their model by using those collected from the other edge devices. This cooperative model update is one-shot while it can be repeatedly applied to synchronize their model. Our approach is evaluated with anomaly detection tasks generated from a driving dataset of cars, a human activity dataset, and MNIST dataset. The results demonstrate that the proposed on-device federated learning can produce a merged model by integrating trained results from multiple edge devices as accurately as traditional backpropagation based neural networks and a traditional federated learning approach with lower computation or communication cost

Alteration of intestinal flora by the intake of enzymatic degradation products of adlay (Coix lachryma-jobi L. var. ma-yuen Stapf) with improvement of skin condition

AbstractAdlay has been used as a traditional Chinese medicine and nutrient for its beneficial effects on bowel movements and skin care. This study examined the effect of enzymatic degradation product of adlay, “Super Hatomugi” (SPH) on human skin and the intestinal flora in a randomized, double-blind placebo-controlled study. The subjects were divided into three groups: 500mg SPH, 1000mg SPH, and placebo, taken daily for 4weeks. Hematological and skin condition examinations as well as an analysis of intestinal flora were performed 2weeks before and 10weeks after the start of the SPH intake. Skin condition was improved by SPH intake as revealed by a reduction in the number of nucleated epidermal cells. In addition, an increase in the fecal population of Bacteroidetes followed the SPH intake. These results show the possibility that SPH improves the skin condition and changes the proportions of intestinal flora

Chemical Characterization of a Volatile Dubnium Compound, DbOCl3

The formation and the chemical characterization of single atoms of dubnium (Db, element 105), in the form of its volatile oxychloride, was investigated using the on-line gas phase chromatography technique, in the temperature range 350–600 °C. Under the exactly same chemical conditions, comparative studies with the lighter homologues of Group 5 in the Periodic Table clearly indicate the volatility sequence being NbOCl3 > TaOCl3 ≥ DbOCl3. From the obtained experimental results, thermochemical data for DbOCl3 were derived. The present study delivers reliable experimental information for theoretical calculations on chemical properties of transactinides

母体免疫亢進時におけるインターロイキン6 の母胎間移行動態

ポスタ

Status Report of Neutral Kaon photo-production study using Neutral Kaon Spectrometer 2 (NKS2) at LNS-Tohoku(I. Nuclear Physics)

The approach described in this paper uses an array of Field Programmable Gate Array (FPGA) devices to implement a fault tolerant hardware system that can be compared to the running of fault tolerant software on a traditional processor. Fault tolerance is achieved is achieved by using FPGA with on the fly partial programmability feature. Major considerations while mapping to the FPGA includes the size of the area to be mapped and communication issues related to their communication. Area size selection is compared to the page size selection in Operating System Design. Communication issues between modules are compared to the software engineering paradigms dealing with module coupling, fan-in, fan-out and cohesiveness. Finally, the overhead associated with the downloading of the reconfiguration files is discussed

Nationwide surveillance of bacterial respiratory pathogens conducted by the surveillance committee of Japanese Society of Chemotherapy, the Japanese Association for Infectious Diseases, and the Japanese Society for Clinical Microbiology in 2010: General view of the pathogens\u27 antibacterial susceptibility

The nationwide surveillance on antimicrobial susceptibility of bacterial respiratory pathogens from patients in Japan, was conducted by Japanese Society of Chemotherapy, Japanese Association for Infectious Diseases and Japanese Society for Clinical Microbiology in 2010.The isolates were collected from clinical specimens obtained from well-diagnosed adult patients with respiratory tract infections during the period from January and April 2010 by three societies. Antimicrobial susceptibility testing was conducted at the central reference laboratory according to the method recommended by Clinical and Laboratory Standard Institutes using maximum 45 antibacterial agents.Susceptibility testing was evaluable with 954 strains (206 Staphylococcus aureus, 189 Streptococcus pneumoniae, 4 Streptococcus pyogenes, 182 Haemophilus influenzae, 74 Moraxella catarrhalis, 139 Klebsiella pneumoniae and 160 Pseudomonas aeruginosa). Ratio of methicillin-resistant S.aureus was as high as 50.5%, and those of penicillin-intermediate and -resistant S.pneumoniae were 1.1% and 0.0%, respectively. Among H.influenzae, 17.6% of them were found to be β-lactamase-non-producing ampicillin (ABPC)-intermediately resistant, 33.5% to be β-lactamase-non-producing ABPC-resistant and 11.0% to be β-lactamase-producing ABPC-resistant strains. Extended spectrum β-lactamase-producing K.pneumoniae and multi-drug resistant P.aeruginosa with metallo β-lactamase were 2.9% and 0.6%, respectively.Continuous national surveillance of antimicrobial susceptibility of respiratory pathogens is crucial in order to monitor changing patterns of susceptibility and to be able to update treatment recommendations on a regular basis