ONCHIP TRAINING OF SPIKING NEURAL ACCELERATORS USING SPIKE-TRAIN LEVEL DIRECT FEEDBACK ALIGNMENT

Spiking Neural Networks (SNNs) are widely researched in recent years and present a promising computing model. Several key properties including biologically plausible information processing and event driven sample learning make SNNs be able for ultra-low power neuromorphic hardware implementation. However, to achieve the same level of performance in training conventional deep artificial neural networks (ANNs), especially for networks with error backpropagation (BP) algorithm, is a significant challenge existing in SNNs training, which is due to inherent complex dynamics and non-differentiable spike activities of spiking neurons. To solve this problem, this thesis proposes the first study on realizing competitive spike-train level backpropagation (BP) like algorithms to enable on-chip BP training of SNNs. This novel alrogithm, called spike-train level direct feedback alignment (ST-DFA), performs better in computation complexity and training latency compared to traditional BP methods. Furthermore, algorithm and hardware cooptimization as well as efficient online neural signal computation are explored for on-chip implementation of ST-DFA. To figure out the performance of this proposed algorithm, the final online version of ST-DFA is tested on the Xilinx ZC706 FPGA board. During testing on real-world speech and image classification applications, it shows excellent performance vs. overhead tradeoffs. SNN neural processors with on-chip ST-DFA training show competitive classification accuracy of 97.23% for the MNIST dataset with 4X input resolution reduction and 87.40% for the challenging 16-speaker TI46 speech corpus, respectively. This experimental result is then compared to the hardware implementation of the state-of-the-art BP algorithm HM2-BP. While trading off classification performance very gracefully, the design of the proposed online ST-DFA training reduces functional resources by 76.7% and backward training latency by 31.6%, which dramatically cut resource and power demand for hardware implementatio

Synaptic weight modification and storage in hardware neural networks

In 2011 the International Technology Roadmap for Semiconductors, ITRS 2011, outlined how the semiconductor industry should proceed to pursue Moore’s Law past the 18nm generation. It envisioned a concept of ‘More than Moore’, in which existing semiconductor technologies can be exploited to enable the fabrication of diverse systems and in particular systems which integrate non-digital and biologically based functionality. A rapid expansion and growing interest in the fields of microbiology, electrophysiology, and computational neuroscience occurred. This activity has provided significant understanding and insight into the function and structure of the human brain leading to the creation of systems which mimic the operation of the biological nervous system. As the systems expand a need for small area, low power devices which replicate the important biological features of neural networks has been established to implement large scale networks. In this thesis work is presented which focuses on the modification and storage of synaptic weights in hardware neural networks. Test devices were incorporated on 3 chip runs; each chip was fabricated in a 0.35μm process from Austria MicroSystems (AMS) and used for parameter extraction, in accordance with the theoretical analysis presented. A compact circuit is presented which can implement STDP, and has advantages over current implementations in that the critical timing window for synaptic modification is implemented within the circuit. The duration of the critical timing window is set by the subthreshold current controlled by the voltage, Vleak, applied to transistor Mleak in the circuit. A physical model to predict the time window for plasticity to occur is formulated and the effects of process variations on the window is analysed. The STDP circuit is implemented using two dedicated circuit blocks, one for potentiation and one for depression where each block consists of 4 transistors and a polysilicon capacitor, and an area of 980µm2. SpectreS simulations of the back-annotated layout of the circuit and experimental results indicate that STDP with biologically plausible critical timing windows over the range 10µs to 100ms can be implemented. Theoretical analysis using parameters extracted from MOS test devices is used to describe the operation of each device and circuit presented. Simulation results and results obtained from fabricated devices confirm the validity of these designs and approaches. Both the WP and WD circuits have a power consumption of approximately 2.4mW, during a weight update. If no weight update occurs the resting currents within the device are in the nA range, thus each circuit has a power consumption of approximately 1µW. A floating gate, FG, device fabricated using a standard CMOS process is presented. This device is to be integrated with both the WP and WD STDP circuits. The FG device is designed to store negative charge on a FG to represent the synaptic weight of the associated synapse. Charge is added or removed from the FG via Fowler-Nordheim tunnelling. This thesis outlines the design criteria and theoretical operation of this device. A model of the charge storage characteristics is presented and verified using HFCV and PCV experimental results. Limited precision weights, LPW, and its potential use in hardware neural networks is also considered. LPW offers a potential solution in the quest to design a compact FG device for use with CTS. The algorithms presented in this thesis show that LPW allows for a reduction in the synaptic weight storage device while permitting the network to function as intended

DALGIÇ MOTORLAR ÜZERİNE BİR İNCELEME

Submersible induction motors which are widely used in oil wells and agricultural irrigations are very similar to the normal asynchronous motors in terms of design and features. A high current density, cooling with water and long of lengths are important features of these motors. In this paper, the studies related with submersible motors in the literature are deal with and analyzed. Besides, the emphasis was placed on improvements to the design and performance characteristics. In the literature, there are many studies on induction motors but are limited studies on submersible induction motors.Petrol kuyularında ve tarımsal sulamalarda yaygın olarak kullanılan dalgıç asenkron motorlar, tasarımı ve özellikleri bakımından normal asenkron motorlara çok benzemektedir. Bu motorların, yüksek akım yoğunluğu ile çalışması, boylarının uzun olması ve su ile soğutulması önemli özellikleri olarak sıralanabilir. Bu makalede dalgıç asenkron motorlar konusunda literatürde yer alan çalışmalar ele alınmış ve bir değerlendirme yapılmıştır. Ayrıca tasarım ve performans karakteristikleri ile ilgili yapılan iyileştirmeler üzerinde durulmuştur. Literatürde, normal asenkron motorlarla ilgili birçok çalışma yer almasına rağmen dalgıç asenkron motorlarla ilgili çalışmaların sınırlı olduğu görülmektedir

Technologies of information transmission and processing

Сборник содержит статьи, тематика которых посвящена научно-теоретическим разработкам в области сетей телекоммуникаций, информационной безопасности, технологий передачи и обработки информации. Предназначен для научных сотрудников в области инфокоммуникаций, преподавателей, аспирантов, магистрантов и студентов технических вузов

Machine Learning of Lifestyle Data for Diabetes

Self-Monitoring of Blood Glucose (SMBG) for Type-2 Diabetes (T2D) remains highly challenging for both patients and doctors due to the complexities of diabetic lifestyle data logging and insufficient short-term and personalized recommendations/advice. The recent mobile diabetes management systems have been proved clinically effective to facilitate self-management. However, most such systems have poor usability and are limited in data analytic functionalities. These two challenges are connected and affected by each other. The ease of data recording brings better data for applicable data analytic algorithms. On the other hand, the irrelevant or inaccurate data input will certainly commit errors and noises. The output of data analysis, as potentially valuable patterns or knowledge, could be the incentives for users to contribute more data. We believe that the incorporation of machine learning technologies in mobile diabetes management could tackle these challenge simultaneously. In this thesis, we propose, build, and evaluate an intelligent mobile diabetes management system, called GlucoGuide for T2D patients. GlucoGuide conveniently aggregates varieties of lifestyle data collected via mobile devices, analyzes the data with machine learning models, and outputs recommendations. The most complicated part of SMBG is diet management. GlucoGuide aims to address this crucial issue using classification models and camera-based automatic data logging. The proposed model classifies each food item into three recommendation classes using its nutrient and textual features. Empirical studies show that the food classification task is effective. A lifestyle-data-driven recommendations framework in GlucoGuide can output short-term and personalized recommendations of lifestyle changes to help patients stabilize their blood glucose level. To evaluate performance and clinical effectiveness of this framework, we conduct a three-month clinical trial on human subjects, in collaboration with Dr. Petrella (MD). Due to the high cost and complexity of trials on humans, a small but representative subject group is involved. Two standard laboratory blood tests for diabetes are used before and after the trial. The results are quite remarkable. Generally speaking, GlucoGuide amounted to turning an early diabetic patient to be pre-diabetic, and pre-diabetic to non-diabetic, in only 3-months, depending on their before-trial diabetic conditions. cThis clinical dataset has also been expanded and enhanced to generate scientifically controlled artificial datasets. Such datasets can be used for varieties of machine learning empirical studies, as our on-going and future research works. GlucoGuide now is a university spin-off, allowing us to collect a large scale of practical diabetic lifestyle data and make potential impact on diabetes treatment and management

Uso de redes neurais artificiais como metamodelo na otimização por algoritmo PSO (particle swarm optimization') em problemas de mapeamento eletromagnético de ambientes

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia Elétrica, Florianópolis, 2017.Este trabalho se propõe a fazer uma análise de ferramentas de otimização e custo computacional através de um estudo de caso proposto por Grubisic (2012), que trata da otimização do posicionamento de antenas em sistemas de comunicação sem fio para ambientes interiores (indoor) por meio de meta-heurísticas populacionais associadas à Técnica de Traçado de Raios, em que algoritmos Genéticos (GA) e Otimizadores por Enxames de Partículas (PSO) foram as duas modalidades de meta-heurísticas utilizadas como ferramentas de otimização. A proposta desta tese baseou-se na utilização da técnica de traçado de raios quase 3D (RTQ3D) para produzir o valor dos campos eletromagnéticos iniciais e calcular a função de mérito (fitness) para 160 receptores de acordo com os possíveis posicionamentos de duas antenas a serem distribuídas no ambiente em questão. As variáveis do problema são compostas pelos valores dos campos magnéticos para os 160 receptores em função das posições das antenas das estações radiobase, que servem como dados de entrada para o algoritmo da Rede Neural Artificial, Perceptron multicamadas, com algoritmo de aprendizado backpropagation Real. Os valores dos campos magnéticos associados às posições das antenas por sua vez entram como valores a serem aprendidos pela rede, ou seja, o professor da RMLP. Após o aprendizado da Rede Neural Artificial, que é o metamodelo utilizado com o objetivo de realizar eficientemente os cálculos do otimizador, entra o otimizador por enxame de partículas (PSO) para efetuar o posicionamento ótimo das antenas com uma redução significativa no custo computacional. Por fim, um dos exemplos propostos por Grubisic (2012) foi implementado como estudo de caso desta pesquisa, utilizando essa nova estrutura de análise, PSO com RMLP, como metamodelo. Essa estrutura é bem recomendada para projetos eletromagnéticos, entretanto ainda não foi aplicada para esse tipo de análise. O objetivo principal seria a diminuição do custo computacional, que no caso em questão é bem significativo. Portanto, essa tese tem um caráter inédito em relação às ferramentas usadas e ao objetivo principal (redução do custo computacional).Abstract : This research has proposed to do an analysis of optimization tools and computational cost using a case study proposed by Grubisic (2012), which addressed optimization of the antennas positioning in wireless communication systems for indoor environments through meta-population heuristics associated with ray tracing technique, in which algorithms Genetic (GA) and Optimizers for Swarms of particles (PSO) were the two types of meta-heuristics used as optimization tools. The purpose of this thesis was based on the use of almost 3D ray tracing technique (RTQ3D) to produce the value of the initial electromagnetic fields and calculating the merit function (fitness) to 160 receivers according to the possible placements of two antennas which are distributed in the environment in a matter. The problem variables consist of the values of the magnetic fields to the 160 receivers depending on the positions of the antennas of the access points, which serve as input data for the algorithm of Artificial Neural Network, multilayer perceptron with Real backpropagation learning algorithm. The problem variables consist of the values of 160 magnetic fields to 160 receivers on the basis of the positions of the antennas of the access points, which serve as input data for the algorithm of Artificial Neural Network, multilayer perceptron with backpropagation real learning algorithm. The values of the magnetic fields associated with the positions of the antennas in turn to input values to be learned by the network, or the teacher RMLP. After learning of Artificial Neural Network, which is the metamodel used in order to enable the calculation of the optimizer, with a lower computational cost, the optimizer particle swarm enters (PSO) to make the optimum positioning of the antennas with a significant reduction the computational cost. Finally, one of the examples proposed by Grubisic (2012) is implemented as a case study of this research using this new analysis structure, PSO using RMLP as metamodel. This structure is well recommended for electromagnetic designs, but has not been applied to this type of analysis. The main objective would be to reduce the computational cost, which in this case is significant. Therefore, this thesis has a unique character in relation to the tools used and the main objective (reducing the computational cost)