The effect of heterogeneity on decorrelation mechanisms in spiking neural networks: a neuromorphic-hardware study

High-level brain function such as memory, classification or reasoning can be realized by means of recurrent networks of simplified model neurons. Analog neuromorphic hardware constitutes a fast and energy efficient substrate for the implementation of such neural computing architectures in technical applications and neuroscientific research. The functional performance of neural networks is often critically dependent on the level of correlations in the neural activity. In finite networks, correlations are typically inevitable due to shared presynaptic input. Recent theoretical studies have shown that inhibitory feedback, abundant in biological neural networks, can actively suppress these shared-input correlations and thereby enable neurons to fire nearly independently. For networks of spiking neurons, the decorrelating effect of inhibitory feedback has so far been explicitly demonstrated only for homogeneous networks of neurons with linear sub-threshold dynamics. Theory, however, suggests that the effect is a general phenomenon, present in any system with sufficient inhibitory feedback, irrespective of the details of the network structure or the neuronal and synaptic properties. Here, we investigate the effect of network heterogeneity on correlations in sparse, random networks of inhibitory neurons with non-linear, conductance-based synapses. Emulations of these networks on the analog neuromorphic hardware system Spikey allow us to test the efficiency of decorrelation by inhibitory feedback in the presence of hardware-specific heterogeneities. The configurability of the hardware substrate enables us to modulate the extent of heterogeneity in a systematic manner. We selectively study the effects of shared input and recurrent connections on correlations in membrane potentials and spike trains. Our results confirm ...Comment: 20 pages, 10 figures, supplement

양성 피드백 전계 효과 트랜지스터를 활용한 저전력 시냅스 소자

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 박병국.신경망 모방 시스템은 폰 노이만 구조의 계산 시스템이 가지는 약점인 복잡한 인식 문제를 해결과 에너지 소비의 효율성의 가능성으로 수년간 많은 분야에서 연구되고 있고 일부는 상용화 단계에까지 이르렀다. 이 신경 모방 시스템은 시냅스 모방 소자와 뉴런 회로로 이루어 지는데 시냅스 모방 소자는 신호전달과 기억 기능을 담당하고 있다. 시냅스는 전체 신경모방 시스템에서 가장 큰 부분을 차지 한다. 따라서 시스템내 대부분의 전력 소비가 시냅스 부분에서 일어나게 되므로 저전력 구현이 필수적인 요소다. 이런 이유로 저전력 소자에 특화된 소자인 터널 전계 효과 트랜지스터 (TFET), 네거티브 커페시터 전계효과 트랜지스터 (NCFET), 강유전계 효과 트랜지스터 (FeFET) 및 피드백 전계 효과 트랜지스터 (FBFET) 등이 연구되고 있다. 이런 다양한 소자중에 현재의 상보형 금속-산화물-반도체 (CMOS) 공정을 그대로 사용할 수 있는 피드백 전계 효과 트랜지스터는 뉴런 회로와 동시에 제작이 필요한 신경망 모방 시스템에서 대량 생산 가능성에 있어서 매우 유리하다. 본 논문에서는 이 피드백 전계 효과 트랜지스터를 기반으로 하고 NAND 플래시 메모리 구조에서 사용하는 파울러 노르다임 터널링(Fowler-Nordheim tunneling)을 방식으로 차치 트랩 층에 시냅스 소자의 가중치를 기억하는 방식의 시냅스 장치를 제안하고 있다. 해당 소자의 저전력 특성과 구동 방법을 테크놀로지 컴퓨터 지원 설계 (TCAD) 시뮬레이션을 사용하여 유효성을 확인 하였고, 서울대 반도체 공동 연구소 (ISRC) 의 CMOS 공정을 사용하여 소자를 제작하였고 전기적 특성 측정을 통해 제안된 방법을 확인 및 검증 하였다.The neuromorphic system has been widely used and commercialized in many fields in recent years due to its potential for complex problem solving and low energy consumption. The basic elements of this neuromorphic system are synapse and neuron circuit, in which synapse research is focused on emerging electronic devices such as resistive change memory (RRAM), phase-change memory (PCRAM), magnetoresistive random-access memory (MRAM), and FET-based devices. Synapse is responsible for the memory function of the neuromorphic system, that is, the current sum quantization with the specific weight value. and the neuron is responsible for integrating signals that have passed through the synapse and transmitting information to the next synapse. Since the synapse element is the largest portion of the whole system, It consumes most of the power of the entire system. So low power implementation is essential for the synapse device. In order to reduce power consumption, it is necessary to lower the off-current leakage and operate on low voltage. To overcome the limitation of MOSFETs in terms of ION/IOFF ratio, small sub-threshold swing and power consumption, various devices such as a tunneling field-effect transistor (TFET), negative capacitor field-effect transistor (NCFET), ferroelectric field-effect transistor (FeFET), and feedback field-effect transistor (FBFET) have been studied. Another important factor in synapse devices is the cost aspect. The deep learning technology that made Alpha-go exist is also an expensive system. As we can see from the coexistence of supercomputers and personal computers in the past, the development of low-cost chips that can be used by individuals, in the end, is inevitable. Because a CMOS compatible process must be possible since the neuron circuit is needed to fabricate at the same time, which helps to ensure mass productivity. FET-based devices are CMOS process compatible, which is suitable for the mass production environment. A positive FBFET (Feedback Field Effect Transistor) device has a very low sub-threshold current, SS (subthreshold swing) performance, and ION/IOFF ratio at the low operating voltage. We are proposing the synaptic device with a positive FBFET with a storage layer. From the simulation study, the operation method is studied for the weight modulation of the synaptic device and electrical measurement confirms accumulated charge change by program and erase condition each. These results for the synaptic transistor in this dissertation can be one of the candidates in low power neuromorphic systems.1 Introduction 1 1.1 Limitation of von Neumann Architecture computing 1 1.2 Biological Synapse 3 1.3 Spiking Neural Network (SNN) 5 1.4 Requirements of synaptic device 7 1.5 Advantage of Feedback Field-effect transistor (FBFET) 9 1.6 Outline of the Dissertation 10 2 Positive Feedback FET with storage layer 11 2.1 Normal operation Principle of FBFET 14 2.2 Operation Mechanism by Drain Input Pulse 16 2.3 Weight Modulation Mechanism 20 2.4 TCAD Simulation Result for Weighted Sum 23 2.5 TCAD Simulation Result for Program and Erase 28 2.6 Array structure and Inhibition scheme 31 3 Fabrication and Measurement 36 3.1 Fabrication process of FBFET synapse 37 3.2 Measurement result 41 3.3 Hysteresis Reduction 49 3.4 Temperature Compensation method 53 4 Modeling and High level simulation 56 4.1 Compact modeling for SPICE 56 4.2 SPICE simulation for VMM 60 5 Conclusion 64 5.1 Review of Overall Work 64 5.2 Future work 65 Abstract (In Korean) 75Docto