낸드 플래시 셀 스트링 기반의 시냅틱 어레이 아키텍처

학위논문(박사) -- 서울대학교대학원 : 공과대학 전기·정보공학부, 2021.8. 이종호.Neuromorphic computing using synaptic devices has been proposed to efficiently process vector-matrix multiplication (VMM) which is a significant task in DNN. Until now, resistive RAM (RRAM) was mainly used as synaptic devices for neuromorphic computing. However, a number of limitations still exist for RRAMs to implement a large-scale synaptic device array due to device nonideality such as variation, endurance and monolithic integration of RRAMs and CMOS peripheral circuits. Due to these problems, SRAM cells, which are mature silicon memory, have been proposed as synaptic devices. However, SRAM occupies large area (~150 F2 per bitcell) and on-chip SRAM capacity (~a few MB) is insufficient to accommodate a large number of parameters. In this dissertation, synaptic architectures based on NAND flash cell strings are proposed for off-chip learning and on-chip learning. A novel synaptic architecture based on NAND cell strings is proposed as a high-density synapse capable of XNOR operation for binary neural networks (BNNs) in off-chip learning. By changing the threshold voltage of NAND flash cells and input voltages in complementary fashion, the XNOR operation is successfully demonstrated. The large on/off current ratio (~7×105) of NAND flash cells can implement high-density and highly reliable BNNs without error correction codes. We propose a novel synaptic architecture based on a NAND flash memory for highly robust and high-density quantized neural networks (QNN) with 4-bit weight. Quantization training can minimize the degradation of the inference accuracy compared to post-training quantization. The proposed operation scheme can implement QNN with higher inference accuracy compared to BNN. On-chip learning can significantly reduce time and energy consumption during training, compensate the weight variation of synaptic devices, and can adapt to changing environment in real time. On-chip learning using the high-density advantage of NAND flash memory structure is of great significance. However, the conventional on-chip learning method used for RRAM array cannot be utilized when using NAND flash cells as synaptic devices because of the cell string structure of NAND flash memory. In this work, a novel synaptic array architecture enabling forward propagation (FP) and backward propagation (BP) in the NAND flash memory is proposed for on-chip learning. In the proposed synaptic architecture, positive synaptic weight and negative synaptic weight are separated in different array to enable weights to be transposed correctly. In addition, source-lines (SL) are separated, which is different from conventional NAND flash memory, to enable both the FP and BP in the NAND flash memory. By applying input and error input to bit-lines (BL) and string-select lines (SSL) in NAND cell array, respectively, accurate vector-matrix multiplication is successfully performed in both FP and BP eliminating the effect of pass cells. The proposed on-chip learning system is much more robust to weight variation compared to the off-chip learning system. Finally, superiority of the proposed on-chip learning architecture is verified by circuit simulation of a neural network.DNN에서 중요한 작업인 벡터-매트릭스 곱셈 (VMM)을 효율적으로 처리하기 위해 시냅스 소자를 사용하는 뉴로모픽 컴퓨팅이 활발히 연구되고 있다. 지금까지 RRAM (Resistive RAM)이 주로 뉴로모픽 컴퓨팅의 시냅스 소자로 사용되었다. 그러나 RRAM은 소자의 산포가 크고 신뢰성이 좋지 않으며 CMOS 주변 회로와 통합이 어려운 문제로 인해 대규모 시냅스 소자 어레이를 구현하는 데는 여전히 많은 제한이 있다. 이러한 문제로 인해 성숙한 실리콘 메모리인 SRAM 셀이 시냅스 소자로 제안되고 있다. 그러나 SRAM은 셀 당 면적 (~150 F2 per bitcell)이 크고 또한 온칩 SRAM 용량 (~a few MB) 은 많은 파라미터를 수용하기에 충분하지 않다. 본 논문에서는 오프 칩 학습과 온 칩 학습을 위해 NAND 플래시 셀 스트링을 기반으로 하는 시냅스 아키텍처를 제안한다. NAND 셀 스트링 기반의 새로운 시냅스 아키텍처는 오프 칩 학습에서 이진 신경망 (BNN)을 위한 XNOR 연산이 가능한 고밀도 시냅스로 사용된다. 상호 보완적인 방식으로 NAND 플래시 셀의 임계 전압과 입력 전압을 변경함으로써 XNOR 연산을 성공적으로 수행한다. NAND 플래시 셀의 큰 온/오프 전류 비율(~ 7x105)은 ECC 없이 고밀도 및 고신뢰성의 BNN을 구현할 수 있다. 우리는 4비트 가중치를 갖는 매우 견고하며 고집적의 양자화된 신경망(QNN)을 위한 NAND 플래시 메모리를 기반의 새로운 시냅틱 아키텍처를 제안한다. 양자화 학습은 훈련 후 양자화에 비해 추론 정확도의 저하를 최소화할 수 있다. 제안하는 동작 방식은 BNN에 비해 더 높은 추론 정확도를 가지는 QNN을 구현할 수 있다. 온 칩 학습은 훈련 중 시간과 에너지 소비를 크게 줄이고 시냅스 소자의 산포를 보상하며 변화하는 환경에 실시간으로 적응할 수 있다. NAND 플래시 메모리 구조의 높은 집적도를 사용한 온 칩 학습은 매우 유용하다. 그러나 기존의 RRAM 어레이에 사용되는 온 칩 학습 방법은 NAND 플래시 메모리의 셀 스트링 구조로 인해 NAND 플래시 셀을 시냅스 소자로 사용하는 경우 활용할 수 없다. 이 연구에서는 온 칩 학습을 위해 NAND 플래시 메모리에서 순방향 전파 (FP) 및 역방향 전파 (BP)를 가능하게 하는 새로운 시냅스 어레이 아키텍처를 제안한다. 제안된 시냅스 아키텍처에서는 가중치가 올바르게 전치될 수 있도록 양의 시냅스 가중치와 음의 시냅스 가중치가 서로 다른 어레이로 분리된다. 또한 기존 NAND 플래시 메모리와 달리 소스 라인 (SL)을 분리하여 NAND 플래시 메모리에서 순방향 전파와 역방향 전파를 모두 연산할 수 있다. NAND 셀 어레이의 비트 라인 (BL) 및 스트링 선택 라인 (SSL)에 각각 입력 및 오류 입력을 인가함으로써 PASS 셀의 효과를 제거하여 순방향 전파 및 역박향 전파 모두에서 정확한 벡터 행렬 곱셈이 성공적으로 수행되도록 한다. 제안된 온 칩 학습 시스템은 오프 칩 학습 시스템에 비해 소자의 산포에 대해 훨씬 영향이 적다. 마지막으로, 제안된 온 칩 학습 아키텍처의 우수성을 신경망의 회로 시뮬레이션을 통해 검증하였다.Chapter 1 Introduction 1 1.1 Background 1 Chapter 2 Binary neural networks based on NAND flash memory 7 2.1 Synaptic architecture for BNN 7 2.2 Measurement results 13 2.3 Binary neuron circuit 23 2.4 Simulation results 27 2.5 Differential scheme 32 2.5.1 Differential synaptic architecture 32 2.5.2 Simulation results 41 Chapter 3 Quantized neural networks based on NAND flash memory 47 3.1 Synaptic architecture for QNN 47 3.2 Measurement results 55 3.3 Simulation results 66 Chapter 4 On-chip learning based on NAND flash memory 74 4.1 Synaptic architecture for on-chip learning 74 4.2 Measurement results 82 4.3 Neuron circuits 90 4.4 Simulation results 93 Chapter 5 Conclusion 100 Bibliography 104 Abstract in Korean 111박

ClaPIM: Scalable Sequence CLAssification using Processing-In-Memory

DNA sequence classification is a fundamental task in computational biology with vast implications for applications such as disease prevention and drug design. Therefore, fast high-quality sequence classifiers are significantly important. This paper introduces ClaPIM, a scalable DNA sequence classification architecture based on the emerging concept of hybrid in-crossbar and near-crossbar memristive processing-in-memory (PIM). We enable efficient and high-quality classification by uniting the filter and search stages within a single algorithm. Specifically, we propose a custom filtering technique that drastically narrows the search space and a search approach that facilitates approximate string matching through a distance function. ClaPIM is the first PIM architecture for scalable approximate string matching that benefits from the high density of memristive crossbar arrays and the massive computational parallelism of PIM. Compared with Kraken2, a state-of-the-art software classifier, ClaPIM provides significantly higher classification quality (up to 20x improvement in F1 score) and also demonstrates a 1.8x throughput improvement. Compared with EDAM, a recently-proposed SRAM-based accelerator that is restricted to small datasets, we observe both a 30.4x improvement in normalized throughput per area and a 7% increase in classification precision

Validation practices for satellite based earth observation data across communities

Assessing the inherent uncertainties in satellite data products is a challenging task. Different technical approaches have been developed in the Earth Observation (EO) communities to address the validation problem which results in a large variety of methods as well as terminology. This paper reviews state-of-the-art methods of satellite validation and documents their similarities and differences. First the overall validation objectives and terminologies are specified, followed by a generic mathematical formulation of the validation problem. Metrics currently used as well as more advanced EO validation approaches are introduced thereafter. An outlook on the applicability and requirements of current EO validation approaches and targets is given

Synchronization Controller To Solve The Mismatched Sampling Rates For Acoustic Echo Cancellation

Aplikasi-aplikasi Suara melalui IP (VoIP) yang menggunakan set komunikasi bebas tangan semakin meluas digunakan. Voice over Internet Protocol (VoIP) applications are extensively used for handsfree communication (audio conferencing and video conferencing). Although handsfree communication systems may encounter acoustic echo problems, such problems can be solved using acoustic echo cancellation (AEC)

Advanced modeling of nanoscale devices for analog applications

L'abstract è presente nell'allegato / the abstract is in the attachmen

COMPUTE-IN-MEMORY WITH EMERGING NON-VOLATILE MEMORIES FOR ACCELERATING DEEP NEURAL NETWORKS

The objective of this research is to accelerate deep neural networks (DNNs) with emerging non-volatile memories (eNVMs) based compute-in-memory (CIM) architecture. The research first focuses on the inference acceleration and proposes a resistive random access memory (RRAM) based CIM architecture. Two generations of RRAM testchips which monolithically integrate the RRAM memory array and CMOS peripheral circuits are designed and fabricated using Winbond 90 nm and TSMC 40 nm commercial embedded RRAM process respectively. The first generation of testchip named XNOR-RRAM is dedicated for binary neural networks (BNNs) and the second generation named Flex-RRAM features 1bit-to-8bit run-time configurable precision and leverages the input sparsity of the DNN model to improve the throughput and energy efficiency. However, the non-ideal characteristics of eNVM devices, especially when utilized as multi-level analog synaptic weights, may incur a notable accuracy degradation for both training and inference. This research develops a PyTorch based framework that incorporates the device characteristics into the DNN model to evaluate the impact of the eNVM nonidealities on training/inference accuracy. The results suggest that it is challenging to directly use eNVMs for in-situ training and resistance drift remains as a critical challenge to maintain a high inference accuracy. Furthermore, to overcome the challenges posed by the asymmetric conductance tuning behavior of typical eNVMs, which is found to be the most critical nonideality that prevents the model from achieving software equivalent training accuracy, this research proposes a novel 2-transistor-1-FeFET (ferroelectric field effect transistor) based synaptic weight cell that exploits hybrid precision for in situ training and inference, which achieves near-software classification accuracy on MNIST and CIFAR-10 dataset.Ph.D

Sensing ECG signals with variable pulse width finite rate of innovation

Mobile health is gradually taking more importance in our society and the need of new power efficient devices acquiring biosignals for long periods of time is becoming substantial. In this thesis, we study the power reduction we could achieve on ECG sensing devices. Emphasis is made on reducing the number of samples both during the sensing phase and the compression phase. To that end, a new scheme called variable pulse width finite rate of innovation (VPW-FRI) is investigated. This new technique relies on the classical finite rate of innovation (FRI) theory and enables the use of a sum of asymmetric Cauchy-based pulses to model ECG signals. Research is done in order to implement VPW in practice and its performance are carefully analysed. Among others, we consider the potential instability of the method, we study its compression effectiveness and compare it with compression schemes widespread in the literature. We also evaluate the spectrum extrapolation performance of VPW when fed with signals sampled at sub-Nyquist rates and propose a modification that improves it. Furthermore, we introduce a method based on the similarities between different heart beats that reduces the computational costs of VPW. The parametric nature of VPW finally allows us to use it as a noise reduction algorithm. In parallel, we review and test a non-uniform sensing technique that adapts the sampling rate to the slope of the signal