신경망을 이진화하는 새로운 방법에 대한 연구

Abstract

학위논문 (석사)-- 서울대학교 대학원 : 전기·정보공학부, 2017. 2. 최기영.Artificial intelligence is one of the most important technologies, and deep neural network is one branch of artificial intelligence. A deep neural network consists of many neurons and synapses which mimic mammals brain. It has attracted many interests from academia and industry in various fields including computer vision and speech recognition for the last decade. It is well known that deep neural networks become more powerful with more layers and neurons. However, as deep neural networks grow larger, they suffer from the requirement of huge memory and computation. Therefore, reducing the overhead of handling them becomes one of key challenges in neural networks nowadays. There are many methodologies to address this issue such as weight quantization, weight pruning, and hashing. This thesis proposes a new approach to binarizing neural networks. It prunes weights and forces remaining weights to degenerate to binary values. Experimental results show that the proposed approach reduces the number of weights down to 5.35% in a fully connected neural network and down to 50.35% in a convolutional neural network. Compared to the floating point convolutional neural network, the proposed approach gives 98.9% reductions in computation and 93.6% reduction in power consumption without any accuracy loss.Chapter 1 Introduction 1 1.1 Thesis organization 2 Chapter 2 Related Work 4 2.1 Weights Pruning 4 2.2 Binarized Neural Network 6 2.3 Approximate Neural Network 9 Chapter 3 Proposed Approach 12 3.1 Motivational Example 12 3.2 Weights Compression 14 3.3 Multiplication in Activation Stage 17 Chapter 4 Implementation 19 Chapter 5 Experimental Result 24 5.1 Convolutional Neural Network 24 5.2 Fully-Connected Neural Network 32 Chapter 6 Conclusion and Future work 41 Bibliography 43 국문초록 46Maste