Bayesian Neural Bandit Using Online SWAG

학위논문(석사) -- 서울대학교대학원 : 데이터사이언스대학원 데이터사이언스학과, 2022. 8. 오민환.In this paper, we propose a Neural SWAG Bandit algorithm that combines a neural network-based bandit algorithm with Stochastic Weight Averaging Gaussian (SWAG), a Bayesian deep learning methodology. Neural Bandit is a bandit algorithm that uses the output of neural networks as an estimated reward. SWAG is a Bayesian Deep Learning method that samples parameters from the gaussian posterior distribution, which has been shown to have state-of-the-art performance and robustness compared to benchmark algorithms. By adapting SWAG into an online setting and combining it with Neural Bandit, we can leverage efficient sampling from deep neural networks while learning online. Our experiment results indicate that Neural SWAG Bandit benefits from Bayesian deep learning as well as exhibits superior performance compared to existing benchmark algorithms.1 INTRODUCTION 1 1.1 Bandit Algorithm 1 1.2 Neural Bandit 2 1.3 Bayesian Deep Learning 3 1.4 SWAG(Stochastic Weight Averaging Gaussian) 4 1.5 Contributions 4 2 BACKGROUND & RELATED WORK 5 2.1 Bandit Algorithm 5 2.1.1 Multi-armed Bandit 5 2.1.2 Contextual Bandit 6 2.1.3 Exploration & Exploitation tradeoff 6 2.1.4 Existing Bandit Algorithms 7 2.2 Neural Bandit 8 2.2.1 Neural Bandit 8 2.2.2 Existing Neural Bandit Algorithms 8 2.3 Bayesian Deep Learning 9 2.3.1 Bayesian and Uncertainty 9 2.3.2 Bayes Rule 9 2.3.3 Bayesian Neural Network 10 2.3.4 Existing Bayesian Deep Learning methods 10 2.4 SWAG Algorithm 11 2.4.1 SGD (Stochastic Gradient Descent) 11 2.4.2 SWA (Stochastic Weight Averaging) 12 2.4.3 SWAG-Diagonal 12 2.4.4 SWAG 12 3 THE NeuralSWAG ALGORITHM 14 4 EVALUATION METHODOLOGY 16 4.1 Cumulative Regret 16 5 EXPERIMENTS 17 5.1 Dataset 17 5.1.1 Simulation Dataset 17 5.1.2 Real-world Dataset 18 5.2 Model 19 5.3 Experiments setting 19 5.3.1 Setting for Simulation Datasets 19 5.3.2 Setting for Real-world Datasets 20 5.4 Compared Algorithms 21 5.5 Experimental Results 21 5.5.1 Results for Simulation Datasets 21 5.5.2 Results for Real-world Datasets 22 6 CONCLUSIONS 25 Bibliography 26석

Deep Ensembling with No Overhead for either Training or Testing:The All-Round Blessings of Dynamic Sparsity

The success of deep ensembles on improving predictive performance, uncertainty estimation, and out-of-distribution robustness has been extensively studied in the machine learning literature. Albeit the promising results, naively training multiple deep neural networks and combining their predictions at inference leads to prohibitive computational costs and memory requirements. Recently proposed efficient ensemble approaches reach the performance of the traditional deep ensembles with significantly lower costs. However, the training resources required by these approaches are still at least the same as training a single dense model. In this work, we draw a unique connection between sparse neural network training and deep ensembles, yielding a novel efficient ensemble learning framework called FreeTickets. Instead of training multiple dense networks and averaging them, we directly train sparse subnetworks from scratch and extract diverse yet accurate subnetworks during this efficient, sparse-to-sparse training. Our framework, FreeTickets, is defined as the ensemble of these relatively cheap sparse subnetworks. Despite being an ensemble method, FreeTickets has even fewer parameters and training FLOPs than a single dense model. This seemingly counter-intuitive outcome is due to the ultra training/inference efficiency of dynamic sparse training. FreeTickets surpasses the dense baseline in all the following criteria: prediction accuracy, uncertainty estimation, out-of-distribution (OoD) robustness, as well as efficiency for both training and inference. Impressively, FreeTickets outperforms the naive deep ensemble with ResNet50 on ImageNet using around only 1/5 of the training FLOPs required by the latter. We have released our source code at https://github.com/VITA-Group/FreeTickets.Comment: published in International Conference on Learning Representations (ICLR 2022

일반화 능력의 향상을 위한 깊은 신경망 양자화

학위논문(박사)--서울대학교 대학원 :공과대학 전기·컴퓨터공학부,2020. 2. 성원용.최근 깊은 신경망(deep neural network, DNN)은 영상, 음성 인식 및 합성 등 다양한 분야에서 좋은 성능을 보이고 있다. 하지만 대부분의 인공신경망은 많은 가중치(parameter) 수와 계산량을 요구하여 임베디드 시스템에서의 동작을 방해한다. 인공신경망은 낮은 정밀도에서도 잘 동작하는 인간의 신경세포를 모방하였기 떄문에 낮은 정밀도에서도 잘 동작할 가능성을 가지고 있다. 인공신경망의 양자화(quantization)는 이러한 특징을 이용한다. 일반적으로 깊은 신경망 고정소수점 양자화는 8-bit 이상의 단어길이에서 부동소수점과 유사한 성능을 얻을 수있지만, 그보다 낮은 1-, 2-bit에서는 성능이 떨어진다. 이러한 문제를 해결하기 위해 기존 연구들은 불균형 양자화기나 적응적 양자화 등의 더 정밀한 인공신경망 양자화 방법을 사용하였다. 본 논문은 기존의 연구와 매우 다른 방법을 제시한다. 본 연구는 고정 소수점 네트워크의 일반화능력을 향상시키는데 초점을 맞추었으며, 이를 위해 재훈련(retraining) 알고리즘에 기반하여 양자화된 인공신경망의 성능을 분석한다. 성능 분석은 레이어별 민감도 측정(layer-wise sensitivity analysis)에 기반한다. 또한 양자화 모델의 넓이와 깊이에 따른 성능도 분석한다. 분석된 결과를 바탕으로 양자화 스텝 적응 훈련법(quantization step size adaptation)과 점진적 양자화 훈련 방법(gradual quantization)을 제안한다. 양자화된 신경망 훈련시 양자화 노이즈를 적당히 조정하여 손실 평면(loss surface)상에 평평한 미니마(minima)에 도달 할 수 있는 양자화 훈련 방법 또한 제안한다. HLHLp (high-low-high-low-precision)로 명명된 훈련 방법은 양자화 정밀도를 훈련중에 높게-낮게-높게-낮게 바꾸면서 훈련한다. 훈련률(learning rate)도 양자화 스텝 사이즈를 고려하여 유동적으로 바뀐다. 제안하는 훈련방법은 일반적인 방법으로 훈련된 양자화 모델에 비해 상당히 좋은 성능을 보였다. 또한 선훈련된 선생 모델로 학생 모델을 훈련하는 지식 증류(knowledge distillation, KD) 기술을 이용하여 양자화의 성능을 높이는 방법을 제안한다. 특히 선생 모델을 선택하는 방법과 지식 증류의 하이퍼파라미터가 성능에 미치는 영향을 분석한다. 부동소수점 선생모델과 양자화 된 선생 모델을 사용하여 훈련 시킨 결과 선생 모델이 만들어내는 소프트맥스(softmax) 분포가 지식증류학습 결과에 크게 영향을 주는 것을 발견하였다. 소프트맥스 분포는 지식증류의 하이퍼파라미터들을 통해 조절될수 있으므로 지식증류 하이퍼파라미터들간의 연관관계 분석을 통해 높은 성능을 얻을 수 있었다. 또한 점진적으로 소프트 손실 함수 반영 비율을 훈련중에 줄여가는 점진적 소프트 손실 감소(gradual soft loss reducing)방법을 제안하였다. 뿐만 아니라 여러 양자화모델을 평균내어 높은 일반화 능력을 갖는 양자화 모델을 얻는 훈련 방법인 확률 양자화 가중치 평균(stochastic quantized weight averaging, SQWA) 훈련법을 제안한다. 제안하는 방법은 (1) 부동소수점 훈련, (2) 부동소수점 모델의 직접 양자화(direct quantization), (3) 재훈련(retraining)과정에서 진동 훈련율(cyclical learning rate)을 사용하여 휸련율이 진동내에서 가장 낮을 때 모델들을 저장, (4) 저장된 모델들을 평균, (5) 평균 된 모델을 낮은 훈련율로 재조정 하는 다중 단계 훈련법이다. 추가로 양자화 가중치 도메인에서 여러 양자화 모델들을 하나의 손실평면내에 동시에 나타낼 수 있는 심상(visualization) 방법을 제안한다. 제안하는 심상 방법을 통해 SQWA로 훈련된 양자화 모델은 손실평면의 가운데 부분에 있다는 것을 보였다.Deep neural networks (DNNs) achieve state-of-the-art performance for various applications such as image recognition and speech synthesis across different fields. However, their implementation in embedded systems is difficult owing to the large number of associated parameters and high computational costs. In general, DNNs operate well using low-precision parameters because they mimic the operation of human neurons; therefore, quantization of DNNs could further improve their operational performance. In many applications, word-length larger than 8 bits leads to DNN performance comparable to that of a full-precision model; however, shorter word-length such as those of 1 or 2 bits can result in significant performance degradation. To alleviate this problem, complex quantization methods implemented via asymmetric or adaptive quantizers have been employed in previous works. In contrast, in this study, we propose a different approach for quantization of DNNs. In particular, we focus on improving the generalization capability of quantized DNNs (QDNNs) instead of employing complex quantizers. To this end, first, we analyze the performance characteristics of quantized DNNs using a retraining algorithm; we employ layer-wise sensitivity analysis to investigate the quantization characteristics of each layer. In addition, we analyze the differences in QDNN performance for different quantized network sizes. Based on our analyses, two simple quantization training techniques, namely \textit{adaptive step size retraining} and \textit{gradual quantization} are proposed. Furthermore, a new training scheme for QDNNs is proposed, which is referred to as high-low-high-low-precision (HLHLp) training scheme, that allows the network to achieve flat minima on its loss surface with the aid of quantization noise. As the name suggests, the proposed training method employs high-low-high-low precision for network training in an alternating manner. Accordingly, the learning rate is also abruptly changed at each stage. Our obtained analysis results include that the proposed training technique leads to good performance improvement for QDNNs compared with previously reported fine tuning-based quantization schemes. Moreover, the knowledge distillation (KD) technique that utilizes a pre-trained teacher model for training a student network is exploited for the optimization of the QDNNs. We explore the effect of teacher network selection and investigate that of different hyperparameters on the quantization of DNNs using KD. In particular, we use several large floating-point and quantized models as teacher networks. Our experiments indicate that, for effective KD training, softmax distribution produced by a teacher network is more important than its performance. Furthermore, because softmax distribution of a teacher network can be controlled using KD hyperparameters, we analyze the interrelationship of each KD component for QDNN training. We show that even a small teacher model can achieve the same distillation performance as a larger teacher model. We also propose the gradual soft loss reducing (GSLR) technique for robust KD-based QDNN optimization, wherein the mixing ratio of hard and soft losses during training is controlled. In addition, we present a new QDNN optimization approach, namely \textit{stochastic quantized weight averaging} (SQWA), to design low-precision DNNs with good generalization capability using model averaging. The proposed approach includes (1) floating-point model training, (2) direct quantization of weights, (3) capture of multiple low-precision models during retraining with cyclical learning rate, (4) averaging of the captured models, and (5) re-quantization of the averaged model and its fine-tuning with low learning rate. Additionally, we present a loss-visualization technique for the quantized weight domain to elucidate the behavior of the proposed method. Our visualization results indicate that a QDNN optimized using our proposed approach is located near the center of the flat minimum on the loss surface.1.Introduction 1 1.1 Quantization of Deep Neural Networks 1 1.2 Generalization Capability of DNNs 3 1.3 Improved Generalization Capability of QDNNs 3 1.4 Outline of the Dissertation 5 2. Analysis of Fixedpoint Quantization of Deep Neural Networks 6 2.1 Introduction 6 2.2 Fixedpoint Performance Analysis of Deep Neural Networks 8 2.2.1 Model Design of Deep Neural Networks 8 2.2.2 Retrainbased Weight Quantization 10 2.2.3 Quantization Sensitivity Analysis 12 2.2.4 Empirical Analysis 13 2.3 Step Size Adaptation and Gradual Quantization for Retraining of DeepNeural Networks 22 2.3.1 Stepsize adaptation during retraining 22 2.3.2 Gradual quantization scheme 24 2.3.3 Experimental Results 24 2.4 Concluding remarks 30 3. HLHLp:Quantized Neural Networks Training for Reaching Flat Minimain Loss Surface 32 3.1 Introduction 32 3.2 Related Works 33 3.2.1 Quantization of Deep Neural Networks 33 3.2.2 Flat Minima in Loss Surfaces 34 3.3 Training QDNN for IMproved Generalization Capability 35 3.3.1 Analysis of Training with Quantized Weights 35 3.3.2 Highlowhighlowprecision Training 38 3.4 Experimental Results 40 3.4.1 Image Classification with CNNs 41 3.4.2 Language Modeling on PTB and WikiText2 44 3.4.3 Speech Recognition on WSJ Corpus 48 3.4.4 Discussion 49 3.5 Concluding Remarks 55 4 Knowledge Distillation for Optimization of Quantized Deep Neural Networks 56 4.1 Introduction 56 4.2 Quantized Deep Neural Netowrk Training Using Knowledge Distillation 57 4.2.1 Quantization of deep neural networks and knowledge distillation 58 4.2.2 Teacher model selection for KD 59 4.2.3 Discussion on hyperparameters of KD 62 4.3 Experimental Results 62 4.3.1 Experimental setup 62 4.3.2 Results on CIFAR10 and CIFAR100 64 4.3.3 Model size and temperature 66 4.3.4 Gradual Soft Loss Reducing 68 4.4 Concluding Remarks 68 5 SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of LowPrecision Deep Neural Networks 70 5.1 Introduction 70 5.2 Related works 71 5.2.1 Quantization of deep neural networks for efficient implementations 71 5.2.2 Stochastic weight averaging and losssurface visualization 72 5.3 Quantization of DNN and loss surface visualization 73 5.3.1 Quantization of deep neural networks 73 5.3.2 Loss surface visualization for QDNNs 75 5.4 SQWA algorithm 76 5.5 Experimental results 80 5.5.1 CIFAR100 80 5.5.2 ImageNet 87 5.6 Concluding remarks 90 6 Conclusion 92 Abstract (In Korean) 110Docto

Coupled Ensembles of Neural Networks

We investigate in this paper the architecture of deep convolutional networks. Building on existing state of the art models, we propose a reconfiguration of the model parameters into several parallel branches at the global network level, with each branch being a standalone CNN. We show that this arrangement is an efficient way to significantly reduce the number of parameters without losing performance or to significantly improve the performance with the same level of performance. The use of branches brings an additional form of regularization. In addition to the split into parallel branches, we propose a tighter coupling of these branches by placing the "fuse (averaging) layer" before the Log-Likelihood and SoftMax layers during training. This gives another significant performance improvement, the tighter coupling favouring the learning of better representations, even at the level of the individual branches. We refer to this branched architecture as "coupled ensembles". The approach is very generic and can be applied with almost any DCNN architecture. With coupled ensembles of DenseNet-BC and parameter budget of 25M, we obtain error rates of 2.92%, 15.68% and 1.50% respectively on CIFAR-10, CIFAR-100 and SVHN tasks. For the same budget, DenseNet-BC has error rate of 3.46%, 17.18%, and 1.8% respectively. With ensembles of coupled ensembles, of DenseNet-BC networks, with 50M total parameters, we obtain error rates of 2.72%, 15.13% and 1.42% respectively on these tasks