Consistent Explanations in the Face of Model Indeterminacy via Ensembling

This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy, which arises due to the existence of multiple (nearly) equally well-performing models for a given dataset and task. Despite their similar performance, such models often exhibit inconsistent or even contradictory explanations for their predictions, posing challenges to end users who rely on these models to make critical decisions. Recognizing this issue, we introduce ensemble methods as an approach to enhance the consistency of the explanations provided in these scenarios. Leveraging insights from recent work on neural network loss landscapes and mode connectivity, we devise ensemble strategies to efficiently explore the underspecification set -- the set of models with performance variations resulting solely from changes in the random seed during training. Experiments on five benchmark financial datasets reveal that ensembling can yield significant improvements when it comes to explanation similarity, and demonstrate the potential of existing ensemble methods to explore the underspecification set efficiently. Our findings highlight the importance of considering model indeterminacy when interpreting explanations and showcase the effectiveness of ensembles in enhancing the reliability of explanations in machine learning

Context-Adaptive Deep Neural Networks via Bridge-Mode Connectivity

The deployment of machine learning models in safety-critical applications comes with the expectation that such models will perform well over a range of contexts (e.g., a vision model for classifying street signs should work in rural, city, and highway settings under varying lighting/weather conditions). However, these one-size-fits-all models are typically optimized for average case performance, encouraging them to achieve high performance in nominal conditions but exposing them to unexpected behavior in challenging or rare contexts. To address this concern, we develop a new method for training context-dependent models. We extend Bridge-Mode Connectivity (BMC) (Garipov et al., 2018) to train an infinite ensemble of models over a continuous measure of context such that we can sample model parameters specifically tuned to the corresponding evaluation context. We explore the definition of context in image classification tasks through multiple lenses including changes in the risk profile, long-tail image statistics/appearance, and context-dependent distribution shift. We develop novel extensions of the BMC optimization for each of these cases and our experiments demonstrate that model performance can be successfully tuned to context in each scenario.Comment: Accepted to the NeurIPS 2022 ML Safety Worksho

from heuristic methods to certified methods

학위논문(박사) -- 서울대학교대학원 : 자연과학대학 수리과학부, 2021.8. 이재욱.Deep learning has shown successful results in many applications. However, it has been demonstrated that deep neural networks are vulnerable to small but adversarially designed perturbations in the input which can fool the neural network. There have been many studies on such adversarial attacks and defenses against them. However, Athalye et al. [1] have shown that most defenses rely on specific predefined adversarial attacks and can be completely broken by stronger adaptive attacks. Thus, certified methods are proposed to guarantee stable prediction of input within a perturbation set. We present this transition from heuristic defense to certified defense, and investigate key features of certified defenses, tightness and smoothness.딥러닝은 다양한 분야에서 성공적인 성능를 보여주고 있다. 그러나 심층신경망은 적대적 공격이라 불리우는, 입력값에 작은 섭동을 주어 신경망을 사용자가 원치 않는 방향으로 행동하도록 하는 공격에 취약하다. 적대적 공격의 발견 이후로, 다양한 적대적 공격과 이에 대한 방어 방법론과 관련하여 많은 연구들이 진행되었다. 그러나 Athalye et al. [1] 에서 대부분의 기존 방어 방법론들이 특정 적대적 공격만을 가정하고 설계되어 더 강한 적응가능한 적대적 공격에 의해 공격 가능하다는 문제점이 밝혀졌다. 따라서 입력값에 대해 섭동가능한 영역내에서 안정적인 행동을 보증할 수 있는 검증가능한 방법론이 제안되어왔다. 본 학위 논문에서는, 휴리스틱 방법론과 검증가능한 방법론에 대해 알아보고, 검증가능한 방법론에서 중요한 요소인 상한의 밀착성과 목적함수의 매끄러움에 대해서 분석한다.1 Introduction 1 2 Heuristic Defense 3 2.1 Heuristic Defense 3 2.1.1 Background 3 2.2 Gradient diversity regularization 5 2.2.1 Randomized neural network 5 2.2.2 Expectation over Transformation (EOT) 5 2.2.3 GradDiv 6 2.2.4 Experiments 11 3 Certified Defense 21 3.1 Certified Defense 21 3.1.1 Background 21 3.2 Tightness of the upper bound 24 3.2.1 Lipschitz-certifiable training with tight outer bound 24 3.2.2 Experiments 31 3.3 Smoothness of the objective 36 3.3.1 Background 36 3.3.2 What factors influence the performance of certifiable training? 39 3.3.3 Tightness and smoothness 46 3.3.4 Experiments 47 4 Conclusion and Open Problems 58 Appendix A Appendix for 2.2 60 A.1 Experimental Settings 60 A.1.1 Network Architectures 60 A.1.2 Batch-size, Training Epoch, Learning rate decay,Warmup, and Ramp-up periods 61 A.2 Variants of GradDiv-mean (2.2.17) 61 A.3 Additional Results on "Effects of GradDiv during Training" 61 A.4 Additional Results on Table 2.1 62 A.5 In the case of n > 20 in Figure 2.7 62 A.6 RSE [48] as a baseline 62 Appendix B Appendix for 3.2 68 B.1 The proof of the proposition 3.1.1 68 B.2 Outer Bound Propagation 69 B.2.1 Intuition behind BCP 69 B.2.2 Power iteration algorithm 69 B.2.3 The circumscribed box $out_\infty(h^{(k+1)}(\mathbb{B}^{(k)}_2))$ 71 B.2.4 BCP through residual layers 71 B.2.5 Complexity Analysis 72 B.3 Experimental Settings 72 B.3.1 Data Description 72 B.3.2 Hyper-parameters 73 B.3.3 Network architectures 73 B.3.4 Additional Experiments 74 Appendix C Appendix for 3.3 81 C.1 Experimental Settings 81 C.1.1 Settings in Section 3.3.2 82 C.1.2 Settings in Table 3.4 83 C.2 Interval Bound Propagation (IBP) 84 C.3 Details on Linear Relaxation 84 C.3.1 Linear relaxation explained in CROWN [79] 84 C.3.2 Dual Optimization View 85 C.4 Learning curves for variants of CROWN-IBP 87 C.5 Mode Connectivity 87 C.6 ReLU 91 C.7 $\beta$- and $\kappa$-schedulings 91 C.8 one-step vs multi-step 92 C.9 Train with $\epsilon_{train}\geq\epsilon_{test}$ 92 C.9.1 $\epsilon_{train}\geq\epsilon_{test}$ on MNIST 92 C.9.2 $\epsilon_{train}=1.1\epsilon_{test}$ on CIFAR-10 93 C.10 Training time 94 C.11 Loss and Tightness violin plots 95 C.12 Comparison with CAP-IBP 95 C.13 ReLU Stability 95 Bibliography 103 Abstract (in Korean) 113박

Revisiting Deep Ensemble for Out-of-Distribution Detection: A Loss Landscape Perspective

Existing Out-of-Distribution (OoD) detection methods address to detect OoD samples from In-Distribution data (InD) mainly by exploring differences in features, logits and gradients in Deep Neural Networks (DNNs). We in this work propose a new perspective upon loss landscape and mode ensemble to investigate OoD detection. In the optimization of DNNs, there exist many local optima in the parameter space, or namely modes. Interestingly, we observe that these independent modes, which all reach low-loss regions with InD data (training and test data), yet yield significantly different loss landscapes with OoD data. Such an observation provides a novel view to investigate the OoD detection from the loss landscape and further suggests significantly fluctuating OoD detection performance across these modes. For instance, FPR values of the RankFeat method can range from 46.58% to 84.70% among 5 modes, showing uncertain detection performance evaluations across independent modes. Motivated by such diversities on OoD loss landscape across modes, we revisit the deep ensemble method for OoD detection through mode ensemble, leading to improved performance and benefiting the OoD detector with reduced variances. Extensive experiments covering varied OoD detectors and network structures illustrate high variances across modes and also validate the superiority of mode ensemble in boosting OoD detection. We hope this work could attract attention in the view of independent modes in the OoD loss landscape and more reliable evaluations on OoD detectors