Zero-shot knowledge distillation in deep networks

Knowledge distillation deals with the problem of training a smaller model (Student) from a high capacity source model (Teacher) so as to retain most of its performance. Existing approaches use either the training data or meta-data extracted from it in order to train the Student. However, accessing the dataset on which the Teacher has been trained may not always be feasible if the dataset is very large or it poses privacy or safety concerns (e.g., bio-metric or medical data). Hence, in this paper, we propose a novel data-free method to train the Student from the Teacher. Without even using any meta-data, we synthesize the Data Impressions from the complex Teacher model and utilize these as surrogates for the original training data samples to transfer its learning to Student via knowledge distillation. We, therefore, dub our method "Zero-Shot Knowledge Distillation" and demonstrate that our framework results in competitive generalization performance as achieved by distillation using the actual training data samples on multiple benchmark datasets

Determining the Relevance of Features for Deep Neural Networks

Deep neural networks are tremendously successful in many applications, but end-to-end trained networks often result in hard to un- derstand black-box classifiers or predictors. In this work, we present a novel method to identify whether a specific feature is relevant to a clas- sifiers decision or not. This relevance is determined at the level of the learned mapping, instead of for a single example. The approach does neither need retraining of the network nor information on intermedi- ate results or gradients. The key idea of our approach builds upon con- cepts from causal inference. We interpret machine learning in a struc- tural causal model and use Reichenbachs common cause principle to infer whether a feature is relevant. We demonstrate empirically that the method is able to successfully evaluate the relevance of given features on three real-life data sets, namely MS COCO, CUB200 and HAM10000

Abstracts of National Conference on Research and Developments in Material Processing, Modelling and Characterization 2020

This book presents the abstracts of the papers presented to the Online National Conference on Research and Developments in Material Processing, Modelling and Characterization 2020 (RDMPMC-2020) held on 26th and 27th August 2020 organized by the Department of Metallurgical and Materials Science in Association with the Department of Production and Industrial Engineering, National Institute of Technology Jamshedpur, Jharkhand, India. Conference Title: National Conference on Research and Developments in Material Processing, Modelling and Characterization 2020Conference Acronym: RDMPMC-2020Conference Date: 26–27 August 2020Conference Location: Online (Virtual Mode)Conference Organizer: Department of Metallurgical and Materials Engineering, National Institute of Technology JamshedpurCo-organizer: Department of Production and Industrial Engineering, National Institute of Technology Jamshedpur, Jharkhand, IndiaConference Sponsor: TEQIP-