Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks

As Deep Neural Networks (DNNs) have demonstrated superhuman performance in a variety of fields, there is an increasing interest in understanding the complex internal mechanisms of DNNs. In this paper, we propose Relative Attributing Propagation (RAP), which decomposes the output predictions of DNNs with a new perspective of separating the relevant (positive) and irrelevant (negative) attributions according to the relative influence between the layers. The relevance of each neuron is identified with respect to its degree of contribution, separated into positive and negative, while preserving the conservation rule. Considering the relevance assigned to neurons in terms of relative priority, RAP allows each neuron to be assigned with a bi-polar importance score concerning the output: from highly relevant to highly irrelevant. Therefore, our method makes it possible to interpret DNNs with much clearer and attentive visualizations of the separated attributions than the conventional explaining methods. To verify that the attributions propagated by RAP correctly account for each meaning, we utilize the evaluation metrics: (i) Outside-inside relevance ratio, (ii) Segmentation mIOU and (iii) Region perturbation. In all experiments and metrics, we present a sizable gap in comparison to the existing literature. Our source code is available in \url{https://github.com/wjNam/Relative_Attributing_Propagation}.Comment: 8 pages, 7 figures, Accepted paper in AAAI Conference on Artificial Intelligence (AAAI), 202

Design of Composite Double-Slab Radar Absorbing Structures Using Forward, Inverse, and Tandem Neural Networks

The survivability and mission of a military aircraft is often designed with minimum radar cross section (RCS) to ensure its long-term operation and maintainability. To reduce aircraft’s RCS, a specially formulated Radar Absorbing Structures (RAS) is primarily applied to its external skins. A Ni-coated glass/epoxy composite is a recent RAS material system designed for decreasing the RCS for the X-band (8.2 – 12.4 GHz), while maintaining efficient and reliable structural performance to function as the skin of an aircraft. Experimentally measured and computationally predicted radar responses (i.e., return loss responses in specific frequency ranges) of multi-layered RASs are expensive and labor-intensive. Solving their inverse problems for optimal RAS design is also challenging due to their complex configuration and physical phenomena. An artificial neural network (ANN) is a machine learning method that uses existing data from experimental results and validated models (i.e., transfer learning) to predict complex behavior. Training an ANN can be computationally expensive; however, training is a one-time cost. In this work, three different Three ANN models are presented for designing dual slab Ni-coated glass/epoxy composite RASs: (1) the feedforward neural network (FNN) model, (2) the inverse neural network (INN) model – an inverse network, which maintains a parallel structure to the FNN model, and (3) the tandem neural network (TNN) model – an alternative to the INN model which uses a pre-trained FNN in the training process. The FNN model takes the thicknesses of dual slab RASs to predict their returns loss in the X-band range. The INN model solves the inverse problem for the FNN model. The TNN model is established with a pre-trained FNN to train an INN that exactly reverses the operation done in the FNN rather than solving the inverse problem independently. These ANN models will assist in reducing the time and cost for designing dual slab (and further extension to multi-layered) RASs

Interpreting Deep Neural Networks with Relative Sectional Propagation by Analyzing Comparative Gradients and Hostile Activations

The clear transparency of Deep Neural Networks (DNNs) is hampered by complex internal structures and nonlinear transformations along deep hierarchies. In this paper, we propose a new attribution method, Relative Sectional Propagation (RSP), for fully decomposing the output predictions with the characteristics of class-discriminative attributions and clear objectness. We carefully revisit some shortcomings of backpropagation-based attribution methods, which are trade-off relations in decomposing DNNs. We define hostile factor as an element that interferes with finding the attributions of the target and propagate it in a distinguishable way to overcome the non-suppressed nature of activated neurons. As a result, it is possible to assign the bi-polar relevance scores of the target (positive) and hostile (negative) attributions while maintaining each attribution aligned with the importance. We also present the purging techniques to prevent the decrement of the gap between the relevance scores of the target and hostile attributions during backward propagation by eliminating the conflicting units to channel attribution map. Therefore, our method makes it possible to decompose the predictions of DNNs with clearer class-discriminativeness and detailed elucidations of activation neurons compared to the conventional attribution methods. In a verified experimental environment, we report the results of the assessments: (i) Pointing Game, (ii) mIoU, and (iii) Model Sensitivity with PASCAL VOC 2007, MS COCO 2014, and ImageNet datasets. The results demonstrate that our method outperforms existing backward decomposition methods, including distinctive and intuitive visualizations.Comment: 9 pages, 8 figures, Accepted paper in AAAI Conference on Artificial Intelligence (AAAI), 202

Development and characterization of polymeric hollow fiber membrane with high CO2 separation performance

In this study, we prepared the polyimide based hollow fiber membrane with High CO2 permeance property. In other to prepare high permeable gas separation membrane, we synthesized novel polyimide material using 6FDA, Durene and PEG monomers. And then general property of the polyimide membrane is characterized using flat sheet type of membrane. The membranes were prepared under various controlled conditions such as retention time and concentration of the polymer. And then the hollow fiber membrane is also prepared and then characterized for confirmation of their potential. The Ionic liquid mainchain polymer is also developed to investigate the gas permeability and potential for utilization to coating materials of hollow fiber membrane. Polyimide with pendant ionic liquid (Im-PpC) membrane showed the high α(CO2/N2) value and the main chain polymer prepared by UV crosslinking with PEG & ILMP crosslinker also showed high α(CO2/N2). Please click Additional Files below to see the full abstract