Graph-Based Similarity of Neural Network Representations

Understanding the black-box representations in Deep Neural Networks (DNN) is an essential problem in deep learning. In this work, we propose Graph-Based Similarity (GBS) to measure the similarity of layer features. Contrary to previous works that compute the similarity directly on the feature maps, GBS measures the correlation based on the graph constructed with hidden layer outputs. By treating each input sample as a node and the corresponding layer output similarity as edges, we construct the graph of DNN representations for each layer. The similarity between graphs of layers identifies the correspondences between representations of models trained in different datasets and initializations. We demonstrate and prove the invariance property of GBS, including invariance to orthogonal transformation and invariance to isotropic scaling, and compare GBS with CKA. GBS shows state-of-the-art performance in reflecting the similarity and provides insights on explaining the adversarial sample behavior on the hidden layer space

Understanding the Dynamics of DNNs Using Graph Modularity

There are good arguments to support the claim that deep neural networks (DNNs) capture better feature representations than the previous hand-crafted feature engineering, which leads to a significant performance improvement. In this paper, we move a tiny step towards understanding the dynamics of feature representations over layers. Specifically, we model the process of class separation of intermediate representations in pre-trained DNNs as the evolution of communities in dynamic graphs. Then, we introduce modularity, a generic metric in graph theory, to quantify the evolution of communities. In the preliminary experiment, we find that modularity roughly tends to increase as the layer goes deeper and the degradation and plateau arise when the model complexity is great relative to the dataset. Through an asymptotic analysis, we prove that modularity can be broadly used for different applications. For example, modularity provides new insights to quantify the difference between feature representations. More crucially, we demonstrate that the degradation and plateau in modularity curves represent redundant layers in DNNs and can be pruned with minimal impact on performance, which provides theoretical guidance for layer pruning. Our code is available at https://github.com/yaolu-zjut/Dynamic-Graphs-Construction.Comment: Accepted by ECCV 202

Progressive changes in detrusor function and micturition patterns with chroinc bladder ischemia

Purpose: Lower urinary tract symptoms (LUTS) are bothersome constellation of voiding symptoms in men and women as they age. Multiple factors and comorbidities are attributed to this problem but underlying mechanisms of nonobstructive nonneurogenic detrusor overactivity, detrusor underactivity and LUTS remain largely unknown. Our goal was to characterize detrusor function and voiding patterns in relation to muscarinic receptors expression, nerve fiber density, and neural ultrastructure in chronic bladder ischemia. Materials and Methods: Iliac artery atherosclerosis and bladder ischemia were produced in male Sprague-Dawley rats. At 8 and 16 weeks after ischemia, micturition patterns and cystometrograms were recorded in conscious rats then bladder blood flow and nonvoiding spontaneous contractions were measured under general anesthesia. Bladder tissues were processed for Western blotting, immunostaining, and transmission electron microscopy. Results: Bladder responses to ischemic insult depended on the duration of ischemia. Micturition patterns and cystometric changes at 8-week ischemia suggested detrusor overactivity, while voiding behavior and cystometrograms at 16-week ischemia implied abnormal detrusor function resembling underactivity. Upregulation of muscarinic M2 receptor was found after 8- and 16 weeks of ischemia. Downregulation of M3 and upregulation of M1 were detected at 16-week ischemia. Neural structural damage and marked neurodegeneration were found after 8 and 16 weeks of ischemia, respectively. Conclusions: Prolonged ischemia may be a mediating variable in progression of overactive bladder to dysfunctional patterns similar to detrusor underactivity. The mechanism appears to involve differential expression of M1, M2, and M3 receptors, neural structural injury, and progressive loss of nerve fibers

LC-MS/MS Analysis Unravels Deep Oxidation of Manganese Superoxide Dismutase in Kidney Cancer

Manganese superoxide dismutase (MNSOD) is one of the major scavengers of reactive oxygen species (ROS) in mitochondria with pivotal regulatory role in ischemic disorders, inflammation and cancer. Here we report oxidative modification of MNSOD in human renal cell carcinoma (RCC) by the shotgun method using data-dependent liquid chromatography tandem mass spectrometry (LC-MS/MS). While 5816 and 5571 proteins were identified in cancer and adjacent tissues, respectively, 208 proteins were found to be up- or down-regulated (p < 0.05). Ontological category, interaction network and Western blotting suggested a close correlation between RCC-mediated proteins and oxidoreductases such as MNSOD. Markedly, oxidative modifications of MNSOD were identified at histidine (H54 and H55), tyrosine (Y58), tryptophan (W147, W149, W205 and W210) and asparagine (N206 and N209) residues additional to methionine. These oxidative insults were located at three hotspots near the hydrophobic pocket of the manganese binding site, of which the oxidation of Y58, W147 and W149 was up-regulated around three folds and the oxidation of H54 and H55 was detected in the cancer tissues only (p < 0.05). When normalized to MNSOD expression levels, relative MNSOD enzymatic activity was decreased in cancer tissues, suggesting impairment of MNSOD enzymatic activity in kidney cancer due to modifications. Thus, LC-MS/MS analysis revealed multiple oxidative modifications of MNSOD at different amino acid residues that might mediate the regulation of the superoxide radicals, mitochondrial ROS scavenging and MNSOD activity in kidney cancer