NeRN -- Learning Neural Representations for Neural Networks

Neural Representations have recently been shown to effectively reconstruct a wide range of signals from 3D meshes and shapes to images and videos. We show that, when adapted correctly, neural representations can be used to directly represent the weights of a pre-trained convolutional neural network, resulting in a Neural Representation for Neural Networks (NeRN). Inspired by coordinate inputs of previous neural representation methods, we assign a coordinate to each convolutional kernel in our network based on its position in the architecture, and optimize a predictor network to map coordinates to their corresponding weights. Similarly to the spatial smoothness of visual scenes, we show that incorporating a smoothness constraint over the original network's weights aids NeRN towards a better reconstruction. In addition, since slight perturbations in pre-trained model weights can result in a considerable accuracy loss, we employ techniques from the field of knowledge distillation to stabilize the learning process. We demonstrate the effectiveness of NeRN in reconstructing widely used architectures on CIFAR-10, CIFAR-100, and ImageNet. Finally, we present two applications using NeRN, demonstrating the capabilities of the learned representations

The cosmic ray distribution in Sagittarius B

Copyright © 2007. The American Astronomical Society. All rights reserved. Printed in U.S.A. Submitted to Cornell University’s online archive www.arXiv.org in 2007 by Roland M. Crocker. Post-print sourced from www.arxiv.orgThe H.E.S.S. instrument has observed a diffuse flux of ∼TeV γ-rays from a large solid angle around the Galactic center (GC). This emission is correlated with the distribution of gas in the region, suggesting that the γ-rays originate in collisions between cosmic-ray hadrons (CRHs) and ambient matter. Of particular interest, H.E.S.S. has detected γ-rays from the Sagittarius (Sgr) B molecular cloud complex. Prompted by the suggestion of a hadronic origin for the γ-rays, we have examined archival 330 and 74 MHz Very Large Array radio data and 843 MHz Sydney University Molonglo Sky Survey data covering Sgr B, looking for synchrotron emission from secondary electrons and positrons (expected to be created in the same interactions that supply the observed γ-rays). Intriguingly, we have uncovered nonthermal emission, but at a level exceeding expectation. Adding to the overall picture, recent observations by the Atacama Pathfinder Experiment telescope show that the cosmic-ray ionization rate is 10 times greater in the Sgr B2 region of Sgr B than the local value. Lastly, Sgr B2 is also a very bright X-ray source. We examine scenarios for the spectra of CRHs and/or primary electrons that would reconcile all these different data. We determine that (1) a hard (∼E -2.2), high-energy (≳TeV) population of CRHs is unavoidably required by the H.E.S.S. γ-ray data, and (2) the remaining broadband, nonthermal phenomenology is explained either by a rather steep (∼E -2.9) spectrum of primary electrons or a (∼E-2.7) population of CRHs. Perhaps unsurprisingly, no single power-law population of either leptons or hadrons can explain the totality of broadband, nonthermal Sgr B phenomenology. © 2007. The American Astronomical Society. All rights reserved.Roland M. Crocker, David Jones, Raymond J. Protheroe, Jürgen Ott, Ron Ekers, Fulvio Melia, Todor Stanev, and Anne Gree