554 research outputs found
BayesNAS: A Bayesian Approach for Neural Architecture Search
One-Shot Neural Architecture Search (NAS) is a promising method to
significantly reduce search time without any separate training. It can be
treated as a Network Compression problem on the architecture parameters from an
over-parameterized network. However, there are two issues associated with most
one-shot NAS methods. First, dependencies between a node and its predecessors
and successors are often disregarded which result in improper treatment over
zero operations. Second, architecture parameters pruning based on their
magnitude is questionable. In this paper, we employ the classic Bayesian
learning approach to alleviate these two issues by modeling architecture
parameters using hierarchical automatic relevance determination (HARD) priors.
Unlike other NAS methods, we train the over-parameterized network for only one
epoch then update the architecture. Impressively, this enabled us to find the
architecture on CIFAR-10 within only 0.2 GPU days using a single GPU.
Competitive performance can be also achieved by transferring to ImageNet. As a
byproduct, our approach can be applied directly to compress convolutional
neural networks by enforcing structural sparsity which achieves extremely
sparse networks without accuracy deterioration.Comment: International Conference on Machine Learning 201
A note on the uniqueness of minimal maps into via singular values
In this note, we derive a uniqueness theorem for minimal graphs of general
codimension under certain restrictions closed related to the convexity (not
strict convexity) of the area functional with respect to singular values,
improving the result in \cite{L-O-T}. The crucial step of the proof is to show
the local linearity of the singular value vectors along the geodesic homotopy
of two given minimal maps.Comment: 8 page
Do humans and machines have the same eyes? Human-machine perceptual differences on image classification
Trained computer vision models are assumed to solve vision tasks by imitating
human behavior learned from training labels. Most efforts in recent vision
research focus on measuring the model task performance using standardized
benchmarks. Limited work has been done to understand the perceptual difference
between humans and machines. To fill this gap, our study first quantifies and
analyzes the statistical distributions of mistakes from the two sources. We
then explore human vs. machine expertise after ranking tasks by difficulty
levels. Even when humans and machines have similar overall accuracies, the
distribution of answers may vary. Leveraging the perceptual difference between
humans and machines, we empirically demonstrate a post-hoc human-machine
collaboration that outperforms humans or machines alone.Comment: Paper under revie
A Survey for High-redshift Gravitationally Lensed Quasars and Close Quasars Pairs. I. the Discoveries of an Intermediately-lensed Quasar and a Kpc-scale Quasar Pair at
We present the first results from a new survey for high-redshift
gravitationally lensed quasars and close quasar pairs. We carry
out candidate selection based on the colors and shapes of objects in public
imaging surveys, then conduct follow-up observations to confirm the nature of
high-priority candidates. In this paper, we report the discoveries of
J0025--0145 () which we identify as an {intermediately-lensed quasar,
and J2329--0522 () which is a kpc-scale close quasar pair. The {\em
Hubble Space Telescope (HST)} image of J0025--0145 shows a foreground lensing
galaxy located 0\farcs6 away from the quasar. However, J0025--0145 does not
exhibit multiple lensed images of the quasar, and we identify J0025--0145 as an
intermediate lensing system (a lensing system that is not multiply imaged but
has a significant magnification). The spectrum of J0025--0145 implies an
extreme Eddington ratio if the quasar luminosity is intrinsic, which could be
explained by a large lensing magnification. The {\em HST} image of J0025--0145
also indicates a tentative detection of the quasar host galaxy in rest-frame
UV, illustrating the power of lensing magnification and distortion in studies
of high-redshift quasar host galaxies. J2329--0522 consists of two resolved
components with significantly different spectral properties, and a lack of
lensing galaxy detection under sub-arcsecond seeing. We identify it as a close
quasar pair, which is the highest confirmed kpc-scale quasar pair to date. We
also report four lensed quasars and quasar pairs at , and discuss
possible improvements to our survey strategy.Comment: 23 pages, 10 figures, 6 tables. Accepted by the Astronomical Journa
Effects of Electrode Off Centre on Trapped Thickness-Shear Modes in Contoured AT-Cut Quartz Resonators
We investigated thickness-shear vibrations of a contoured, AT-cut quartz resonator with a pair of electrodes displaced from the resonator centre. The scalar differential equations by Stevens and Tiersten for thickness-shear vibrations of electroded and unelectroded quartz plates were employed. Based on the variational formulation of the scalar differential equations established in a previous paper and the variation-based Ritz method with trigonometric functions as basis functions, free vibration resonance frequencies and trapped thickness-shear modes were obtained. The effects of the electrode off centre on resonance frequencies and mode shapes were examined. When the electrode off centre is about one hundredth of the electrode length, the relative frequency shift is of the order of one part per million, significant in certain resonator design and applications. The electrode off centre also causes the loss of symmetry of modes, which has an adverse effect on resonator frequency stability under a normal acceleration
Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
We propose to address quadrupedal locomotion tasks using Reinforcement
Learning (RL) with a Transformer-based model that learns to combine
proprioceptive information and high-dimensional depth sensor inputs. While
learning-based locomotion has made great advances using RL, most methods still
rely on domain randomization for training blind agents that generalize to
challenging terrains. Our key insight is that proprioceptive states only offer
contact measurements for immediate reaction, whereas an agent equipped with
visual sensory observations can learn to proactively maneuver environments with
obstacles and uneven terrain by anticipating changes in the environment many
steps ahead. In this paper, we introduce LocoTransformer, an end-to-end RL
method for quadrupedal locomotion that leverages a Transformer-based model for
fusing proprioceptive states and visual observations. We evaluate our method in
challenging simulated environments with different obstacles and uneven terrain.
We show that our method obtains significant improvements over policies with
only proprioceptive state inputs, and that Transformer-based models further
improve generalization across environments. Our project page with videos is at
https://RchalYang.github.io/LocoTransformer .Comment: Our project page with videos is at
https://RchalYang.github.io/LocoTransforme
When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks
Recent advances in adversarial attacks uncover the intrinsic vulnerability of
modern deep neural networks. Since then, extensive efforts have been devoted to
enhancing the robustness of deep networks via specialized learning algorithms
and loss functions. In this work, we take an architectural perspective and
investigate the patterns of network architectures that are resilient to
adversarial attacks. To obtain the large number of networks needed for this
study, we adopt one-shot neural architecture search, training a large network
for once and then finetuning the sub-networks sampled therefrom. The sampled
architectures together with the accuracies they achieve provide a rich basis
for our study. Our "robust architecture Odyssey" reveals several valuable
observations: 1) densely connected patterns result in improved robustness; 2)
under computational budget, adding convolution operations to direct connection
edge is effective; 3) flow of solution procedure (FSP) matrix is a good
indicator of network robustness. Based on these observations, we discover a
family of robust architectures (RobNets). On various datasets, including CIFAR,
SVHN, Tiny-ImageNet, and ImageNet, RobNets exhibit superior robustness
performance to other widely used architectures. Notably, RobNets substantially
improve the robust accuracy (~5% absolute gains) under both white-box and
black-box attacks, even with fewer parameter numbers. Code is available at
https://github.com/gmh14/RobNets.Comment: CVPR 2020. First two authors contributed equall
- …