2,306 research outputs found
On the Adversarial Robustness of Vision Transformers
Following the success in advancing natural language processing and
understanding, transformers are expected to bring revolutionary changes to
computer vision. This work provides the first and comprehensive study on the
robustness of vision transformers (ViTs) against adversarial perturbations.
Tested on various white-box and transfer attack settings, we find that ViTs
possess better adversarial robustness when compared with convolutional neural
networks (CNNs). This observation also holds for certified robustness. We
summarize the following main observations contributing to the improved
robustness of ViTs:
1) Features learned by ViTs contain less low-level information and are more
generalizable, which contributes to superior robustness against adversarial
perturbations.
2) Introducing convolutional or tokens-to-token blocks for learning low-level
features in ViTs can improve classification accuracy but at the cost of
adversarial robustness.
3) Increasing the proportion of transformers in the model structure (when the
model consists of both transformer and CNN blocks) leads to better robustness.
But for a pure transformer model, simply increasing the size or adding layers
cannot guarantee a similar effect.
4) Pre-training on larger datasets does not significantly improve adversarial
robustness though it is critical for training ViTs.
5) Adversarial training is also applicable to ViT for training robust models.
Furthermore, feature visualization and frequency analysis are conducted for
explanation. The results show that ViTs are less sensitive to high-frequency
perturbations than CNNs and there is a high correlation between how well the
model learns low-level features and its robustness against different
frequency-based perturbations
A nanogapped hysteresis-free field-effect transistor
We propose a semi-suspended device structure and construct nanogapped,
hysteresis-free field-effect transistors (FETs), based on the van der Waals
stacking technique. The structure, which features a semi-suspended channel
above a submicron-long wedge-like nanogap, is fulfilled by transferring
ultraclean BN-supported MoS channels directly onto dielectric-spaced
vertical source/drain stacks. Electronic characterization and analyses reveal a
high overall device quality, including ultraclean channel interfaces,
negligible electrical scanning hysteresis, and Ohmic contacts in the
structures. The unique hollow FET structure holds the potential for exploiting
reliable electronics, as well as nanofluid and pressure sensors.Comment: 22 pages, 4 figures, with S
Dilute bismides for near and mid-infrared applications
Dilute bismides are a group of emerging materials with unique properties. Incorporation of a small amount of Bi in common III-V host materials results in large band-gap reduction and strong spin-orbit splitting, leading to potential applications in near-infrared (NIR) and mid-infrared (MIR) optoelectronics. Recent progresses on molecular beam epitaxy (MBE) of novel III-Sb-Bi, i.e. GaSbBi and InSbBi thin films from our group are summarised in this paper. Quantum well structures based on GaSbBi and InGaAsBi aiming for the optical communication window were grown and characterized. © 2013 IEEE
- …