1 research outputs found
Reliability Analysis of Vision Transformers
Vision Transformers (ViTs) that leverage self-attention mechanism have shown
superior performance on many classical vision tasks compared to convolutional
neural networks (CNNs) and gain increasing popularity recently. Existing ViTs
works mainly optimize performance and accuracy, but ViTs reliability issues
induced by soft errors in large-scale VLSI designs have generally been
overlooked. In this work, we mainly study the reliability of ViTs and
investigate the vulnerability from different architecture granularities ranging
from models, layers, modules, and patches for the first time. The investigation
reveals that ViTs with the self-attention mechanism are generally more
resilient on linear computing including general matrix-matrix multiplication
(GEMM) and full connection (FC) and show a relatively even vulnerability
distribution across the patches. ViTs involve more fragile non-linear computing
such as softmax and GELU compared to typical CNNs. With the above observations,
we propose a lightweight block-wise algorithm-based fault tolerance (LB-ABFT)
approach to protect the linear computing implemented with distinct sizes of
GEMM and apply a range-based protection scheme to mitigate soft errors in
non-linear computing. According to our experiments, the proposed fault-tolerant
approaches enhance ViTs accuracy significantly with minor computing overhead in
presence of various soft errors