Post-training quantization attracts increasing attention due to its
convenience in deploying quantized neural networks. Although
rounding-to-nearest remains the prevailing method for DNN quantization, prior
research has demonstrated its suboptimal nature when applied to weight
quantization. They propose optimizing weight rounding schemes by leveraging
output error rather than the traditional weight quantization error. Our study
reveals that similar rounding challenges also extend to activation
quantization. Despite the easy generalization, the challenges lie in the
dynamic nature of activation. Adaptive rounding is expected for varying
activations and the method is subjected to runtime overhead. To tackle this, we
propose the AQuant quantization framework with a novel perspective to reduce
output error by adjusting rounding schemes of activations. Instead of using the
constant rounding border 0.5 of the rounding-to-nearest operation, we make the
border become a function w.r.t. the activation value to change the activation
rounding by the adaptive border. To deal with the runtime overhead, we use a
coarse-grained version of the border function. Finally, we introduce our
framework to optimize the border function. Extensive experiments show that
AQuant achieves notable improvements compared to state-of-the-art works and
pushes the accuracy of ResNet-18 up to 60.31% under the 2-bit weight and
activation quantization