RUSH: Robust Contrastive Learning via Randomized Smoothing

Recently, adversarial training has been incorporated in self-supervised contrastive pre-training to augment label efficiency with exciting adversarial robustness. However, the robustness came at a cost of expensive adversarial training. In this paper, we show a surprising fact that contrastive pre-training has an interesting yet implicit connection with robustness, and such natural robustness in the pre trained representation enables us to design a powerful robust algorithm against adversarial attacks, RUSH, that combines the standard contrastive pre-training and randomized smoothing. It boosts both standard accuracy and robust accuracy, and significantly reduces training costs as compared with adversarial training. We use extensive empirical studies to show that the proposed RUSH outperforms robust classifiers from adversarial training, by a significant margin on common benchmarks (CIFAR-10, CIFAR-100, and STL-10) under first-order attacks. In particular, under $\ell_{\infty}$-norm perturbations of size 8/255 PGD attack on CIFAR-10, our model using ResNet-18 as backbone reached 77.8% robust accuracy and 87.9% standard accuracy. Our work has an improvement of over 15% in robust accuracy and a slight improvement in standard accuracy, compared to the state-of-the-arts.Comment: 12 pages, 2 figure

Study on the Indicators Evaluating Innovation Abilities of High-end Equipment Manufacturing Industry in Sichuan Province

Innovation is the only way to enhance the competitiveness of high-end equipment manufacturing industry in Sichuan province. However, due to the lack of relevant evaluation standards, many high-end equipment manufacturing enterprises cannot evaluate their own innovation abilities effectively. Using the expert-interview method, this paper constructs an evaluation index system composed of three primary indicators, six secondary indicators and nine tertiary indicators. This paper determines the weights for each indicator through order-relationship analysis, based on which suggestions are put forward to improve the innovation ability for such enterprises

FCG-ASpredictor: An Approach for the Prediction of Average Speed of Road Segments with Floating Car GPS Data

The average speed (AS) of a road segment is an important factor for predicting traffic congestion, because the accuracy of AS can directly affect the implementation of traffic management. The traffic environment, spatiotemporal information, and the dynamic interaction between these two factors impact the predictive accuracy of AS in the existing literature, and floating car data comprehensively reflect the operation of urban road vehicles. In this paper, we proposed a novel road segment AS predictive model, which is based on floating car data. First, the impact of historical AS, weather, and date attributes on AS prediction has been analyzed. Then, through spatiotemporal correlations calculation based on the data from Global Positioning System (GPS), the predictive method utilizes the recursive least squares method to fuse the historical AS with other factors (such as weather, date attributes, etc.) and adopts an extended Kalman filter algorithm to accurately predict the AS of the target segment. Finally, we applied our approach on the traffic congestion prediction on four road segments in Chengdu, China. The results showed that the proposed predictive model is highly feasible and accurate. Document type: Articl

What is valued most by patients with type 2 diabetes mellitus when selecting second-line antihyperglycemic medications in China

Objective: To estimate patient preferences for second-line antihyperglycemic medications in China. Methods: A face to face survey with the best-worst scaling (BWS) choices was administered in patients with diagnosed type 2 diabetes mellitus (T2DM). Study participants were asked to indicate which attribute they valued most and which attribute they valued least in 11 choice sets, each of which consisted of five alternatives out of 11 antihyperglycemic medication-specific attributes (treatment efficacy, weight change, hypoglycemic events, gastrointestinal side effects, cardiovascular health, urinary tract infection and genital infection side effects, edema, mode of administration, bone fracture, dosing frequency and out-of-pocket cost). A counting approach, a conditional logit model, and K-means clustering were used to estimate the relative importance of items and preference heterogeneity. Results: A total of 362 participants were included with a mean age of 63.6 (standard deviation: 11.8) years. There were 56.4% of participants were women, and 56.3% being diagnosed with diabetes for at least 5 years. Efficacy, cardiovascular health and hypoglycemic events were valued most, while dosing frequency, mode of administration and bone fracture were valued least. The K-means clustering further showed preference heterogeneity in out-of-pocket cost across the participants. Conclusion: Our study suggests that treatment efficacy, cardiovascular health and hypoglycemic events are valued most by Chinese patients with T2DM when selecting second-line antihyperglycemic medications. The study improves the understanding of patients’ preferences for second-line antihyperglycemic medications in China

NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers

The complicated architecture and high training cost of vision transformers urge the exploration of post-training quantization. However, the heavy-tailed distribution of vision transformer activations hinders the effectiveness of previous post-training quantization methods, even with advanced quantizer designs. Instead of tuning the quantizer to better fit the complicated activation distribution, this paper proposes NoisyQuant, a quantizer-agnostic enhancement for the post-training activation quantization performance of vision transformers. We make a surprising theoretical discovery that for a given quantizer, adding a fixed Uniform noisy bias to the values being quantized can significantly reduce the quantization error under provable conditions. Building on the theoretical insight, NoisyQuant achieves the first success on actively altering the heavy-tailed activation distribution with additive noisy bias to fit a given quantizer. Extensive experiments show NoisyQuant largely improves the post-training quantization performance of vision transformer with minimal computation overhead. For instance, on linear uniform 6-bit activation quantization, NoisyQuant improves SOTA top-1 accuracy on ImageNet by up to 1.7%, 1.1% and 0.5% for ViT, DeiT, and Swin Transformer respectively, achieving on-par or even higher performance than previous nonlinear, mixed-precision quantization.Comment: Accepted to CVPR202

Application of Base Force Element Method to Mesomechanics Analysis for Recycled Aggregate Concrete

The base force element method (BFEM) on potential energy principle is used to analyze recycled aggregate concrete (RAC) on mesolevel. The model of BFEM with triangular element is derived. The recycled aggregate concrete is taken as five-phase composites consisting of natural coarse aggregate, new mortar, new interfacial transition zone (ITZ), old mortar, and old ITZ on meso-level. The random aggregate model is used to simulate the mesostructure of recycled aggregate concrete. The mechanics properties of uniaxial compression and tension tests for RAC are simulated using the BFEM, respectively. The simulation results agree with the test results. This research method is a new way for investigating fracture mechanism and numerical simulation of mechanics properties for recycled aggregate concrete