167 research outputs found

    RUSH: Robust Contrastive Learning via Randomized Smoothing

    Full text link
    Recently, adversarial training has been incorporated in self-supervised contrastive pre-training to augment label efficiency with exciting adversarial robustness. However, the robustness came at a cost of expensive adversarial training. In this paper, we show a surprising fact that contrastive pre-training has an interesting yet implicit connection with robustness, and such natural robustness in the pre trained representation enables us to design a powerful robust algorithm against adversarial attacks, RUSH, that combines the standard contrastive pre-training and randomized smoothing. It boosts both standard accuracy and robust accuracy, and significantly reduces training costs as compared with adversarial training. We use extensive empirical studies to show that the proposed RUSH outperforms robust classifiers from adversarial training, by a significant margin on common benchmarks (CIFAR-10, CIFAR-100, and STL-10) under first-order attacks. In particular, under ℓ∞\ell_{\infty}-norm perturbations of size 8/255 PGD attack on CIFAR-10, our model using ResNet-18 as backbone reached 77.8% robust accuracy and 87.9% standard accuracy. Our work has an improvement of over 15% in robust accuracy and a slight improvement in standard accuracy, compared to the state-of-the-arts.Comment: 12 pages, 2 figure

    Study on the Indicators Evaluating Innovation Abilities of High-end Equipment Manufacturing Industry in Sichuan Province

    Get PDF
    Innovation is the only way to enhance the competitiveness of high-end equipment manufacturing industry in Sichuan province. However, due to the lack of relevant evaluation standards, many high-end equipment manufacturing enterprises cannot evaluate their own innovation abilities effectively. Using the expert-interview method, this paper constructs an evaluation index system composed of three primary indicators, six secondary indicators and nine tertiary indicators. This paper determines the weights for each indicator through order-relationship analysis, based on which suggestions are put forward to improve the innovation ability for such enterprises

    FCG-ASpredictor: An Approach for the Prediction of Average Speed of Road Segments with Floating Car GPS Data

    Get PDF
    The average speed (AS) of a road segment is an important factor for predicting traffic congestion, because the accuracy of AS can directly affect the implementation of traffic management. The traffic environment, spatiotemporal information, and the dynamic interaction between these two factors impact the predictive accuracy of AS in the existing literature, and floating car data comprehensively reflect the operation of urban road vehicles. In this paper, we proposed a novel road segment AS predictive model, which is based on floating car data. First, the impact of historical AS, weather, and date attributes on AS prediction has been analyzed. Then, through spatiotemporal correlations calculation based on the data from Global Positioning System (GPS), the predictive method utilizes the recursive least squares method to fuse the historical AS with other factors (such as weather, date attributes, etc.) and adopts an extended Kalman filter algorithm to accurately predict the AS of the target segment. Finally, we applied our approach on the traffic congestion prediction on four road segments in Chengdu, China. The results showed that the proposed predictive model is highly feasible and accurate. Document type: Articl

    What is valued most by patients with type 2 diabetes mellitus when selecting second-line antihyperglycemic medications in China

    Get PDF
    Objective: To estimate patient preferences for second-line antihyperglycemic medications in China. Methods: A face to face survey with the best-worst scaling (BWS) choices was administered in patients with diagnosed type 2 diabetes mellitus (T2DM). Study participants were asked to indicate which attribute they valued most and which attribute they valued least in 11 choice sets, each of which consisted of five alternatives out of 11 antihyperglycemic medication-specific attributes (treatment efficacy, weight change, hypoglycemic events, gastrointestinal side effects, cardiovascular health, urinary tract infection and genital infection side effects, edema, mode of administration, bone fracture, dosing frequency and out-of-pocket cost). A counting approach, a conditional logit model, and K-means clustering were used to estimate the relative importance of items and preference heterogeneity. Results: A total of 362 participants were included with a mean age of 63.6 (standard deviation: 11.8) years. There were 56.4% of participants were women, and 56.3% being diagnosed with diabetes for at least 5 years. Efficacy, cardiovascular health and hypoglycemic events were valued most, while dosing frequency, mode of administration and bone fracture were valued least. The K-means clustering further showed preference heterogeneity in out-of-pocket cost across the participants. Conclusion: Our study suggests that treatment efficacy, cardiovascular health and hypoglycemic events are valued most by Chinese patients with T2DM when selecting second-line antihyperglycemic medications. The study improves the understanding of patients’ preferences for second-line antihyperglycemic medications in China

    NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers

    Full text link
    The complicated architecture and high training cost of vision transformers urge the exploration of post-training quantization. However, the heavy-tailed distribution of vision transformer activations hinders the effectiveness of previous post-training quantization methods, even with advanced quantizer designs. Instead of tuning the quantizer to better fit the complicated activation distribution, this paper proposes NoisyQuant, a quantizer-agnostic enhancement for the post-training activation quantization performance of vision transformers. We make a surprising theoretical discovery that for a given quantizer, adding a fixed Uniform noisy bias to the values being quantized can significantly reduce the quantization error under provable conditions. Building on the theoretical insight, NoisyQuant achieves the first success on actively altering the heavy-tailed activation distribution with additive noisy bias to fit a given quantizer. Extensive experiments show NoisyQuant largely improves the post-training quantization performance of vision transformer with minimal computation overhead. For instance, on linear uniform 6-bit activation quantization, NoisyQuant improves SOTA top-1 accuracy on ImageNet by up to 1.7%, 1.1% and 0.5% for ViT, DeiT, and Swin Transformer respectively, achieving on-par or even higher performance than previous nonlinear, mixed-precision quantization.Comment: Accepted to CVPR202

    Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficient Finetuning

    Full text link
    Large Language Models (LLMs) have demonstrated significant potential in performing multiple tasks in multimedia applications, ranging from content generation to interactive entertainment, and artistic creation. However, the diversity of downstream tasks in multitask scenarios presents substantial adaptation challenges for LLMs. While traditional methods often succumb to knowledge confusion on their monolithic dense models, Mixture-of-Experts (MoE) has been emerged as a promising solution with its sparse architecture for effective task decoupling. Inspired by the principles of human cognitive neuroscience, we design a novel framework \texttt{Intuition-MoR1E} that leverages the inherent semantic clustering of instances to mimic the human brain to deal with multitask, offering implicit guidance to router for optimized feature allocation. Moreover, we introduce cutting-edge Rank-1 Experts formulation designed to manage a spectrum of intuitions, demonstrating enhanced parameter efficiency and effectiveness in multitask LLM finetuning. Extensive experiments demonstrate that Intuition-MoR1E achieves superior efficiency and 2.15\% overall accuracy improvement across 14 public datasets against other state-of-the-art baselines.Comment: 13 pages, 5 figure
    • …
    corecore