Q-HyViT: Post-Training Quantization for Hybrid Vision Transformer with Bridge Block Reconstruction

Recently, vision transformers (ViTs) have superseded convolutional neural networks in numerous applications, including classification, detection, and segmentation. However, the high computational requirements of ViTs hinder their widespread implementation. To address this issue, researchers have proposed efficient hybrid transformer architectures that combine convolutional and transformer layers with optimized attention computation of linear complexity. Additionally, post-training quantization has been proposed as a means of mitigating computational demands. For mobile devices, achieving optimal acceleration for ViTs necessitates the strategic integration of quantization techniques and efficient hybrid transformer structures. However, no prior investigation has applied quantization to efficient hybrid transformers. In this paper, we discover that applying existing PTQ methods for ViTs to efficient hybrid transformers leads to a drastic accuracy drop, attributed to the four following challenges: (i) highly dynamic ranges, (ii) zero-point overflow, (iii) diverse normalization, and (iv) limited model parameters ($<$5M). To overcome these challenges, we propose a new post-training quantization method, which is the first to quantize efficient hybrid ViTs (MobileViTv1, MobileViTv2, Mobile-Former, EfficientFormerV1, EfficientFormerV2) with a significant margin (an average improvement of 8.32\% for 8-bit and 26.02\% for 6-bit) compared to existing PTQ methods (EasyQuant, FQ-ViT, and PTQ4ViT). We plan to release our code at \url{https://github.com/Q-HyViT}.Comment: 12 pages, 8 figure

The association between skin auto-fluorescence of palmoplantar sites and microvascular complications in Asian patients with type 2 diabetes mellitus

Abstract Skin auto-fluorescence (SAF) has generated broad interest about the prospects for non-invasive advanced glycation end product assessment and its direct interplay with the development of microvascular complications, but clinical application of the existing SAF measuring of non-palmoplantar sites in non-Caucasian subjects with dark skin type is still controversial. Here, we tested the diabetic complication screening performance of a novel SAF measuring system in Asian type 2 diabetes mellitus (T2DM) subjects. A total of 166 Korean patients with T2DM were enrolled in this study and palmoplantar SAF was measured by a newly developed transmission-geometry noninvasive optical system. We found that transmitted SAF values of palmoplantar sites, 1st dorsal interossei muscles of the hand, in a complication group were significantly higher than in a non-complication group while no differences were observed between the two groups in reflected SAF of non-palmoplantar sites. The transmitted SAF values of palmoplantar sites were dramatically increased in subjects with multiple complications and were tightly correlated with the duration of microvascular complications. In conclusion, the SAF measurement in the palmoplantar sites with a non-invasive transmission-geometry optical system provided better microvascular complication screening performance compared to the SAF measurement of non-palmoplantar sites specifically in Asian T2DM subjects

The Effect of DPP-4 Inhibitors on Metabolic Parameters in Patients with Type 2 Diabetes

BackgroundWe evaluated the effects of two dipeptidyl peptidase-4 (DPP-4) inhibitors, sitagliptin and vildagliptin, on metabolic parameters in patients with type 2 diabetes mellitus.MethodsA total of 170 type 2 diabetes patients treated with sitagliptin or vildagliptin for more than 24 weeks were selected. The patients were separated into two groups, sitagliptin (100 mg once daily, n=93) and vildagliptin (50 mg twice daily, n=77). We compared the effect of each DPP-4 inhibitor on metabolic parameters, including the fasting plasma glucose (FPG), postprandial glucose (PPG), glycated hemoglobin (HbA1c), and glycated albumin (GA) levels, and lipid parameters at baseline and after 24 weeks of treatment.ResultsThe HbA1c, FPG, and GA levels were similar between the two groups at baseline, but the sitagliptin group displayed a higher PPG level (P=0.03). After 24 weeks of treatment, all of the glucose-related parameters were significantly decreased in both groups (P=0.001). The levels of total cholesterol and triglycerides were only reduced in the vildagliptin group (P=0.001), although the sitagliptin group received a larger quantity of statins than the vildagliptin group (P=0.002).The mean change in the glucose- and lipid-related parameters after 24 weeks of treatment were not significantly different between the two groups (P=not significant). Neither sitagliptin nor vildagliptin treatment was associated with a reduction in the high sensitive C-reactive protein level (P=0.714).ConclusionVildagliptin and sitagliptin exert a similar effect on metabolic parameters, but vildagliptin exerts a more potent beneficial effect on lipid parameters

FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction

International audienceFederated Learning (FL) is very appealing for its privacy benefits: essentially, a global model is trained with updates computed on mobile devices while keeping the data of users local. Standard FL infrastructures are however designed to have no energy or performance impact on mobile devices, and are therefore not suitable for applications that require frequent (online) model updates, such as news recommenders. This paper presents FLeet, the first Online FL system, acting as a middleware between the Android OS and the machine learning application. FLeet combines the privacy of Standard FL with the precision of online learning thanks to two core components: (i) I-Prof, a new lightweight profiler that predicts and controls the impact of learning tasks on mobile devices, and (ii) AdaSGD, a new adaptive learning algorithm that is resilient to delayed updates. Our extensive evaluation shows that Online FL, as implemented by FLeet, can deliver a 2.3× quality boost compared to Standard FL, while only consuming 0.036% of the battery per day. I-Prof can accurately control the impact of learning tasks by improving the prediction accuracy up to 3.6× (computation time) and up to 19× (energy). AdaSGD outperforms alternative FL approaches by 18.4% in terms of convergence speed on heterogeneous data