DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices

Recent years have witnessed the great success of vision transformer (ViT), which has achieved state-of-the-art performance on multiple computer vision benchmarks. However, ViT models suffer from vast amounts of parameters and high computation cost, leading to difficult deployment on resource-constrained edge devices. Existing solutions mostly compress ViT models to a compact model but still cannot achieve real-time inference. To tackle this issue, we propose to explore the divisibility of transformer structure, and decompose the large ViT into multiple small models for collaborative inference at edge devices. Our objective is to achieve fast and energy-efficient collaborative inference while maintaining comparable accuracy compared with large ViTs. To this end, we first propose a collaborative inference framework termed DeViT to facilitate edge deployment by decomposing large ViTs. Subsequently, we design a decomposition-and-ensemble algorithm based on knowledge distillation, termed DEKD, to fuse multiple small decomposed models while dramatically reducing communication overheads, and handle heterogeneous models by developing a feature matching module to promote the imitations of decomposed models from the large ViT. Extensive experiments for three representative ViT backbones on four widely-used datasets demonstrate our method achieves efficient collaborative inference for ViTs and outperforms existing lightweight ViTs, striking a good trade-off between efficiency and accuracy. For example, our DeViTs improves end-to-end latency by 2.89$\times$ with only 1.65% accuracy sacrifice using CIFAR-100 compared to the large ViT, ViT-L/16, on the GPU server. DeDeiTs surpasses the recent efficient ViT, MobileViT-S, by 3.54% in accuracy on ImageNet-1K, while running 1.72$\times$ faster and requiring 55.28% lower energy consumption on the edge device.Comment: Accepted by IEEE Transactions on Mobile Computin

Quantitative Assessment of the Effects of Oxidants on Antigen-Antibody Binding In Vitro

Objective. We quantitatively assessed the influence of oxidants on antigen-antibody-binding activity. Methods. We used several immunological detection methods, including precipitation reactions, agglutination reactions, and enzyme immunoassays, to determine antibody activity. The oxidation-reduction potential was measured in order to determine total serum antioxidant capacity. Results. Certain concentrations of oxidants resulted in significant inhibition of antibody activity but had little influence on total serum antioxidant capacity. Conclusions. Oxidants had a significant influence on interactions between antigen and antibody, but minimal effect on the peptide of the antibody molecule

Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering

This publication is the Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering from July 6-8, 2022. The EG-ICE International Workshop on Intelligent Computing in Engineering brings together international experts working on the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolution of challenges such as supporting multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways. &nbsp

Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering

This publication is the Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering from July 6-8, 2022. The EG-ICE International Workshop on Intelligent Computing in Engineering brings together international experts working on the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolution of challenges such as supporting multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways. &nbsp

The Efficiency of Energy Infrastructure Investment and Its Regional Economic Impact

This study constructed an input–output and environmental indicator combination framework to evaluate the efficiency of energy infrastructure investment. Furthermore, the study used a three-stage DEA model to evaluate the efficiency of energy infrastructure investment projects in Jiangsu Province. Subsequently, the study constructed a system of indicators to measure regional economic development and assigns weights to them using the entropy value method, to obtain a comprehensive regional economic development score. Finally, this study analyzed the impact of energy investment efficiency on regional economic growth, economic stability, and industrial structure optimization. The study results show that the efficiency of energy infrastructure investment varies widely across Jiangsu and is highly correlated with the regional economic development model, the level of economic development, and the importance of the industry. The study also reveals that the improvement of energy infrastructure investment efficiency in Jiangsu fails to reflect the level of regional economic development; however, it has a crucial role in increasing social wealth and transforming the regional industrial structure and economy. Based on these results, this study further proposes countermeasures, such as planning a reasonable scale of investment, implementing differentiated regional investment, and upgrading management and technology

A novel optimization method for belief rule base expert system with activation rate

Abstract Although the belief rule base (BRB) expert system has many advantages, such as the effective use of semi-quantitative information, objective description of uncertainty, and efficient nonlinear modeling capability, it is always limited by the problem of combinatorial explosion. The main reason is that the optimization of a BRB with many rules will consume many computing resources, which makes it unable to meet the real-time requirements in some complex systems. Another reason is that the optimization process will destroy the interpretability of those parameters that belong to the inadequately activated rules given by experts. To solve these problems, a novel optimization method for BRB is proposed in this paper. Through the activation rate, the rules that have never been activated or inadequately activated are pruned during the optimization process. Furthermore, even if there is a complete data set and all rules are activated, the activation rate can also be used in the parallel optimization process of the BRB expert system, where the training data set is divided into some subprocesses. The proposed method effectively solves the combinatorial explosion problem of BRB and can make full use of quantitative data without destroying the original interpretability provided by experts. Case studies prove the advantages and effectiveness of the proposed method, which greatly expands the application fields of the BRB expert system