EfficientViT: Lightweight Multi-Scale Attention for On-Device Semantic Segmentation

Semantic segmentation enables many appealing real-world applications, such as computational photography, autonomous driving, etc. However, the vast computational cost makes deploying state-of-the-art semantic segmentation models on edge devices with limited hardware resources difficult. This work presents EfficientViT, a new family of semantic segmentation models with a novel lightweight multi-scale attention for on-device semantic segmentation. Unlike prior semantic segmentation models that rely on heavy self-attention, hardware-inefficient large-kernel convolution, or complicated topology structure to obtain good performances, our lightweight multi-scale attention achieves a global receptive field and multi-scale learning (two critical features for semantic segmentation models) with only lightweight and hardware-efficient operations. As such, EfficientViT delivers remarkable performance gains over previous state-of-the-art semantic segmentation models across popular benchmark datasets with significant speedup on the mobile platform. Without performance loss on Cityscapes, our EfficientViT provides up to 15x and 9.3x mobile latency reduction over SegFormer and SegNeXt, respectively. Maintaining the same mobile latency, EfficientViT provides +7.4 mIoU gain on ADE20K over SegNeXt. Code: https://github.com/mit-han-lab/efficientvit.Comment: Tech repor

OLLIE: Derivation-based Tensor Program Optimizer

Boosting the runtime performance of deep neural networks (DNNs) is critical due to their wide adoption in real-world tasks. Existing approaches to optimizing the tensor algebra expression of a DNN only consider expressions representable by a fixed set of predefined operators, missing possible optimization opportunities between general expressions. We propose OLLIE, the first derivation-based tensor program optimizer. OLLIE optimizes tensor programs by leveraging transformations between general tensor algebra expressions, enabling a significantly larger expression search space that includes those supported by prior work as special cases. OLLIE uses a hybrid derivation-based optimizer that effectively combines explorative and guided derivations to quickly discover highly optimized expressions. Evaluation on seven DNNs shows that OLLIE can outperform existing optimizers by up to 2.73$\times$ (1.46$\times$ on average) on an A100 GPU and up to 2.68$\times$ (1.51$\times$) on a V100 GPU, respectively

Inhibition of the Keap1/Nrf2 Signaling Pathway Significantly Promotes the Progression of Type 1 Diabetes Mellitus

Type 1 diabetes mellitus (T1DM) is an autoimmune disease characterized by insulin deficiency due to pancreatic β-cell damage and leads to hyperglycemia. The precise molecular mechanisms of the etiology of T1DM are not completely understood. Oxidative stress and the antioxidant status of pancreatic β-cells play a vital role in the pathogenesis and progression of T1DM. The Keap1/Nrf2 signaling pathway plays a critical role in cellular resistance to oxidative stress. This study is aimed at investigating the role of the Keap1/Nrf2 signaling pathway in the progression of T1DM. An alloxan- (ALX-) stimulated T1DM animal model in wild-type (WT) and Nrf2 knockout (Nrf2-/-) C57BL/6J mice and a mouse pancreatic β-cell line (MIN6) were established. Compared with the tolerant (ALX exposure, nondiabetic) WT mice, the sensitive (ALX exposure, diabetic) WT mice exhibited higher blood glucose levels and lower plasma insulin levels. The Keap1/Nrf2 signaling pathway was significantly inhibited in the sensitive WT mice, which was reflected by overexpression of Keap1 and low expression of Nrf2, accompanied by a marked decrease in the expression of the antioxidative enzymes. Compared with WT mice, the Nrf2-/- mice had an increased incidence of T1DM and exhibited more severe pancreatic β-cell damage. The results of in vitro experiments showed that ALX significantly inhibited the viability and proliferation and promoted the apoptosis of MIN6 cells. ALX also markedly increased intracellular ROS production and caused DNA damage in MIN6 cells. In addition, the Keap1/Nrf2 signaling pathway was significantly inhibited in the damaged MIN6 cells. Moreover, Nrf2 silencing by transfection with Nrf2 siRNA markedly exacerbated ALX-induced MIN6 cell injury. Conclusively, this study demonstrates that inhibition of the Keap1/Nrf2 signaling pathway could significantly promote the incidence of T1DM. This study indicates that activation of Keap1/Nrf2 signaling in pancreatic β-cells may be a useful pharmacological strategy for the clinical prevention and treatment of T1DM