DisWOT: Student Architecture Search for Distillation WithOut Training

Knowledge distillation (KD) is an effective training strategy to improve the lightweight student models under the guidance of cumbersome teachers. However, the large architecture difference across the teacher-student pairs limits the distillation gains. In contrast to previous adaptive distillation methods to reduce the teacher-student gap, we explore a novel training-free framework to search for the best student architectures for a given teacher. Our work first empirically show that the optimal model under vanilla training cannot be the winner in distillation. Secondly, we find that the similarity of feature semantics and sample relations between random-initialized teacher-student networks have good correlations with final distillation performances. Thus, we efficiently measure similarity matrixs conditioned on the semantic activation maps to select the optimal student via an evolutionary algorithm without any training. In this way, our student architecture search for Distillation WithOut Training (DisWOT) significantly improves the performance of the model in the distillation stage with at least 180$\times$ training acceleration. Additionally, we extend similarity metrics in DisWOT as new distillers and KD-based zero-proxies. Our experiments on CIFAR, ImageNet and NAS-Bench-201 demonstrate that our technique achieves state-of-the-art results on different search spaces. Our project and code are available at https://lilujunai.github.io/DisWOT-CVPR2023/.Comment: Accepted by CVPR202

EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization

Mixed-Precision Quantization~(MQ) can achieve a competitive accuracy-complexity trade-off for models. Conventional training-based search methods require time-consuming candidate training to search optimized per-layer bit-width configurations in MQ. Recently, some training-free approaches have presented various MQ proxies and significantly improve search efficiency. However, the correlation between these proxies and quantization accuracy is poorly understood. To address the gap, we first build the MQ-Bench-101, which involves different bit configurations and quantization results. Then, we observe that the existing training-free proxies perform weak correlations on the MQ-Bench-101. To efficiently seek superior proxies, we develop an automatic search of proxies framework for MQ via evolving algorithms. In particular, we devise an elaborate search space involving the existing proxies and perform an evolution search to discover the best correlated MQ proxy. We proposed a diversity-prompting selection strategy and compatibility screening protocol to avoid premature convergence and improve search efficiency. In this way, our Evolving proxies for Mixed-precision Quantization~(EMQ) framework allows the auto-generation of proxies without heavy tuning and expert knowledge. Extensive experiments on ImageNet with various ResNet and MobileNet families demonstrate that our EMQ obtains superior performance than state-of-the-art mixed-precision methods at a significantly reduced cost. The code will be released.Comment: Accepted by ICCV202

Research on prevention and control methods of land subsidence induced by groundwater overexploitation based on three-dimensional fluid solid coupling model—a case study of Guangrao County

Land subsidence is an environmental geological phenomenon with slowly decreasing ground elevation, The North China Plain is one of the areas with the most serious land subsidence in China, and Guangrao County is one of the subsidence centers. This paper is based on the hydrogeological and engineering geological data of Guangrao County, the groundwater monitoring data for many years and the land subsidence monitoring data, systematically analyzes the dynamic characteristics of groundwater, the distribution and evolution of land subsidence, and the correlation between groundwater exploitation and land subsidence development in different layers of this area. Based on Biot porous medium consolidation theory, establishes a three-dimensional fluid solid coupling numerical model of land subsidence in Guangrao County, restores the development process of land subsidence, predicts and analyzes the subsidence evolution law under different groundwater exploitation schemes, and proposes targeted prevention and control measures. The research results show that: the shallow groundwater forms a cone of depression with Guangbei Salt Field as the center, and the deep groundwater forms an elliptical regional cone of depression with the urban area as the center. The ground is gradually formed two small settlement areas with the urban area of Guangrao County and Guangbei Salt Field as the settlement center, and there is a trend of interrelated expansion. The three-dimensional fluid solid coupling model of land subsidence accurately restored the development process of land subsidence in the study area, predicted that under the current groundwater exploitation conditions, by 2040, the settlement of Guangrao urban settlement center will increase to 1,350 mm, forming a large regional funnel centered around the urban area, and gradually developing and expanding around. Prohibition of groundwater exploitation in the main funnel area is a more reasonable and effective exploitation plan to prevent the development of land subsidence

Near-net forming complex shaped Zr-based bulk metallic glasses by high pressure die casting

Forming complex geometries using the casting process is a big challenge for bulk metallic glasses (BMGs), because of a lack of time of the window for shaping under the required high cooling rate. In this work, we open an approach named the “entire process vacuum high pressure die casting” (EPV-HPDC), which delivers the ability to fill die with molten metal in milliseconds, and create solidification under high pressure. Based on this process, various Zr-based BMGs were prepared by using industrial grade raw material. The results indicate that the EPV-HPDC process is feasible to produce a glassy structure for most Zr-based BMGs, with a size of 3 mm × 10 mm and with a high strength. In addition, it has been found that EPV-HPDC process allows complex industrial BMG parts, some of which are hard to be formed by any other metal processes, to be net shaped precisely. The BMG components prepared by the EVP-HPDC process possess the advantages of dimensional accuracy, efficiency, and cost compared with the ones formed by other methods. The EVP-HPDC process paves the way for the large-scale application of BMGs

Survival and Clinicopathological Significance of SIRT1 Expression in Cancers: A Meta-Analysis

Background: Silent information regulator 2 homolog 1 (SIRT1) is an evolutionarily conserved enzymes with nicotinamide adenine dinucleotide (NAD)+-dependent deacetylase activity. SIRT1 is involved in a large variety of cellular processes, such as genomic stability, energy metabolism, senescence, gene transcription, and oxidative stress. SIRT1 has long been recognized as both a tumor promoter and tumor suppressor. Its prognostic role in cancers remains controversial.Methods: A meta-analysis of 13,138 subjects in 63 articles from PubMed, EMBASE, and Cochrane Library was performed to evaluate survival and clinicopathological significance of SIRT1 expression in various cancers.Results: The pooled results of meta-analysis showed that elevated expression of SIRT1 implies a poor overall survival (OS) of cancer patients [Hazard Ratio (HR) = 1.566, 95% CI: 1.293–1.895, P < 0.0001], disease free survival (DFS) (HR = 1.631, 95% CI: 1.250–2.130, P = 0.0003), event free survival (EFS) (HR = 2.534, 95% CI: 1.602–4.009, P = 0.0001), and progress-free survival (PFS) (HR = 3.325 95% CI: 2.762–4.003, P < 0.0001). Elevated SIRT1 level was associated with tumor stage [Relative Risk (RR) = 1.299, 95% CI: 1.114–1.514, P = 0.0008], lymph node metastasis (RR = 1.172, 95% CI: 1.010–1.360, P = 0.0363), and distant metastasis (RR = 1.562, 95% CI: 1.022–2.387, P = 0.0392). Meta-regression and subgroup analysis revealed that ethnic background has influence on the role of SIRT1 expression in predicting survival and clinicopathological characteristics of cancers. Overexpression of SIRT1 predicted a worse OS and higher TNM stage and lymphatic metastasis in Asian population especially in China.Conclusion: Our data suggested that elevated expression of SIRT1 predicted a poor OS, DFS, EFS, PFS, but not for recurrence-free survival (RFS) and cancer-specific survival (CCS). SIRT1 overexpression was associated with higher tumor stage, lymph node metastasis, and distant metastasis. SIRT1-mediated molecular events and biological processes could be an underlying mechanism for metastasis and SIRT1 is a therapeutic target for inhibiting metastasis, leading to good prognosis

NADPH Oxidase: a Target for the Modulation of the Excessive Oxidase Damage Induced by Overtraining in Rat Neutrophils

Objective: The purpose of this study is to demonstrate that NADPH oxidase mediating the ROS production is the major pathway for ROS generation in neutrophils during exercise. NADPH oxidase, as a target can modulate oxidative damage induced by overtraining, which can be value to the prevention of exercise-induced immunosuppression.Methods: Thirty male Wistar rats were randomly divided into three groups: a negative control group (C, n = 10), an overtraining group (E, n = 10) and an overtraining + DPI intervention group (D, n =10). Groups E and D were trained on a standard treadmill with progressive load for 11 weeks. After 36-40 h from the last training, eight rats were randomly selected from each group, and blood was sampled from the orbital vein. ELISAs were used to measure serum cytokine levels and lipid peroxidation in blood plasma. Flow cytometry with Annexin V / PI double staining was used to measure neutrophil apoptosis and necrosis. DNA damage in lymphocytes was tested using single cell gel electrophoresis (SCGE). The co-localization between gp91phox and p47phox of the NADPH-oxidase was detected using immunocytochemistry and confocal microscopy.Results: 1) Compared with group C, the concentrations of IL-1&#946;, IL-8, and TNF-&#945; were significantly increased and MCP-1, and CINC were significantly decreased in blood plasma from group E (P &#60; 0.01 and P &#60; 0.05, respectively). Concentrations of IL-1&#946; and MCP-1 were decreased (P &#60; 0.05), and IL-8 and TNF-&#945; were significantly increased (P &#60;0.05) in blood plasma from group D. MDA and MPO were elevated in plasma from groups E and D (P &#60; 0.01 and P &#60; 0.05, respectively). 2) Compared with group C, the percentage of neutrophils apoptosis were significantly elevated (P &#60; 0.01) in both groups E and D, and the percentage of cell death was raised in group E (P &#60; 0.05). No significant change was observed in group D. 3) Compared with group C, the number of comet cells, an indicator of DNA damage, was significantly increased (P &#60; 0.01), and the width and tail length of comet cells were notably increased in group E, while no significant increase was observed in group D. 4) The p47phox protein translocated to the cell membrane and co-localized with the gp91phox subunit of NADPH oxidase in neutrophils activated by overtraining.Conclusion: 1) Excessive exercise led to an increased secretion of inflammatory cytokines and chemokines in peripheral blood, and it may have induced tissue inflammation 2) Overtraining can activate the NADPH oxidase-mediated overproduction of ROS, leading to increased lipid peroxidation. 3) NADPHoxidase in neutrophils as a target, was responsible for ROS, oxidative damage to phagocytes and lymphocytes and changes to inflammatory cytokines and immune regulatory factors all affect cellular immune functions and may be causative factors for exercise-induced immunosuppression.</p

Evidence of disruption of Si-rich microstructure in engineering-lightweight Al–12.2at.%Si alloy melt above liquidus temperature

International audienceThe exploration of microstructures in high temperature alloy melts is important for manufacturing of metallic components but extremely challenging. Here, we report experimental evidence of the disruption of Si-rich microstructure in engineering-lightweight Al-12.2at.%Si alloy melt at 1100 °C, via melt-spinning (MS) of Al 1−x Si x (x = 0.03,0.07,0.122,0.2) alloy melts from different initial melt temperatures, 800 °C and 1100 °C, under the super-high cooling rate of ~ 10 6 °C/s, in cooperation with the small angle neutron scattering (SANS) measurement. Si particles in 1100 °C MS alloys are abnormally smaller and increased in number at Al-12.2at.%Si, compared with 800 °C MS alloys, which demonstrates the disruption of Si-rich microstructure in Al-12.2at.%Si alloy melt at 1100 °C. SANS experiment verifies that large quantities of small (0-10 nm) Si-rich microstructures and small quantities of large (10-240 nm) Si-rich microstructures exist in Al-12.2at.%Si alloy melt, and the large Si-rich microstructures disrupt into small Si-rich microstructures with increasing of melt temperature from 800 to 1100 °C. Microstructure analysis of the MS alloys indicates that the large Si-rich microstructures in Al-12.2at.%Si alloy melt are probably aggregates comprising multiple small Si-rich microstructures. This work also provides a pathway for the exploration of microstructures in other high temperature alloy melts. The structural materials especially metallic alloys are basic support of modern society 1,2. Most of the metallic alloy components are manufactured from the initial melt state via the metallurgy and casting route, and the microstructure in alloy melts will affect the subsequent solidification and mechanical properties of metallic components 3-7. However, the exploration of microstructure in alloy melts is always extraordinary challenging, due to the high temperature of alloy melts that usually ranges from several hundreds to ~ 3500 °C. The Al-Si based alloys are important lightweight engineering alloys in automotive, aerospace and other industries, which constitute ~ 90 % of all aluminium shape castings 8-14. As a special system, Al-Si alloy comprises two elements with a large melting point discrepancy at 754 °C (Al: 660 °C, Si: 1414 °C), which provides theoretical possibility for the existence of Si-rich microstructure in the molten state 15. In addition, the measurement of irreversible changes in physical properties such as density during heating and cooling cycles supported the microstructure evolution in Al-Si alloy melts with changing of melt temperature 16 , however, we can hardly get information of the detail microstructure evolution in the Al-Si alloy melts from the measurement of physical properties. ope

Physical Activity, a Critical Exposure Factor of Environmental Pollution in Children and Adolescents Health Risk Assessment

It is an extremely urgent problem that physical fitness promotion must face not only the increasing air pollution but also the decline of physical activity level of children and adolescents worldwide at present, which is the major reason that forms an inactive lifestyle and does harm to adolescents’ health. Thus, it is necessary to focus on the exposure factor in environmental health risk assessment (EHRA) which conducts supervision of environmental pollution and survey of adolescents’ activity patterns according to the harmful characteristics of air pollutant and relationship between dose and response. Some countries, such as USA, Canada and Australia, regard both respiratory rate and physical activity pattern as main exposure factors for adolescents in both air pollution health risk assessment and exercise risk assessment to forecast a safe exposing condition of pollutant for adolescents while they are doing exercise outdoors. In addition, it suggests that the testing indexes and testing methods of these two exposure factors, such as investigating the time of daily physical activity, strength, and characteristic of frequency, help to set up the quantitative relationship between environmental pollution index and the time, strength, frequency of daily activities, and formulate children’s and adolescents’ activity instructions under different levels of environmental pollutions. As smog becomes increasingly serious at present, it is meaningful to take physical activity as a critical composition of exposure factor and establish physical activity guideline, so as to reduce the risk of air pollution, and promote physical health of children and adolescents effectively