Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retraining

A challenge of channel pruning is designing efficient and effective criteria to select channels to prune. A widely used criterion is minimal performance degeneration. To accurately evaluate the truth performance degeneration requires retraining the survived weights to convergence, which is prohibitively slow. Hence existing pruning methods use previous weights (without retraining) to evaluate the performance degeneration. However, we observe the loss changes differ significantly with and without retraining. It motivates us to develop a technique to evaluate true loss changes without retraining, with which channels to prune can be selected more reliably and confidently. We first derive a closed-form estimator of the true loss change per pruning mask change, using influence functions without retraining. Influence function which is from robust statistics reveals the impacts of a training sample on the model's prediction and is repurposed by us to assess impacts on true loss changes. We then show how to assess the importance of all channels simultaneously and develop a novel global channel pruning algorithm accordingly. We conduct extensive experiments to verify the effectiveness of the proposed algorithm. To the best of our knowledge, we are the first that shows evaluating true loss changes for pruning without retraining is possible. This finding will open up opportunities for a series of new paradigms to emerge that differ from existing pruning methods. The code is available at https://github.com/hrcheng1066/IFSO.Comment: chrome-extension://ogjibjphoadhljaoicdnjnmgokohngcc/assets/icon-50207e67.pn

A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations

Modern deep neural networks, particularly recent large language models, come with massive model sizes that require significant computational and storage resources. To enable the deployment of modern models on resource-constrained environments and accelerate inference time, researchers have increasingly explored pruning techniques as a popular research direction in neural network compression. However, there is a dearth of up-to-date comprehensive review papers on pruning. To address this issue, in this survey, we provide a comprehensive review of existing research works on deep neural network pruning in a taxonomy of 1) universal/specific speedup, 2) when to prune, 3) how to prune, and 4) fusion of pruning and other compression techniques. We then provide a thorough comparative analysis of seven pairs of contrast settings for pruning (e.g., unstructured/structured) and explore emerging topics, including post-training pruning, different levels of supervision for pruning, and broader applications (e.g., adversarial robustness) to shed light on the commonalities and differences of existing methods and lay the foundation for further method development. To facilitate future research, we build a curated collection of datasets, networks, and evaluations on different applications. Finally, we provide some valuable recommendations on selecting pruning methods and prospect promising research directions. We build a repository at https://github.com/hrcheng1066/awesome-pruning

Effects of Rain Events on Carbon Fluxes from Biological Soil Crusts

In dry ecosystems, biological soil crusts (BSCs) have been suggested as one of the factors responsible for the large rate of annual CO2 net uptake (Xie et al. 2009). However, most studies carried out on carbon (C) fluxes in arid and semi-arid ecosystems, such as soil respiration, have neglected the carbon fluxes from BSCs. Although BSCs are a vital component of the dry-land soil C cycle, few studies have parameterized the conditions required for photosynthesis in BSCs or determined BSCs respiration (Elbert et al. 2009, Castillo-Monroy et al. 2011). Precipitation in dry land is dominated by small events (Lauenroth and Bradford 2009). Even the smallest events will influence the carbon fluxes of BSCs, while intermediate pulses might wet the subsurface biotic community, and typically only larger events are used by plants for carbon gain or growth of roots or shoots (Belnap et al. 2005). As BSCs dry quickly and are hence very responsive to moisture pulses, the pulsed nature of precipitation can lead to highly variable carbon fluxes from BSCs (Bowling et al. 2011). Therefore, it is very important to study the effect of rain events upon carbon fluxes through BSCs in the dry ecosystem

Alkyl substituted cucurbit[6]uril assisted competitive fluorescence recognition of lysine and methionine in aqueous solution

The use of competitive ratiometric fluorescence indicator displacement chemosensors derived from two alkyl substituted cucurbit[6]uril-based host-guest complexes is reported. In particular, the differing binding abilities of two cucurbit[6]uril derivatives towards the target analytes led to a useful ratiometric detection signal output for the discrimination of lysine and methionine versus the other tested α-amino acids in aqueous solution

Thiamine Alleviates High-Concentrate-Diet-Induced Oxidative Stress, Apoptosis, and Protects the Rumen Epithelial Barrier Function in Goats

High-concentrate diets are continually used in ruminants to meet the needs of milk yield, which can lead to the occurrence of subacute rumen acidosis in ruminants. This study investigated the protective effects of dietary thiamine supplementation on the damage of the ruminal epithelium barrier function in goats fed a high-concentrate diet. Twenty-four healthy Boer goats (live weight of 35.62 ± 2.4 kg; age, 1 year) were randomly assigned into three treatments, with eight goats in each treatment, consuming one of three diets: a low-concentrate diet (CON; concentrate/forage, 30:70), a high-concentrate diet (HC; concentrate/forage, 70:30), or a high-concentrate diet with 200 mg of thiamine/kg of dry matter intake (HCT; concentrate/forage, 70:30) for 12 weeks. The additional dose of thiamine was based on our previous study wherein thiamine ameliorates inflammation. Compared with HC treatment, the HCT treatment had markedly higher concentrations of glutathione, superoxide dismutase, and glutathione peroxidase and total antioxidant capacity (P < 0.05) in plasma and rumen epithelium. The results showed that the apoptosis index was lower (P < 0.05) in the HCT treatment than in that of the HC treatment. Compared with the HC treatment, permeability and the electrophysiology parameter short circuit current for ruminal epithelial tissue were significantly decreased (P < 0.05) in the HCT treatment. The immunohistochemical results showed that the expression distribution of tight junctions including claudin-1, claudin-4, occludin, and zonula occludin-1 (ZO-1) was greater (P < 0.05) in the HCT treatments than in the HC treatment. The mRNA expression in the rumen epithelium of ZO-1, occludin, claudin-1, B-cell lymphoma/leukemia 2, nuclear factor erythroid-2 related factor 2 (Nrf2), superoxide dismutase 2 (SOD2), glutathione peroxidase 1, and the phase II metabolizing enzymes quinone oxidoreductase and heme oxygenase in the HCT group was significantly increased in comparison with the HC diet treatment (P < 0.05), whereas the mRNA expression of caspase 3, caspase 8, caspase 9, bcl-2 associated X protein, lipopolysaccharide binding protein, toll-like receptor 4, nuclear factor kappa-B (NFκB), tumor necrosis factor alpha, interleukin-1β, interleukin, and tumor necrosis factor receptor-associated factor 6 decreased significantly in the HCT treatment (P < 0.05). Compared with the HC treatment, the HCT diet significantly increased the protein expression of ZO-1, occludin, claudin-1, NQO1, HO-1, SOD2, serine/threonine kinase, p-Akt, Nrf2, and p-Nrf2; conversely, the expression of NFκB-related proteins p65 and pp65 was significantly decreased (P < 0.05). In addition, thiamine relieved the damage on the ruminal epithelium caused by the HC diet. The results show that dietary thiamine supplementation improves the rumen epithelial barrier function by regulating Nrf2–NFκB signaling pathways during high-concentrate-diet feeding

Carbon Sequestration in Relation to Shrub Size in the Desert Ecosystem

Desert ecosystems have been reported as the location of the long-sought ‘missing sink’ for atmospheric carbon dioxide and as a potentially important area for carbon sequestering from fossil fuel combustion in the future (Stone 2008). Researchers have found that net uptake of carbon in the Mojave Desert ranged from 102 to 127 g C m2/yr during a 3-year period, which is equivalent to the net ecosystem production of many forest ecosystems with a much higher biomass (Luyssaert et al. 2007; Wohlfahrt et al. 2008). Shrub is the dominant plant of desert ecosystems (Gratani et al. 2011); hence, it is important to understand the dynamics of carbon sequestration by shrubs as well as their role in desert ecosystem carbon balance. Information on the carbon sequestration associated with shrub size is limited. Our objective was, therefore, to find out the relationship between carbon sequestration potential and size of shrubs

Experimental Demonstration of High-Dimensional Quantum Steering With NN Measurement Settings

High-dimensional (HD) quantum systems possess advantages of larger channel capacity, stronger noise resilience and higher security against attacks in quantum communication compared with qubit systems. Here, we experimentally demonstrate HD quantum steering criterion with $n$ measurement settings in high-noise environments. We verify the unbounded violation of steering equalities, revealing a high strength of steering without full-state tomography. Moreover, our results exhibit that the noise resilience can be enhanced with both extra dimension and measurement settings. We experimentally certify 11-dimensional steering with 60.5% isotropic noise, exceeding the upper bound of 2-setting steering criteria. Our work paves the way for the certification of HD quantum correlations with high noise in quantum information processing

Variation of polarization distribution of reflected beam caused by spin separation

The variation of polarization distribution of reflected beam at specular interface and far field caused by spin separation has been studied. Due to the diffraction effect, we find a distinct difference of light polarization at the two regions. The variation of polarization distribution of reflected light provides a new method to measure the spin separation displacement caused by Spin Hall Effect of light.Comment: 7 pages, 5 figure

Spin polarization separation of reflected light at Brewster angle

A novel spin polarization separation of reflected light is observed, when a linearly polarized Gaussian beam impinges on an air-glass interface at Brewster angle. In the far-field zone, spins of photons are oppositely polarized in two regions along the direction perpendicular to incident plane. Spatial scale of this polarization is related to optical properties of dielectric and can be controlled by experimental configuration. We believe that this study benefits the manipulation of spins of photons and the development of methods for investigating optical properties of materials.Comment: 4 pages, 2 figure