391 research outputs found
Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retraining
A challenge of channel pruning is designing efficient and effective criteria
to select channels to prune. A widely used criterion is minimal performance
degeneration. To accurately evaluate the truth performance degeneration
requires retraining the survived weights to convergence, which is prohibitively
slow. Hence existing pruning methods use previous weights (without retraining)
to evaluate the performance degeneration. However, we observe the loss changes
differ significantly with and without retraining. It motivates us to develop a
technique to evaluate true loss changes without retraining, with which channels
to prune can be selected more reliably and confidently. We first derive a
closed-form estimator of the true loss change per pruning mask change, using
influence functions without retraining. Influence function which is from robust
statistics reveals the impacts of a training sample on the model's prediction
and is repurposed by us to assess impacts on true loss changes. We then show
how to assess the importance of all channels simultaneously and develop a novel
global channel pruning algorithm accordingly. We conduct extensive experiments
to verify the effectiveness of the proposed algorithm. To the best of our
knowledge, we are the first that shows evaluating true loss changes for pruning
without retraining is possible. This finding will open up opportunities for a
series of new paradigms to emerge that differ from existing pruning methods.
The code is available at https://github.com/hrcheng1066/IFSO.Comment: chrome-extension://ogjibjphoadhljaoicdnjnmgokohngcc/assets/icon-50207e67.pn
A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations
Modern deep neural networks, particularly recent large language models, come
with massive model sizes that require significant computational and storage
resources. To enable the deployment of modern models on resource-constrained
environments and accelerate inference time, researchers have increasingly
explored pruning techniques as a popular research direction in neural network
compression. However, there is a dearth of up-to-date comprehensive review
papers on pruning. To address this issue, in this survey, we provide a
comprehensive review of existing research works on deep neural network pruning
in a taxonomy of 1) universal/specific speedup, 2) when to prune, 3) how to
prune, and 4) fusion of pruning and other compression techniques. We then
provide a thorough comparative analysis of seven pairs of contrast settings for
pruning (e.g., unstructured/structured) and explore emerging topics, including
post-training pruning, different levels of supervision for pruning, and broader
applications (e.g., adversarial robustness) to shed light on the commonalities
and differences of existing methods and lay the foundation for further method
development. To facilitate future research, we build a curated collection of
datasets, networks, and evaluations on different applications. Finally, we
provide some valuable recommendations on selecting pruning methods and prospect
promising research directions. We build a repository at
https://github.com/hrcheng1066/awesome-pruning
Alkyl substituted cucurbit[6]uril assisted competitive fluorescence recognition of lysine and methionine in aqueous solution
The use of competitive ratiometric fluorescence indicator displacement chemosensors derived from two alkyl substituted cucurbit[6]uril-based host-guest complexes is reported. In particular, the differing binding abilities of two cucurbit[6]uril derivatives towards the target analytes led to a useful ratiometric detection signal output for the discrimination of lysine and methionine versus the other tested α-amino acids in aqueous solution
Effects of Rain Events on Carbon Fluxes from Biological Soil Crusts
In dry ecosystems, biological soil crusts (BSCs) have been suggested as one of the factors responsible for the large rate of annual CO2 net uptake (Xie et al. 2009). However, most studies carried out on carbon (C) fluxes in arid and semi-arid ecosystems, such as soil respiration, have neglected the carbon fluxes from BSCs. Although BSCs are a vital component of the dry-land soil C cycle, few studies have parameterized the conditions required for photosynthesis in BSCs or determined BSCs respiration (Elbert et al. 2009, Castillo-Monroy et al. 2011). Precipitation in dry land is dominated by small events (Lauenroth and Bradford 2009). Even the smallest events will influence the carbon fluxes of BSCs, while intermediate pulses might wet the subsurface biotic community, and typically only larger events are used by plants for carbon gain or growth of roots or shoots (Belnap et al. 2005). As BSCs dry quickly and are hence very responsive to moisture pulses, the pulsed nature of precipitation can lead to highly variable carbon fluxes from BSCs (Bowling et al. 2011). Therefore, it is very important to study the effect of rain events upon carbon fluxes through BSCs in the dry ecosystem
Thiamine Alleviates High-Concentrate-Diet-Induced Oxidative Stress, Apoptosis, and Protects the Rumen Epithelial Barrier Function in Goats
High-concentrate diets are continually used in ruminants to meet the needs of milk yield, which can lead to the occurrence of subacute rumen acidosis in ruminants. This study investigated the protective effects of dietary thiamine supplementation on the damage of the ruminal epithelium barrier function in goats fed a high-concentrate diet. Twenty-four healthy Boer goats (live weight of 35.62 ± 2.4 kg; age, 1 year) were randomly assigned into three treatments, with eight goats in each treatment, consuming one of three diets: a low-concentrate diet (CON; concentrate/forage, 30:70), a high-concentrate diet (HC; concentrate/forage, 70:30), or a high-concentrate diet with 200 mg of thiamine/kg of dry matter intake (HCT; concentrate/forage, 70:30) for 12 weeks. The additional dose of thiamine was based on our previous study wherein thiamine ameliorates inflammation. Compared with HC treatment, the HCT treatment had markedly higher concentrations of glutathione, superoxide dismutase, and glutathione peroxidase and total antioxidant capacity (P < 0.05) in plasma and rumen epithelium. The results showed that the apoptosis index was lower (P < 0.05) in the HCT treatment than in that of the HC treatment. Compared with the HC treatment, permeability and the electrophysiology parameter short circuit current for ruminal epithelial tissue were significantly decreased (P < 0.05) in the HCT treatment. The immunohistochemical results showed that the expression distribution of tight junctions including claudin-1, claudin-4, occludin, and zonula occludin-1 (ZO-1) was greater (P < 0.05) in the HCT treatments than in the HC treatment. The mRNA expression in the rumen epithelium of ZO-1, occludin, claudin-1, B-cell lymphoma/leukemia 2, nuclear factor erythroid-2 related factor 2 (Nrf2), superoxide dismutase 2 (SOD2), glutathione peroxidase 1, and the phase II metabolizing enzymes quinone oxidoreductase and heme oxygenase in the HCT group was significantly increased in comparison with the HC diet treatment (P < 0.05), whereas the mRNA expression of caspase 3, caspase 8, caspase 9, bcl-2 associated X protein, lipopolysaccharide binding protein, toll-like receptor 4, nuclear factor kappa-B (NFκB), tumor necrosis factor alpha, interleukin-1β, interleukin, and tumor necrosis factor receptor-associated factor 6 decreased significantly in the HCT treatment (P < 0.05). Compared with the HC treatment, the HCT diet significantly increased the protein expression of ZO-1, occludin, claudin-1, NQO1, HO-1, SOD2, serine/threonine kinase, p-Akt, Nrf2, and p-Nrf2; conversely, the expression of NFκB-related proteins p65 and pp65 was significantly decreased (P < 0.05). In addition, thiamine relieved the damage on the ruminal epithelium caused by the HC diet. The results show that dietary thiamine supplementation improves the rumen epithelial barrier function by regulating Nrf2–NFκB signaling pathways during high-concentrate-diet feeding
Carbon Sequestration in Relation to Shrub Size in the Desert Ecosystem
Desert ecosystems have been reported as the location of the long-sought ‘missing sink’ for atmospheric carbon dioxide and as a potentially important area for carbon sequestering from fossil fuel combustion in the future (Stone 2008). Researchers have found that net uptake of carbon in the Mojave Desert ranged from 102 to 127 g C m2/yr during a 3-year period, which is equivalent to the net ecosystem production of many forest ecosystems with a much higher biomass (Luyssaert et al. 2007; Wohlfahrt et al. 2008). Shrub is the dominant plant of desert ecosystems (Gratani et al. 2011); hence, it is important to understand the dynamics of carbon sequestration by shrubs as well as their role in desert ecosystem carbon balance. Information on the carbon sequestration associated with shrub size is limited. Our objective was, therefore, to find out the relationship between carbon sequestration potential and size of shrubs
Experimental Demonstration of High-Dimensional Quantum Steering With Measurement Settings
High-dimensional (HD) quantum systems possess advantages of larger channel
capacity, stronger noise resilience and higher security against attacks in
quantum communication compared with qubit systems. Here, we experimentally
demonstrate HD quantum steering criterion with measurement settings in
high-noise environments. We verify the unbounded violation of steering
equalities, revealing a high strength of steering without full-state
tomography. Moreover, our results exhibit that the noise resilience can be
enhanced with both extra dimension and measurement settings. We experimentally
certify 11-dimensional steering with 60.5% isotropic noise, exceeding the upper
bound of 2-setting steering criteria. Our work paves the way for the
certification of HD quantum correlations with high noise in quantum information
processing
Spin polarization separation of reflected light at Brewster angle
A novel spin polarization separation of reflected light is observed, when a
linearly polarized Gaussian beam impinges on an air-glass interface at Brewster
angle. In the far-field zone, spins of photons are oppositely polarized in two
regions along the direction perpendicular to incident plane. Spatial scale of
this polarization is related to optical properties of dielectric and can be
controlled by experimental configuration. We believe that this study benefits
the manipulation of spins of photons and the development of methods for
investigating optical properties of materials.Comment: 4 pages, 2 figure
Variation of polarization distribution of reflected beam caused by spin separation
The variation of polarization distribution of reflected beam at specular
interface and far field caused by spin separation has been studied. Due to the
diffraction effect, we find a distinct difference of light polarization at the
two regions. The variation of polarization distribution of reflected light
provides a new method to measure the spin separation displacement caused by
Spin Hall Effect of light.Comment: 7 pages, 5 figure
- …