35 research outputs found
Efficient Adaptive Activation Rounding for Post-Training Quantization
Post-training quantization attracts increasing attention due to its
convenience in deploying quantized neural networks. Although
rounding-to-nearest remains the prevailing method for DNN quantization, prior
research has demonstrated its suboptimal nature when applied to weight
quantization. They propose optimizing weight rounding schemes by leveraging
output error rather than the traditional weight quantization error. Our study
reveals that similar rounding challenges also extend to activation
quantization. Despite the easy generalization, the challenges lie in the
dynamic nature of activation. Adaptive rounding is expected for varying
activations and the method is subjected to runtime overhead. To tackle this, we
propose the AQuant quantization framework with a novel perspective to reduce
output error by adjusting rounding schemes of activations. Instead of using the
constant rounding border 0.5 of the rounding-to-nearest operation, we make the
border become a function w.r.t. the activation value to change the activation
rounding by the adaptive border. To deal with the runtime overhead, we use a
coarse-grained version of the border function. Finally, we introduce our
framework to optimize the border function. Extensive experiments show that
AQuant achieves notable improvements compared to state-of-the-art works and
pushes the accuracy of ResNet-18 up to 60.31% under the 2-bit weight and
activation quantization
The M-T hook structure increases the potency of HIV-1 fusion inhibitor sifuvirtide and overcomes drug resistance
Objectives Peptides derived from the C-terminal heptad repeat (CHR) of HIV-1 gp41 are potent fusion inhibitors. We have recently demonstrated that the unique M-T hook structure preceding the pocket-binding motif of CHR peptide-based inhibitors can greatly improve their antiviral activity. In this study, we applied the M-T hook structure to optimize sifuvirtide (SFT), a potent CHR-derived inhibitor currently under Phase III clinical trials in China. Methods The peptide MT-SFT was generated by incorporating two M-T hook residues (Met-Thr) into the N-terminus of sifuvirtide. Multiple structural and functional approaches were used to determine the biophysical properties and antiviral activity of MT-SFT. Results The high-resolution crystal structure of MT-SFT reveals a highly conserved M-T hook conformation. Compared with sifuvirtide, MT-SFT exhibited a significant improvement in the ability to bind to the N-terminal heptad repeat, to block the formation of the six helix bundle and to inhibit HIV-1 Env-mediated cell fusion, viral entry and infection. Importantly, MT-SFT was fully active against sifuvirtide- and enfuvirtide (T20)-resistant HIV-1 variants and displayed a high genetic barrier to developing drug resistance. Conclusions Our studies have verified that the M-T hook structure offers a general strategy for designing novel HIV-1 fusion inhibitors and provide new insights into viral entry and inhibitio
Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training
An activation function is an element-wise mathematical function and plays a
crucial role in deep neural networks (DNN). Many novel and sophisticated
activation functions have been proposed to improve the DNN accuracy but also
consume massive memory in the training process with back-propagation. In this
study, we propose the nested forward automatic differentiation (Forward-AD),
specifically for the element-wise activation function for memory-efficient DNN
training. We deploy nested Forward-AD in two widely-used deep learning
frameworks, TensorFlow and PyTorch, which support the static and dynamic
computation graph, respectively. Our evaluation shows that nested Forward-AD
reduces the memory footprint by up to 1.97x than the baseline model and
outperforms the recomputation by 20% under the same memory reduction ratio.Comment: 8 pages, ICCD 202
A novel trifunctional IgG-like bispecific antibody to inhibit HIV-1 infection and enhance lysis of HIV by targeting activation of complement
BACKGROUND: The complement system is not only a key component of innate immunity but also provides a first line of defense against invading pathogens, especially for viral pathogens. Human immunodeficiency virus (HIV), however, possesses several mechanisms to evade complement-mediated lysis (CoML) and exploit the complement system to enhance viral infectivity. Responsible for this intrinsic resistance against complement-mediated virolysis are complement regulatory membrane proteins derived from the host cell that inherently downregulates complement activation at several stages of the cascade. In addition, HIV is protected from complement-mediated lysis by binding soluble factor H (fH) through the viral envelope proteins, gp120 and gp41. Whereas inhibition of complement activity is the desired outcome in the vast majority of therapeutic approaches, there is a broader potential for complement-mediated inhibition of HIV by complement local stimulation. PRESENTATION OF THE HYPOTHESIS: Our previous studies have proven that the complement-mediated antibody-dependent enhancement of HIV infection is mediated by the association of complement receptor type 2 bound to the C3 fragment and deposited on the surface of HIV virions. Thus, we hypothesize that another new activator of complement, consisting of two dsFv (against gp120 and against C3d respectively) linked to a complement-activating human IgG1 Fc domain ((anti-gp120 × anti-C3d)-Fc), can not only target and amplify complement activation on HIV virions for enhancing the efficiency of HIV lysis, but also reduce the infectivity of HIV through blocking the gp120 and C3d on the surface of HIV. TESTING THE HYPOTHESIS: Our hypothesis was tested using cell-free HIV-1 virions cultivated in vitro and assessment of virus opsonization was performed by incubating appropriate dilutions of virus with medium containing normal human serum and purified (anti-gp120 × anti-C3d)-Fc proteins. As a control group, viruses were incubated with normal human serum under the same conditions. Virus neutralization assays were used to estimate the degree of (anti-gp120 × anti-C3d)-Fc lysis of HIV compared to untreated virus. IMPLICATIONS OF THE HYPOTHESIS: The targeted complement activator, (anti-gp120 × anti-C3d)-Fc, can be used as a novel approach to HIV therapy by abrogating the complement-enhanced HIV infection of cells
Dynamic Slicing for Deep Neural Networks
Program slicing has been widely applied in a variety of software engineering
tasks. However, existing program slicing techniques only deal with traditional
programs that are constructed with instructions and variables, rather than
neural networks that are composed of neurons and synapses. In this paper, we
propose NNSlicer, the first approach for slicing deep neural networks based on
data flow analysis. Our method understands the reaction of each neuron to an
input based on the difference between its behavior activated by the input and
the average behavior over the whole dataset. Then we quantify the neuron
contributions to the slicing criterion by recursively backtracking from the
output neurons, and calculate the slice as the neurons and the synapses with
larger contributions. We demonstrate the usefulness and effectiveness of
NNSlicer with three applications, including adversarial input detection, model
pruning, and selective model protection. In all applications, NNSlicer
significantly outperforms other baselines that do not rely on data flow
analysis.Comment: 11 pages, ESEC/FSE '2
Preparation of an Environmentally Friendly Nano-Insecticide through Encapsulation in Polymeric Liposomes and Its Insecticidal Activities against the Fall Armyworm, Spodoptera frugiperda
The insecticide emamectin benzoate (EB) was formulated with nanoparticles composed of DSPE-PEG2000-NH2 by the co-solvent method to determine its adverse impacts on the environment and to reinforce its dispersion, adhesion, and biocompatibility. A good encapsulation efficiency (70.5 ± 1.5%) of EB loaded in DSPE-PEG2000-NH2 polymeric liposomes was confirmed. Dynamic light scattering (DLS), transmission electron microscopy (TEM), and contact angle meter measurements revealed that the DSPE-EB nanoparticles had a regular distribution, spherical shape, and good leaf wettability. The contact angle on corn leaves was 47.26°, and the maximum retention was higher than that of the reference product. DSPE-EB nanoparticles had strong adhesion on maize foliage and a good, sustained release property. The efficacy trial showed that the DSPE-EB nanoparticles had a strong control effect on S. frugiperda larvae, with the LC50 of 0.046 mg/L against the third-instar S. furgiperda larve after 48 h treatment. All these results indicate that DSPE-EB nanoparticles can serve as an insecticide carrier with lower environmental impact, sustained release property, and effective control of pests