688 research outputs found
Network Binarization via Contrastive Learning
Neural network binarization accelerates deep models by quantizing their
weights and activations into 1-bit. However, there is still a huge performance
gap between Binary Neural Networks (BNNs) and their full-precision (FP)
counterparts. As the quantization error caused by weights binarization has been
reduced in earlier works, the activations binarization becomes the major
obstacle for further improvement of the accuracy. BNN characterises a unique
and interesting structure, where the binary and latent FP activations exist in
the same forward pass (i.e., ).
To mitigate the information degradation caused by the binarization operation
from FP to binary activations, we establish a novel contrastive learning
framework while training BNNs through the lens of Mutual Information (MI)
maximization. MI is introduced as the metric to measure the information shared
between binary and FP activations, which assists binarization with contrastive
learning. Specifically, the representation ability of the BNNs is greatly
strengthened via pulling the positive pairs with binary and FP activations from
the same input samples, as well as pushing negative pairs from different
samples (the number of negative pairs can be exponentially large). This
benefits the downstream tasks, not only classification but also segmentation
and depth estimation, etc. The experimental results show that our method can be
implemented as a pile-up module on existing state-of-the-art binarization
methods and can remarkably improve the performance over them on CIFAR-10/100
and ImageNet, in addition to the great generalization ability on NYUD-v2.Comment: Accepted to ECCV 202
LP-BFGS attack: An adversarial attack based on the Hessian with limited pixels
Deep neural networks are vulnerable to adversarial attacks. Most white-box
attacks are based on the gradient of models to the input. Since the computation
and memory budget, adversarial attacks based on the Hessian information are not
paid enough attention. In this work, we study the attack performance and
computation cost of the attack method based on the Hessian with a limited
perturbation pixel number. Specifically, we propose the Limited Pixel BFGS
(LP-BFGS) attack method by incorporating the BFGS algorithm. Some pixels are
selected as perturbation pixels by the Integrated Gradient algorithm, which are
regarded as optimization variables of the LP-BFGS attack. Experimental results
across different networks and datasets with various perturbation pixel numbers
demonstrate our approach has a comparable attack with an acceptable computation
compared with existing solutions.Comment: 5 pages, 4 figure
Temporal-spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit
In recent years, medical information technology has made it possible for
electronic health record (EHR) to store fairly complete clinical data. This has
brought health care into the era of "big data". However, medical data are often
sparse and strongly correlated, which means that medical problems cannot be
solved effectively. With the rapid development of deep learning in recent
years, it has provided opportunities for the use of big data in healthcare. In
this paper, we propose a temporal-saptial correlation attention network (TSCAN)
to handle some clinical characteristic prediction problems, such as predicting
death, predicting length of stay, detecting physiologic decline, and
classifying phenotypes. Based on the design of the attention mechanism model,
our approach can effectively remove irrelevant items in clinical data and
irrelevant nodes in time according to different tasks, so as to obtain more
accurate prediction results. Our method can also find key clinical indicators
of important outcomes that can be used to improve treatment options. Our
experiments use information from the Medical Information Mart for Intensive
Care (MIMIC-IV) database, which is open to the public. Finally, we have
achieved significant performance benefits of 2.0\% (metric) compared to other
SOTA prediction methods. We achieved a staggering 90.7\% on mortality rate,
45.1\% on length of stay. The source code can be find:
\url{https://github.com/yuyuheintju/TSCAN}
The P Protein of Spring Viremia of Carp Virus Negatively Regulates the Fish Interferon Response by Inhibiting the Kinase Activity of TANK-Binding Kinase 1
Spring viremia of carp virus (SVCV) is an efficient pathogen causing high mortality in the common carp. Fish interferon (IFN) is a powerful cytokine enabling host cells to establish an antiviral response; therefore, the strategies that SVCV uses to avoid the cellular IFN response were investigated. Here, we report that the SVCV P protein is phosphorylated by cellular TANK-binding kinase 1 (TBK1), which decreases IFN regulatory factor 3 (IRF3) phosphorylation and suppresses IFN production. First, overexpression of P protein inhibited the IFN promoter activation induced by SVCV and the IFN activity activated by the mitochondrial antiviral signaling protein (MAVS) although TBK1 activity was not blocked by P protein. Second, P protein colocalized and interacted with TBK1. Dominant negative experiments suggested that the TBK1 N-terminal kinase domain interacted with P protein and was essential for P protein and IRF3 phosphorylation. Finally, P protein overexpression reduced the IRF3 phosphorylation activated by TBK1 and reduced host cellular ifn transcription. Collectively, our data demonstrated that the SVCV P protein is a decoy substrate for the host phosphokinase TBK1, preventing IFN production and facilitating SVCV replication. IMPORTANCE TBK1 is a pivotal phosphokinase that activates host IFN production to defend against viral infection; thus, it is a potential target for viruses to negatively regulate IFN response and facilitate viral evasion. We report that the SVCV P protein functions as a decoy substrate for cellular TBK1, leading to the reduction of IRF3 phosphorylation and suppression of IFN expression. These findings reveal a novel immune evasion mechanism of SVCV.</p
Lipschitz Continuity Retained Binary Neural Network
Relying on the premise that the performance of a binary neural network can be
largely restored with eliminated quantization error between full-precision
weight vectors and their corresponding binary vectors, existing works of
network binarization frequently adopt the idea of model robustness to reach the
aforementioned objective. However, robustness remains to be an ill-defined
concept without solid theoretical support. In this work, we introduce the
Lipschitz continuity, a well-defined functional property, as the rigorous
criteria to define the model robustness for BNN. We then propose to retain the
Lipschitz continuity as a regularization term to improve the model robustness.
Particularly, while the popular Lipschitz-involved regularization methods often
collapse in BNN due to its extreme sparsity, we design the Retention Matrices
to approximate spectral norms of the targeted weight matrices, which can be
deployed as the approximation for the Lipschitz constant of BNNs without the
exact Lipschitz constant computation (NP-hard). Our experiments prove that our
BNN-specific regularization method can effectively strengthen the robustness of
BNN (testified on ImageNet-C), achieving state-of-the-art performance on CIFAR
and ImageNet.Comment: Paper accepted to ECCV 202
Deep Reinforcement Learning Framework for Thoracic Diseases Classification via Prior Knowledge Guidance
The chest X-ray is often utilized for diagnosing common thoracic diseases. In
recent years, many approaches have been proposed to handle the problem of
automatic diagnosis based on chest X-rays. However, the scarcity of labeled
data for related diseases still poses a huge challenge to an accurate
diagnosis. In this paper, we focus on the thorax disease diagnostic problem and
propose a novel deep reinforcement learning framework, which introduces prior
knowledge to direct the learning of diagnostic agents and the model parameters
can also be continuously updated as the data increases, like a person's
learning process. Especially, 1) prior knowledge can be learned from the
pre-trained model based on old data or other domains' similar data, which can
effectively reduce the dependence on target domain data, and 2) the framework
of reinforcement learning can make the diagnostic agent as exploratory as a
human being and improve the accuracy of diagnosis through continuous
exploration. The method can also effectively solve the model learning problem
in the case of few-shot data and improve the generalization ability of the
model. Finally, our approach's performance was demonstrated using the
well-known NIH ChestX-ray 14 and CheXpert datasets, and we achieved competitive
results. The source code can be found here:
\url{https://github.com/NeaseZ/MARL}
Spin-density-wave transition in double-layer nickelate La3Ni2O7
Recently, a signature of high-temperature superconductivity above the liquid
nitrogen temperature (77 K) was reported for La3Ni2O7 under pressure. This
finding immediately stimulates intense interest in the possible high-Tc
superconducting mechanism in double-layer nickelates. Interestingly, the
pressure-dependent phase diagram inferred from transport measurements indicates
that superconductivity under high pressure emerges from the suppression of a
density-wave-like transition at ambient pressure, which is similar to
high-temperature superconductors. Therefore, clarifying the exact nature of the
density-wave-like transition is important for determining the mechanism of
superconductivity in double-layer nickelates. Here, nuclear magnetic resonance
(NMR) spectroscopy of 139La nuclei was performed to study the density-wave-like
transition in a single crystal of La3Ni2O7. The temperature-dependent 139La NMR
spectrum and nuclear spin-lattice relaxation rate (1/T1) provide unambiguous
evidence for a spin-density-wave (SDW) transition with a transition temperature
TSDW of ~ 150 K. Furthermore, the anisotropic splitting of the NMR spectrum
suggests a possible double spin stripe with magnetic moments along the c axis.
In addition, the present NMR measurements also revealed spatial inhomogeneity
of magnetism due to inner apical oxygen vacancies. All these results will be
helpful for building a connection between superconductivity and magnetic
interactions in double-layer nickelates.Comment: 14 pages, 4 figure
- …