309 research outputs found
MDFL: Multi-domain Diffusion-driven Feature Learning
High-dimensional images, known for their rich semantic information, are
widely applied in remote sensing and other fields. The spatial information in
these images reflects the object's texture features, while the spectral
information reveals the potential spectral representations across different
bands. Currently, the understanding of high-dimensional images remains limited
to a single-domain perspective with performance degradation. Motivated by the
masking texture effect observed in the human visual system, we present a
multi-domain diffusion-driven feature learning network (MDFL) , a scheme to
redefine the effective information domain that the model really focuses on.
This method employs diffusion-based posterior sampling to explicitly consider
joint information interactions between the high-dimensional manifold structures
in the spectral, spatial, and frequency domains, thereby eliminating the
influence of masking texture effects in visual models. Additionally, we
introduce a feature reuse mechanism to gather deep and raw features of
high-dimensional data. We demonstrate that MDFL significantly improves the
feature extraction performance of high-dimensional data, thereby providing a
powerful aid for revealing the intrinsic patterns and structures of such data.
The experimental results on three multi-modal remote sensing datasets show that
MDFL reaches an average overall accuracy of 98.25%, outperforming various
state-of-the-art baseline schemes. The code will be released, contributing to
the computer vision community
Guided Hybrid Quantization for Object detection in Multimodal Remote Sensing Imagery via One-to-one Self-teaching
Considering the computation complexity, we propose a Guided Hybrid
Quantization with One-to-one Self-Teaching (GHOST}) framework. More concretely,
we first design a structure called guided quantization self-distillation
(GQSD), which is an innovative idea for realizing lightweight through the
synergy of quantization and distillation. The training process of the
quantization model is guided by its full-precision model, which is time-saving
and cost-saving without preparing a huge pre-trained model in advance. Second,
we put forward a hybrid quantization (HQ) module to obtain the optimal bit
width automatically under a constrained condition where a threshold for
distribution distance between the center and samples is applied in the weight
value search space. Third, in order to improve information transformation, we
propose a one-to-one self-teaching (OST) module to give the student network a
ability of self-judgment. A switch control machine (SCM) builds a bridge
between the student network and teacher network in the same location to help
the teacher to reduce wrong guidance and impart vital knowledge to the student.
This distillation method allows a model to learn from itself and gain
substantial improvement without any additional supervision. Extensive
experiments on a multimodal dataset (VEDAI) and single-modality datasets (DOTA,
NWPU, and DIOR) show that object detection based on GHOST outperforms the
existing detectors. The tiny parameters (<9.7 MB) and Bit-Operations (BOPs)
(<2158 G) compared with any remote sensing-based, lightweight or
distillation-based algorithms demonstrate the superiority in the lightweight
design domain. Our code and model will be released at
https://github.com/icey-zhang/GHOST.Comment: This article has been delivered to TRGS and is under revie
Fault Detection and Diagnosis for Gas Turbines Based on a Kernelized Information Entropy Model
Gas turbines are considered as one kind of the most important devices in power engineering and have been widely used in power generation, airplanes, and naval ships and also in oil drilling platforms. However, they are monitored without man on duty in the most cases. It is highly desirable to develop techniques and systems to remotely monitor their conditions and analyze their faults. In this work, we introduce a remote system for online condition monitoring and fault diagnosis of gas turbine on offshore oil well drilling platforms based on a kernelized information entropy model. Shannon information entropy is generalized for measuring the uniformity of exhaust temperatures, which reflect the overall states of the gas paths of gas turbine. In addition, we also extend the entropy to compute the information quantity of features in kernel spaces, which help to select the informative features for a certain recognition task. Finally, we introduce the information entropy based decision tree algorithm to extract rules from fault samples. The experiments on some real-world data show the effectiveness of the proposed algorithms
Protein-ligand binding representation learning from fine-grained interactions
The binding between proteins and ligands plays a crucial role in the realm of
drug discovery. Previous deep learning approaches have shown promising results
over traditional computationally intensive methods, but resulting in poor
generalization due to limited supervised data. In this paper, we propose to
learn protein-ligand binding representation in a self-supervised learning
manner. Different from existing pre-training approaches which treat proteins
and ligands individually, we emphasize to discern the intricate binding
patterns from fine-grained interactions. Specifically, this self-supervised
learning problem is formulated as a prediction of the conclusive binding
complex structure given a pocket and ligand with a Transformer based
interaction module, which naturally emulates the binding process. To ensure the
representation of rich binding information, we introduce two pre-training
tasks, i.e.~atomic pairwise distance map prediction and mask ligand
reconstruction, which comprehensively model the fine-grained interactions from
both structure and feature space. Extensive experiments have demonstrated the
superiority of our method across various binding tasks, including
protein-ligand affinity prediction, virtual screening and protein-ligand
docking
An EEG-Based Multi-Modal Emotion Database With Both Posed And Authentic Facial Actions For Emotion Analysis
Emotion is an experience associated with a particular pattern of physiological activity along with different physiological, behavioral and cognitive changes. One behavioral change is facial expression, which has been studied extensively over the past few decades. Facial behavior varies with a person\u27s emotion according to differences in terms of culture, personality, age, context, and environment. In recent years, physiological activities have been used to study emotional responses. A typical signal is the electroencephalogram (EEG), which measures brain activity. Most of existing EEG-based emotion analysis has overlooked the role of facial expression changes. There exits little research on the relationship between facial behavior and brain signals due to the lack of dataset measuring both EEG and facial action signals simultaneously. To address this problem, we propose to develop a new database by collecting facial expressions, action units, and EEGs simultaneously. We recorded the EEGs and face videos of both posed facial actions and spontaneous expressions from 29 participants with different ages, genders, ethnic backgrounds. Differing from existing approaches, we designed a protocol to capture the EEG signals by evoking participants\u27 individual action units explicitly. We also investigated the relation between the EEG signals and facial action units. As a baseline, the database has been evaluated through the experiments on both posed and spontaneous emotion recognition with images alone, EEG alone, and EEG fused with images, respectively. The database will be released to the research community to advance the state of the art for automatic emotion recognition
- …