309 research outputs found
A General Unfolding Speech Enhancement Method Motivated by Taylor's Theorem
While deep neural networks have facilitated significant advancements in the
field of speech enhancement, most existing methods are developed following
either empirical or relatively blind criteria, lacking adequate guidelines in
pipeline design. Inspired by Taylor's theorem, we propose a general unfolding
framework for both single- and multi-channel speech enhancement tasks.
Concretely, we formulate the complex spectrum recovery into the spectral
magnitude mapping in the neighborhood space of the noisy mixture, in which an
unknown sparse term is introduced and applied for phase modification in
advance. Based on that, the mapping function is decomposed into the
superimposition of the 0th-order and high-order polynomials in Taylor's series,
where the former coarsely removes the interference in the magnitude domain and
the latter progressively complements the remaining spectral detail in the
complex spectrum domain. In addition, we study the relation between adjacent
order terms and reveal that each high-order term can be recursively estimated
with its lower-order term, and each high-order term is then proposed to
evaluate using a surrogate function with trainable weights so that the whole
system can be trained in an end-to-end manner. Given that the proposed
framework is devised based on Taylor's theorem, it possesses improved internal
flexibility. Extensive experiments are conducted on WSJ0-SI84, DNS-Challenge,
Voicebank+Demand, spatialized Librispeech, and L3DAS22 multi-channel speech
enhancement challenge datasets. Quantitative results show that the proposed
approach yields competitive performance over existing top-performing approaches
in terms of multiple objective metrics.Comment: Submitted to TASLP, revised version, 17 page
CLIPC8: Face liveness detection algorithm based on image-text pairs and contrastive learning
Face recognition technology is widely used in the financial field, and
various types of liveness attack behaviors need to be addressed. Existing
liveness detection algorithms are trained on specific training datasets and
tested on testing datasets, but their performance and robustness in
transferring to unseen datasets are relatively poor. To tackle this issue, we
propose a face liveness detection method based on image-text pairs and
contrastive learning, dividing liveness attack problems in the financial field
into eight categories and using text information to describe the images of
these eight types of attacks. The text encoder and image encoder are used to
extract feature vector representations for the classification description text
and face images, respectively. By maximizing the similarity of positive samples
and minimizing the similarity of negative samples, the model learns shared
representations between images and texts. The proposed method is capable of
effectively detecting specific liveness attack behaviors in certain scenarios,
such as those occurring in dark environments or involving the tampering of ID
card photos. Additionally, it is also effective in detecting traditional
liveness attack methods, such as printing photo attacks and screen remake
attacks. The zero-shot capabilities of face liveness detection on five public
datasets, including NUAA, CASIA-FASD, Replay-Attack, OULU-NPU and MSU-MFSD also
reaches the level of commercial algorithms. The detection capability of
proposed algorithm was verified on 5 types of testing datasets, and the results
show that the method outperformed commercial algorithms, and the detection
rates reached 100% on multiple datasets. Demonstrating the effectiveness and
robustness of introducing image-text pairs and contrastive learning into
liveness detection tasks as proposed in this paper
Generation of Spatiotemporal Vortex Pulses by Simple Diffractive Grating
Spatiotemporal vortex pulses are wave packets that carry transverse orbital
angular momentum, exhibiting exotic structured wavefronts that can twist
through space and time. Existing methods to generate these pulses require
complex setups like spatial light modulators or computer-optimized structures.
Here, we demonstrate a new approach to generate spatiotemporal vortex pulses
using just a simple diffractive grating. The key is constructing a phase vortex
in frequency-momentum space by leveraging symmetry, resonance, and diffraction.
Our approach is applicable to any wave system. We use a liquid surface wave
platform to directly demonstrate and observe the real-time generation and
evolution of spatiotemporal vortex pulses. This straightforward technique
provides opportunities to explore pulse dynamics and potential applications
across different disciplines
- …