6 research outputs found
Map Reconstruction of radio observations with Conditional Invertible Neural Networks
In radio astronomy, the challenge of reconstructing a sky map from time
ordered data (TOD) is known as an inverse problem. Standard map-making
techniques and gridding algorithms are commonly employed to address this
problem, each offering its own benefits such as producing minimum-variance
maps. However, these approaches also carry limitations such as computational
inefficiency and numerical instability in map-making and the inability to
remove beam effects in grid-based methods. To overcome these challenges, this
study proposes a novel solution through the use of the conditional invertible
neural network (cINN) for efficient sky map reconstruction. With the aid of
forward modeling, where the simulated TODs are generated from a given sky model
with a specific observation, the trained neural network can produce accurate
reconstructed sky maps. Using the five-hundred-meter aperture spherical radio
telescope (FAST) as an example, cINN demonstrates remarkable performance in map
reconstruction from simulated TODs, achieving a mean squared error of , a structural similarity index of ,
and a peak signal-to-noise ratio of at the level.
Furthermore, by sampling in the latent space of cINN, the reconstruction errors
for each pixel can be accurately quantified.Comment: Accepted for publication in Research in Astronomy and Astrophysics
(RAA); 20 pages, 10 figure
Explicit Intensity Control for Accented Text-to-speech
Accented text-to-speech (TTS) synthesis seeks to generate speech with an
accent (L2) as a variant of the standard version (L1). How to control the
intensity of accent in the process of TTS is a very interesting research
direction, and has attracted more and more attention. Recent work design a
speaker-adversarial loss to disentangle the speaker and accent information, and
then adjust the loss weight to control the accent intensity. However, such a
control method lacks interpretability, and there is no direct correlation
between the controlling factor and natural accent intensity. To this end, this
paper propose a new intuitive and explicit accent intensity control scheme for
accented TTS. Specifically, we first extract the posterior probability, called
as ``goodness of pronunciation (GoP)'' from the L1 speech recognition model to
quantify the phoneme accent intensity for accented speech, then design a
FastSpeech2 based TTS model, named Ai-TTS, to take the accent intensity
expression into account during speech generation. Experiments show that the our
method outperforms the baseline model in terms of accent rendering and
intensity control.Comment: 5 pages, 3 figures. Submitted to ICASSP 2023. arXiv admin note: text
overlap with arXiv:2209.1080
Learning Diverse Tone Styles for Image Retouching
Image retouching, aiming to regenerate the visually pleasing renditions of
given images, is a subjective task where the users are with different aesthetic
sensations. Most existing methods deploy a deterministic model to learn the
retouching style from a specific expert, making it less flexible to meet
diverse subjective preferences. Besides, the intrinsic diversity of an expert
due to the targeted processing on different images is also deficiently
described. To circumvent such issues, we propose to learn diverse image
retouching with normalizing flow-based architectures. Unlike current flow-based
methods which directly generate the output image, we argue that learning in a
style domain could (i) disentangle the retouching styles from the image
content, (ii) lead to a stable style presentation form, and (iii) avoid the
spatial disharmony effects. For obtaining meaningful image tone style
representations, a joint-training pipeline is delicately designed, which is
composed of a style encoder, a conditional RetouchNet, and the image tone style
normalizing flow (TSFlow) module. In particular, the style encoder predicts the
target style representation of an input image, which serves as the conditional
information in the RetouchNet for retouching, while the TSFlow maps the style
representation vector into a Gaussian distribution in the forward pass. After
training, the TSFlow can generate diverse image tone style vectors by sampling
from the Gaussian distribution. Extensive experiments on MIT-Adobe FiveK and
PPR10K datasets show that our proposed method performs favorably against
state-of-the-art methods and is effective in generating diverse results to
satisfy different human aesthetic preferences. Source code and pre-trained
models are publicly available at https://github.com/SSRHeart/TSFlow
Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities
Multimodal emotion recognition leverages complementary information across
modalities to gain performance. However, we cannot guarantee that the data of
all modalities are always present in practice. In the studies to predict the
missing data across modalities, the inherent difference between heterogeneous
modalities, namely the modality gap, presents a challenge. To address this, we
propose to use invariant features for a missing modality imagination network
(IF-MMIN) which includes two novel mechanisms: 1) an invariant feature learning
strategy that is based on the central moment discrepancy (CMD) distance under
the full-modality scenario; 2) an invariant feature based imagination module
(IF-IM) to alleviate the modality gap during the missing modalities prediction,
thus improving the robustness of multimodal joint representation. Comprehensive
experiments on the benchmark dataset IEMOCAP demonstrate that the proposed
model outperforms all baselines and invariantly improves the overall emotion
recognition performance under uncertain missing-modality conditions. We release
the code at: https://github.com/ZhuoYulang/IF-MMIN.Comment: 5 pages, 3 figures, 1 table. Submitted to ICASSP 2023. We release the
code at: https://github.com/ZhuoYulang/IF-MMI