5 research outputs found
Synthesizing Speech Test Cases with Text-to-Speech? An Empirical Study on the False Alarms in Automated Speech Recognition Testing
Recent studies have proposed the use of Text-To-Speech (TTS) systems to
automatically synthesise speech test cases on a scale and uncover a large
number of failures in ASR systems. However, the failures uncovered by synthetic
test cases may not reflect the actual performance of an ASR system when it
transcribes human audio, which we refer to as false alarms. Given a failed test
case synthesised from TTS systems, which consists of TTS-generated audio and
the corresponding ground truth text, we feed the human audio stating the same
text to an ASR system. If human audio can be correctly transcribed, an instance
of a false alarm is detected. In this study, we investigate false alarm
occurrences in five popular ASR systems using synthetic audio generated from
four TTS systems and human audio obtained from two commonly used datasets. Our
results show that the least number of false alarms is identified when testing
Deepspeech, and the number of false alarms is the highest when testing
Wav2vec2. On average, false alarm rates range from 21% to 34% in all five ASR
systems. Among the TTS systems used, Google TTS produces the least number of
false alarms (17%), and Espeak TTS produces the highest number of false alarms
(32%) among the four TTS systems. Additionally, we build a false alarm
estimator that flags potential false alarms, which achieves promising results:
a precision of 98.3%, a recall of 96.4%, an accuracy of 98.5%, and an F1 score
of 97.3%. Our study provides insight into the appropriate selection of TTS
systems to generate high-quality speech to test ASR systems. Additionally, a
false alarm estimator can be a way to minimise the impact of false alarms and
help developers choose suitable test inputs when evaluating ASR systems. The
source code used in this paper is publicly available on GitHub at
https://github.com/julianyonghao/FAinASRtest.Comment: 12 pages, Accepted at ISSTA202
Synthesizing speech test cases with text-to-speech? An empirical study on the false alarms in automated speech recognition testing
Identification and characterization of GLDC as host susceptibility gene to severe influenza
Abstract Glycine decarboxylase (GLDC) was prioritized as a candidate susceptibility gene to severe influenza in humans. The higher expression of GLDC derived from genetic variations may confer a higher risk to H7N9 and severe H1N1 infection. We sought to characterize GLDC as functional susceptibility gene that GLDC may intrinsically regulate antiviral response, thereby impacting viral replication and disease outcome. We demonstrated that GLDC inhibitor AOAA and siRNA depletion boosted IFNβ‐ and IFN‐stimulated genes (ISGs) in combination with PolyI:C stimulation. GLDC inhibition and depletion significantly amplified antiviral response of type I IFNs and ISGs upon viral infection and suppressed the replication of H1N1 and H7N9 viruses. Consistently, GLDC overexpression significantly promoted viral replication due to the attenuated antiviral responses. Moreover, GLDC inhibition in H1N1‐infected BALB/c mice recapitulated the amplified antiviral response and suppressed viral growth. AOAA provided potent protection to the infected mice from lethal infection, comparable to a standard antiviral against influenza viruses. Collectively, GLDC regulates cellular antiviral response and orchestrates viral growth. GLDC is a functional susceptibility gene to severe influenza in humans