On-device automatic speech recognition systems face several challenges
compared to server-based systems. They have to meet stricter constraints in
terms of speed, disk size and memory while maintaining the same accuracy. Often
they have to serve several applications with different distributions at once,
such as communicating with a virtual assistant and speech-to-text. The simplest
solution to serve multiple applications is to build application-specific
(language) models, but this leads to an increase in memory. Therefore, we
explore different data- and architecture-driven language modeling approaches to
build a single application-agnostic model. We propose two novel feed-forward
architectures that find an optimal trade off between different on-device
constraints. In comparison to the application-specific solution, one of our
novel approaches reduces the disk size by half, while maintaining speed and
accuracy of the original model.Comment: accepted for ACL 2023 industry trac