Bayesian Adaptive Learning of Deep Latent Variables for Acoustic Knowledge Transfer

Abstract

In this dissertation, we propose a Bayesian adaptive learning framework by focusing our attention on estimating a manageable number of latent variables of deep neural networks (DNNs).The deep latent variables here refer to the unobservable representations of data, and they usually correspond to intermediate hidden embedding outputs from a specific layer of DNNs. Within this framework, we explore two Bayesian estimation techniques: Variational Bayes (VB) and Maximum A Posteriori (MAP). The VB approach utilizes variational inference to approximate the entire posterior distribution and introduces Gaussian mean-field variational inference and empirical Bayes to handle different parallel data scenarios across domains. The MAP approach, on the other hand, focuses on point estimation of latent variables using Gaussian-based and Dirichlet-based distribution assumptions.Experimental validations on acoustic adaptation tasks demonstrate the superiority of the proposed Bayesian adaptation approaches over other knowledge transfer methods. Furthermore, these Bayesian adaptation techniques have been successfully applied to large pre-trained models like Wav2Vec2 and Whisper, showcasing considerable advantages and affirming their efficacy in complex acoustic settings, thus advancing effective model adaptation strategies.Ph.D.Electrical and Computer Engineerin