95 research outputs found
On-Line Bayesian Speaker Adaptation By Using Tree-Structured Transformation and Robust Priors
This paper presents new results by using our previously proposed on-line Bayesian learning approach for affine transformation parameter estimation in speaker adaptation. The on-line Bayesian learning technique allows updating parameter estimates after each utterance and it can accommodate flexible forms of transformation functions as well as prior probability density functions. We show through experimental results the robustness of heavy tailed priors to mismatch in prior density estimation. We also show that by properly choosing the transformation matrices and depths of hierarchical trees, recognition performance improved significantly
Slim Embedding Layers for Recurrent Neural Language Models
Recurrent neural language models are the state-of-the-art models for language
modeling. When the vocabulary size is large, the space taken to store the model
parameters becomes the bottleneck for the use of recurrent neural language
models. In this paper, we introduce a simple space compression method that
randomly shares the structured parameters at both the input and output
embedding layers of the recurrent neural language models to significantly
reduce the size of model parameters, but still compactly represent the original
input and output embedding layers. The method is easy to implement and tune.
Experiments on several data sets show that the new method can get similar
perplexity and BLEU score results while only using a very tiny fraction of
parameters.Comment: To appear at AAAI 201
Semantic N-gram Language Modeling with the Latent Maximum Entropy Principle
We describe a unified probabilistic framework for statistical language modeling-the latent maximum entropy principle-which can effectively incorporate various aspects of natural language, such as local word interaction, syntactic structure and semantic document information. Unlike previous work on maximum entropy methods for language modeling, which only allow explicit features to be modeled, our framework also allows relationships over hidden features to be captured, resulting in a more expressive language model. We describe efficient algorithms for marginalization, inference and normalization in our extended models. We then present experimental results for our approach on the Wall Street Journal corpus
Learning Mixture Models with the Latent Maximum Entropy Principle
We present a new approach to estimating mixture models based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes’ maximum entropy principle and from standard maximum likelihood estimation. We demonstrate the LME principle by deriving new algorithms for mixture model estimation, and show how robust new variants of the EM algorithm can be developed. Our experiments show that estimation based on LME generally yields better results than maximum likelihood estimation, particularly when inferring latent variable models from small amounts of data
Boltzmann Machine Learning with the Latent Maximum Entropy Principle
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation. We demonstrate the LME principle BY deriving new algorithms for Boltzmann machine parameter estimation, and show how robust and fast new variant of the EM algorithm can be developed. Our experiments show that estimation based on LME generally yields better results than maximum likelihood estimation, particularly when inferring hidden units from small amounts of data
Empowering LLM to use Smartphone for Intelligent Task Automation
Mobile task automation is an attractive technique that aims to enable
voice-based hands-free user interaction with smartphones. However, existing
approaches suffer from poor scalability due to the limited language
understanding ability and the non-trivial manual efforts required from
developers or end-users. The recent advance of large language models (LLMs) in
language understanding and reasoning inspires us to rethink the problem from a
model-centric perspective, where task preparation, comprehension, and execution
are handled by a unified language model. In this work, we introduce AutoDroid,
a mobile task automation system that can handle arbitrary tasks on any Android
application without manual efforts. The key insight is to combine the
commonsense knowledge of LLMs and domain-specific knowledge of apps through
automated dynamic analysis. The main components include a functionality-aware
UI representation method that bridges the UI with the LLM, exploration-based
memory injection techniques that augment the app-specific domain knowledge of
LLM, and a multi-granularity query optimization module that reduces the cost of
model inference. We integrate AutoDroid with off-the-shelf LLMs including
online GPT-4/GPT-3.5 and on-device Vicuna, and evaluate its performance on a
new benchmark for memory-augmented Android task automation with 158 common
tasks. The results demonstrated that AutoDroid is able to precisely generate
actions with an accuracy of 90.9%, and complete tasks with a success rate of
71.3%, outperforming the GPT-4-powered baselines by 36.4% and 39.7%. The demo,
benchmark suites, and source code of AutoDroid will be released at
url{https://autodroid-sys.github.io/}
- …