On-device LLM-driven Application Function Calling for Natural Language Interaction

Large language models can enable application programmers to translate user instructions in natural language into a sequence of corresponding function calls; however, currently LLMs that provide such capabilities are housed in the cloud. Use of remotely hosted LLMs can introduce operational latency, affect reliability, and can be infeasible in certain cases. This disclosure describes the use of an on-device LLM to enable app developers to support natural language interactions with users without having to implement code within their applications. Per the techniques, with user permission, a user command in natural language as received by an application is provided to the on-device LLM that generates an appropriate sequence of function calls that can be executed one at a time, with each function receiving the output of the prior function, to perform the requested task. The techniques enable a deep level of natural language conversational interaction within any application by utilizing application and user context as inputs to the LLM that generates the function calls. The use of application-defined function definitions makes it easier for developers to control the actions of the model, providing greater stability than other approaches such as direct UI control

JaxPruner: A concise library for sparsity research

This paper introduces JaxPruner, an open-source JAX-based pruning and sparse training library for machine learning research. JaxPruner aims to accelerate research on sparse neural networks by providing concise implementations of popular pruning and sparse training algorithms with minimal memory and latency overhead. Algorithms implemented in JaxPruner use a common API and work seamlessly with the popular optimization library Optax, which, in turn, enables easy integration with existing JAX based libraries. We demonstrate this ease of integration by providing examples in four different codebases: Scenic, t5x, Dopamine and FedJAX and provide baseline experiments on popular benchmarks.Comment: Jaxpruner is hosted at http://github.com/google-research/jaxprune