Voice interface responses based on prior user interactions

Voice interfaces, such as those provided by virtual assistant applications on smartphones, smart speakers, and other devices utilize natural language when providing responses to user queries. Per techniques of this disclosure, user interaction patterns are used to generate natural language responses to queries posed over a voice user interface. With user permission and express consent, data that pertain to prior user interaction, e.g., text entry such as search terms entered; other input operations such as clicks, taps, swipes, mouse hovers; viewports for the user, etc. are obtained. For ease of interaction, such permission is obtained, e.g., at initial setup, and is modifiable. Such data is analyzed to derive insight into context-based topics of interest to users. User preferences gathered across multiple contexts are utilized to generate natural language responses for voice-based user interaction