A virtual, intelligent, or computational assistant (e.g., also referred to simply as an “assistant”) is described that relies on supplemental data (e.g., contextual information, user information, etc.) to determine media, including movies, television shows, music, etc., that a user may want to consume and provide recommendations or automatically perform actions based on to assist the user in consuming the media. With explicit permission from a user, the assistant may access a user’s location history, calendar, e-mail, messages, past assistant interactions, contacts, photos, search history, sensor data, social network accounts, and other contextual or user information develop recommendations. The supplemental data can be stored locally on a device that is executing the assistant or in a cloud computing environment that is accessible to the assistant from the device. This way, the assistant can better understand what kinds of media a user may be interested in consuming and automatically make recommendations for, or take specific actions on, specific media that will satisfy the user’s interest, without requiring the user to search for, pre-record, or otherwise be aware that such media exists
