Skip Button for Uninteresting Irrelevant Content Segments

Many audio and video playback applications include functionality to skip over parts of the content being played. However, such controls skip predetermined duration of content and can inadvertently lead users to skip over content portions of interest. This disclosure describes techniques that automatically determine locations and lengths of uninteresting or non-information segments in content based on user interests and context, obtained with permission. Users are provided with a suitable user interface (UI) mechanism to skip over to the next moment of interest within the content. If the user permits, data on the portions that were skipped can be aggregated and made available to content creators to help them understand user engagement with their content

Managing Devices in a Device Ecosystem by Detecting Eye Gaze, Gestures, and User Medical Diagnostics

A network of interconnected user devices and peripherals is used together in such a way that eye tracking, gestures, and medical diagnostics detected by one or more devices in the device ecosystem are used to control, manage, and interact with other devices in the ecosystem

Improving Online Presentations based on Audience Usage of Conversational Assistant

During a presentation made via video conference, attendees may use a conversational assistant within the video conference to seek clarifications for terms or concepts that they don’t understand. Speakers that are unaware of such audience activity cannot adjust their presentation to clarify presentation content. This disclosure describes techniques that enable a presenter to receive live feedback regarding content of their presentation that may not be understood (or may be misunderstood) by the audience and provide clarifications in real time. During the presentation, a presenter\u27s interface is augmented with indications that a portion of the audience is having difficulty understanding certain portions of the presentation, e.g., struggling with certain acronyms or terms used in the presentation. Detection of such activity may be based on the use of a conversational assistant by the attendees. The presenter can then choose to verbally clarify or directly substitute or expand the confusing term on the slide (e.g., with automatic suggestions from a virtual assistant) with more comprehensible text

Providing Cues Indicating Device Component Location for Improved User Experience

Devices such as smartphones include many input and output components located in different parts of the device. When the device is used for particular purposes, such as capturing images, recording audio, etc., issues such as misalignment between the view on the display and the actual object (due to mismatch of camera location and device location), muffled audio in the recording (due to obstructed microphone), etc. can arise. Such issues can be difficult to detect and fix because users are typically unaware of the locations of these various input and output components. This disclosure describes techniques to automatically detect inadvertent issues with input and/or output and to alert the user via audio and/or visual cues to take mitigating action. The cues can indicate the location of the component and the action to be performed

Generative AI Driven Content Quality Enhancement for User-generated Content

Online content posted by amateur content creators may not be of sufficient quality to drive engagement. For example, such content may have audio or video glitches. Viewer feedback can indicate the problems with user-generated content. This disclosure describes techniques that analyze viewer comments and/or the user-generated content to generate prompts for a generative AI model. With permission from content creators, the capability of generative artificial intelligence models to execute audio-enhancement tasks such as volume equalization, noise reduction, audio filtering, etc. and video-related tasks such as video smoothing, video stabilization, etc.is utilized via the generated prompts to automatically improve their content. The content creator has control over the versions generated prior to release and can specify the types of edits as well as the portions of content that are modified. Automatic editing of content can help creators that are unskilled or lack access to advanced tools to improve the quality of their content and can improve the viewer experience of platforms that host such content

Automatically Highlighting Browser Tabs Related to Current Tab

Users that have a large number of open browser tabs can struggle to identify tabs that are relevant to their current task. While tab search functionality is available in some browsers, such functionality does not automatically identify relevant tabs. This disclosure describes a web browser that, with user permission, determines the user’s current context based on the currently active browser tab and automatically highlights other relevant tabs that are currently open. The content and metadata of browser tabs are analyzed, e.g., using on-device machine learning techniques, to identify the relevant tabs. The relevant tabs can be highlighted with different colors and/or color gradation to indicate relative match to the current tab. The techniques are applicable for any application tabbed or stacked user interface

SAFER AUDIO CONSUMPTION WHILE DRIVING

A computing device (e.g., a smartphone, a laptop computer, a tablet computer, a smartwatch, etc.) may monitor audio streams and adjust non-navigational audio streams based on a detected real-life context. The computing device may evaluate the real-life context and identify situations requiring reduced distractions. The computing device may determine, based on the evaluation of the audio streams, whether a user may be focused on an activity and should not be distracted from potential hazards. The computing device may additionally evaluate the audio streams and identify navigational and non-navigational streams. Based on the identified real-world context and audio streams, the computing device may pause non-navigational audio and censor potentially distracting sounds and words. The computing device may pause or modify audio streams while a user of the computing device is engaged in any number of activities such as running, walking, driving, biking, etc. as well as alert a user to potential hazards. For example, a driver may approach a section of road with multiple tight turns while distracting audio streams like music or podcasts are playing. The computing device may improve safety by pausing the distracting audio streams. In another example, an assistant on the computing device is reading a book to a listener while the listener is approaching a sidewalk. The computing device may pause reading the book so the listener can hear vehicles approaching the sidewalk. The computing device may alert the listener of the oncoming traffic in addition to pausing the audio

Contextual User Interface that Reflects Stages of Travel in Multi-hop Public Transport

Travelers on a public transport system such as metro rail are often engrossed in viewing content on their mobile devices and may forget the route segment they are currently on and can miss an upcoming transfer point, leading to time-consuming re-routing. This disclosure describes techniques that provide, via a mobile device and/or a wearable computer, subtle visual hints to a traveler regarding their current location on a multi-hop public transport system such as metro rail and upcoming transfers. The visual hints can be provided by matching the color of operating system controlled user interface elements such as wallpaper, status indicators, notification panels, etc. or via application UI. Application behavior can also be updated to account for the traveler’s present location and travel route, e.g., by displaying search results that match the route, displaying search results with a similar color as a route segment that it is close to, etc

Context-based Selective Content Co-Consumption Experience in a Smart Home

Interactive digital displays in a user’s home can be configured to showcase photos, videos, and other content from the user’s personal library. However, display of such content may be unsuitable when strangers visit the home. This disclosure describes techniques to detect the likely presence of a stranger and/or a known individual within a home and to configure smart devices accordingly to restrict display of personal information, and to generate and show content relevant to the known person. The techniques can provide a customized content consumption experience that is aware of the user’s context. The selected content can be displayed via a digital display or playback device such as a smart display, smart speaker, or other devices within the home. The described techniques preserve the convenience of personalized device configurations within a home, and enhance the experience of shared media viewing, while preserving user privacy when strangers visit. The techniques can be implemented within a smart home ecosystem and can be configured/ controlled via a smart home management application

Systems and Methods for Contextually Relevant Search Results

The present disclosure describes computer-implemented systems and methods for generating image search results pertaining to both the objects within an image as well as state of the objects. A user may provide an image to an image search system. The image search system may determine an imperfect object within the image and generate search results tailored toward the object being damaged or malformed. Search results may include, but are not limited to, replacement parts, third party repair services, how-to repair videos, or similar