# US 2026/0099345 A1 · Interactive application widgets rendered with assistant content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2026/0099345 A1 - Document ID: US20260099345A1 - Status: Published application - Kind: A1 - Application number: 19/416,351 - Filed: 2025-12-11 - Published: 2026-04-09 - Earliest claimed: 2022-08-18 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Evgeny Cherepanov; Olga Kapralova; Dan Vallejo; Wendy Look; Mikhail Reutov - Web page: https://patentgrep.com/patent/US20260099345A1 ## CPC - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 9/453: {Help systems} - G06F 3/005: {Input arrangements through a video camera} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - H04N 21/47815: {Electronic shopping (payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12)} - H04N 21/482: End-user interface for programme selection - H04N 23/63: by using electronic viewfinders - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Implementations set forth herein relate to an automated assistant that can provide interactive application widgets based on their relevance to content that a user may have expressed interest in. The automated assistant can render the application widgets according to an estimated familiarity of the user with the content they expressed interest in. Each application widget can correspond to an application that can be accessed separately from the automated assistant. An application widget can be rendered at a display interface simultaneous to the user accessing the content that served as the basis for rendering the application widget. When the user interacts with the application widget, the automated assistant can communicate selection data to a corresponding application, which can respond with supplemental data that can be rendered at the display interface. ## Claims ### Claim 1 A method implemented by one or more processors, the method comprising: receiving, by an automated assistant, a spoken utterance of a user that embodies a request for the automated assistant to render particular content at a display interface of a computing device via which the automated assistant is accessible; determining, by the automated assistant, a relevancy metric that characterizes a degree of relevance of the particular content to one or more instances of application widget data that is accessible to the automated assistant, wherein the application widget data is provided by one or more applications that are accessible separately from the automated assistant; determining whether the relevancy metric satisfies a relevancy threshold for rendering one or more application widgets; and in response to determining that the relevancy metric satisfies the relevancy threshold for rendering one or more application widgets for the user: generating, based on the relevancy metric and the one or more instances of application widget data, interface data that characterizes one or more application widgets, wherein each application widget of the one or more application widgets corresponds to a respective application of the one or more applications; and causing, by the automated assistant and based on the interface data, a display interface of the computing device to render the one or more applications widgets, wherein a particular application widget of the one or more application widgets includes certain content provided by a particular application of the one or more applications. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the relevancy metric that characterizes the degree of relevance of the content to the one or more application widget data includes: processing content data, corresponding to the particular content, and a particular instance of application widget data using one or more trained machine learning models. ### Claim 3 (depends on 2) The method of claim 2, wherein the one or more trained machine learning models are trained using training data that is generated based on one or more prior interactions between the user and the automated assistant. ### Claim 4 (depends on 2) The method of claim 2, wherein the particular application includes a navigation application and the certain content rendered with the particular application widget includes an image of a map that includes a location associated with particular content requested by the user. ### Claim 5 (depends on 4) The method of claim 4, wherein the particular application widget is rendered simultaneously with an additional application widget corresponding to an additional application of the one or more applications. ### Claim 6 (depends on 1) The method of claim 1, wherein the particular application includes a navigation application and the certain content rendered with the particular application widget includes an image of a map that includes a location associated with particular content requested by the user. ### Claim 7 (depends on 1) The method of claim 1, wherein the particular application widget is rendered simultaneously with an additional application widget corresponding to an additional application of the one or more applications. ### Claim 8 (depends on 1) The method of claim 1, wherein an instance of the application widget data is generated by a corresponding application and characterizes interactive content that is available for the user to directly access via the corresponding application. ### Claim 9 (depends on 1) The method of claim 1, wherein each application widget of the one or more application widgets includes an interactive graphical user interface (GUI) element. ### Claim 10 (depends on 9) The method of claim 9, further comprising: receiving, by the automated assistant, a user selection of the interactive GUI element; and causing, in response to the user selection of the interactive GUI element, the automated assistant to communicate selection data to the particular application of the one or more applications. ### Claim 11 A computing device comprising: a display interface; memory storing instructions; and one or more processors operable to execute the instructions to: receive a spoken utterance of a user that embodies a request to render particular content at the display interface; determine a relevancy metric that characterizes a degree of relevance of the particular content to one or more instances of application widget data, wherein the application widget data is provided by one or more applications; determine whether the relevancy metric satisfies a relevancy threshold for rendering one or more application widgets; and in response to determining that the relevancy metric satisfies the relevancy threshold for rendering one or more application widgets for the user: generate, based on the relevancy metric and the one or more instances of application widget data, interface data that characterizes one or more application widgets, wherein each application widget of the one or more application widgets corresponds to a respective application of the one or more applications; and cause, based on the interface data, the display interface to render the one or more applications widgets, wherein a particular application widget of the one or more application widgets includes certain content provided by a particular application of the one or more applications. ### Claim 12 (depends on 11) The computing device of claim 11, wherein in determining the relevancy metric that characterizes the degree of relevance of the content to the one or more application widget data one or more of the processors are to: process content data, corresponding to the particular content, and a particular instance of application widget data using one or more trained machine learning models. ### Claim 13 (depends on 12) The computing device of claim 12, wherein the one or more trained machine learning models are trained using training data that is generated based on one or more prior interactions of the user. ### Claim 14 (depends on 12) The computing device of claim 12, wherein the particular application includes a navigation application and the certain content rendered with the particular application widget includes an image of a map that includes a location associated with particular content requested by the user. ### Claim 15 (depends on 14) The computing device of claim 14, wherein the particular application widget is rendered simultaneously with an additional application widget corresponding to an additional application of the one or more applications. ### Claim 16 (depends on 11) The computing device of claim 11, wherein the particular application includes a navigation application and the certain content rendered with the particular application widget includes an image of a map that includes a location associated with particular content requested by the user. ### Claim 17 (depends on 16) The computing device of claim 16, wherein the particular application widget is rendered simultaneously with an additional application widget corresponding to an additional application of the one or more applications. ### Claim 18 (depends on 11) The computing device of claim 11, wherein an instance of the application widget data is generated by a corresponding application and characterizes interactive content that is available for the user to directly access via the corresponding application. ### Claim 19 (depends on 11) The computing device of claim 11, wherein each application widget of the one or more application widgets includes an interactive graphical user interface (GUI) element. ### Claim 20 (depends on 19) The computing device of claim 19, wherein one or more of the processors are further operable to execute the instructions to: receive a user selection of the interactive GUI element; and cause, in response to the user selection of the interactive GUI element, communication of selection data to the particular application of the one or more applications.