# US 2018/0324115 A1 · Initializing a conversation with an automated agent via selectable graphical element > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0324115 A1 - Document ID: US20180324115A1 - Status: Published application - Kind: A1 - Application number: 15/589,645 - Filed: 2017-05-08 - Published: 2018-11-08 - Earliest claimed: 2017-05-08 - Assignees: Google Inc. - Applicants: Google Inc. - Inventors: Vikram Aggarwal; Dina Elhaddad - Web page: https://patentgrep.com/patent/US20180324115A1 ## CPC - H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages - G06F 9/06: using stored programs, i.e. using an internal store of processing equipment to receive or retain programs - G06F 9/453: {Help systems} - G06F 16/3329: Natural language query formulation - G06F 40/10: Text processing (natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40) - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - H04L 51/046: Interoperability with other network applications or services - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Methods, apparatus, systems, and computer-readable media are provided for invoking an agent module in an automated assistant application in response to user selection of a selectable element presented at a graphical user interface rendered by a non-automated assistant application. The invoked agent module can be associated with other content rendered in the non-automated assistant graphical user interface, and can optionally be invoked with values that are based on user interactions via the non-automated assistant application. Responsive content can be received from the agent module in response to the invocation, and corresponding content provided by the automated assistant application via an automated assistant interface. In these and other manners, selection of the selectable element causes transition from a non-conversational interface, to a conversational automated assistant interface—where an agent (relevant to content in the non-conversational interface) is invoked in the automated assistant interface. ## Claims ### Claim 1 A method implemented by one or more processors, comprising: receiving a selection of a selectable element at a graphical user interface rendered by a non-automated assistant application of a computing device, wherein the selectable element indicates an agent associated with the graphical user interface can be invoked via an automated assistant application that is separate from the non-automated assistant application; in response to the selection of the selectable element, invoking the agent via the automated assistant application, the agent being one of a plurality of available agents that can be invoked via the automated assistant application; receiving responsive content from the agent in response to invoking the agent; and providing, by the automated assistant application via an automated assistant interface, output that is based on the responsive content received from the agent. ### Claim 2 (depends on 1) The method of claim 1, wherein receiving the selection of the selectable element comprises: receiving agent content that identifies the agent, and receiving a modality parameter that indicates a modality of dialog with the automated assistant, wherein invoking the agent comprises invoking the agent with the modality of dialog indicated by the modality parameter, and the output is provided in the modality of dialog indicated by the modality parameter. ### Claim 3 (depends on 2) The method of claim 2, wherein the modality is one of voice or text. ### Claim 4 (depends on 1) The method of claim 1, wherein receiving the selection of the selectable element comprises receiving a link that is unique to the agent, wherein invoking the agent is based on the link and is contingent on the automated assistant application being installed on the computing device. ### Claim 5 (depends on 4) The method of claim 4, wherein, when the automated assistant application is not installed on the computing device, the link causes display of further content related to the agent, without invocation of the agent. ### Claim 6 (depends on 1) The method of claim 1, wherein receiving the selection of the selectable element comprises: receiving agent content that indicates the agent and an intent for the agent; and wherein invoking the agent comprises invoking the agent with the intent indicated by the agent content. ### Claim 7 (depends on 6) The method of claim 6, wherein the agent content further indicates intent parameters for the intent, and invoking the agent further comprises invoking the agent with the intent parameters indicated by the agent content. ### Claim 8 (depends on 7) The method of claim 7, wherein the intent parameters are dynamically generated through interaction with the non-automated assistant application. ### Claim 9 (depends on 1) The method of claim 1, wherein the agent is a third-party agent that is controlled by a third-party that is unique from a party that controls the automated assistant application. ### Claim 10 (depends on 1) The method of claim 1, wherein the automated assistant application operates at a separate computing device from the non-automated assistant application and the output is provided by the separate computing device. ### Claim 11 (depends on 1) The method of claim 1, wherein the output that is based on the responsive content received from the selected agent is a prompt, and the method further comprises: receiving natural language input in response to the prompt; providing, to the agent, further content that is based on the natural language input; receiving further responsive content from the agent in response to providing the further content; and providing, by the automated assistant application via the automated assistant interface, output that is based on the further responsive content received from the selected agent. ### Claim at position 12 12-20. (canceled) ### Claim 21 A method implemented by one or more processors, comprising: causing a selectable element to be displayed at a computing device that is operating a non-automated assistant application, the selectable element configured to cause an automated assistant to initialize an agent module associated with the non-automated assistant application; receiving a selection of the selectable element; in response to receiving the selection of the selectable element: determining whether the automated assistant is accessible to the computing device; and when it is determined that the automated assistant is inaccessible to the computing device: executing a link corresponding to the selectable element to cause a default webpage to open and present speakable command phrases for communicating with the agent module. ### Claim 22 (depends on 21) The method of claim 21, further comprising: when it is determined that the automated assistant is accessible to the computing device: initializing the automated assistant for communicating with the agent module. ### Claim 23 (depends on 21) The method of claim 21, wherein the link identifies the agent module and an action to be performed by the agent module. ### Claim 24 (depends on 23) The method of claim 23, wherein the speakable command phrases are based on the action identified in the link. ### Claim 25 (depends on 24) The method of claim 24, further comprising: when it is determined that the automated assistant is accessible to the computing device: transmitting an invocation request to the agent module; receiving responsive content from the agent module in response to the invocation request; and providing, through an interface associated with the automated assistant, output that is based on the responsive content. ### Claim 26 (depends on 21) The method of claim 21, further comprising: sending a signal over a network to determine whether the automated assistant is available at a separate computing device on the network. ### Claim 27 A non-transitory computer readable medium configured to store instructions that, when executed by one or more processors, cause the one or more processors to perform steps that include: determining that a user is viewing an application interface of a non-automated assistant application, wherein the application interface includes a first selectable element for initializing communications with an agent module via an automated assistant application, the agent module configured to perform an action associated with the non-automated assistant application; receiving a selection of the first selectable element, wherein the first selectable element includes a link that identifies the agent module and a parameter for performing the action; causing a conversational interface to be presented to the user, wherein the conversational interface is configured by the automated assistant to be an intermediary between the user and the agent module; and providing a second selectable element at the conversational interface, wherein the second selectable element is based on the parameter identified in the link to further the action. ### Claim 28 (depends on 27) The non-transitory computer readable medium of claim 27, wherein the steps further include: receiving a selection of the second selectable element at the conversational interface; and providing data to the agent module for fulfilling the parameter identified in the link. ### Claim 29 (depends on 27) The non-transitory computer readable medium of claim 27, wherein the second selectable element includes a speakable command phrase associated with a previous action performed by the user via the non-automated assistant application. ### Claim 30 (depends on 29) The non-transitory computer readable medium of claim 29, wherein the steps further include: receiving audio data corresponding to the speakable command phrase; and converting the audio data into textual command data for transmitting to the agent module.