# US 12,118,028 B2 · Coordination of parallel processing of audio queries across multiple devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,028 B2 - Document ID: US12118028B2 - Status: Granted - Kind: B2 - Application number: 18/376,712 - Filed: 2023-10-04 - Published: 2024-01-25 - Granted: 2024-10-15 - Earliest claimed: 2021-10-11 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Jian Wei Leong; Nicu Cornea; Maya Tekke Anant - Web page: https://patentgrep.com/patent/US12118028B2 ## CPC - G06F 16/433: {using audio data} - G06F 16/90339: {by using parallel associative memories or content-addressable memories} - G06F 9/3885: {using a plurality of independent parallel functional units} - G06F 16/2379: {Updates performed during online database operations; commit processing} - G06F 16/243: {Natural language query formulation} - G06F 16/252: {between a Database Management System and a front-end application} - G06F 16/60: of audio data - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The present disclosure is generally related to a data processing system to coordinate parallel processing of audio queries across multiple devices. A data processing system can receive an audio input signal detected the display device and parse the audio input signal to identify an entity. The data processing system can transmit a query command to the display device to cause a multimedia content application to perform a search for the entity. The data processing system can access at least one of an address database and a multimedia content provider to identify a reference address for the entity. The data processing system can provide the reference address for the entity to cause the display device to present a content selection interface. The content selection interface can include an element for the reference address, prior to completion of the search for the entity performed by the multimedia content application. ## Claims ### Claim 1 A method implemented by one or more processors, the method comprising: receiving, from a remote data processing system, a query command that includes an entity; in response to receiving the query command, causing a multimedia content application, executing on a display device, to perform a search for the entity; receiving, from the remote data processing system, a search result corresponding to the entity, wherein the search result corresponding to the entity is identified by the remote data processing system; and in response to receiving the search result from the remote data processing system, presenting, on a display of the display device and prior to completion of the search for the entity performed by the multimedia content application, a content selection interface, the content selection interface including a selection element for the search result from the remote data processing system and a placeholder element for a search result from the multimedia content application. ### Claim 2 (depends on 1) The method according to claim 1, wherein the search result from the remote data processing system comprises a reference address identified by the remote data processing system using an address database. ### Claim 3 (depends on 1) The method according to claim 1, wherein the search result from the remote data processing system comprises a reference address identified by the remote data processing system using a multimedia content provider. ### Claim 4 (depends on 1) The method according to claim 1, further comprising, in response to the completion of the search for the entity performed by the multimedia content application, replacing the placeholder element with an additional selection element on the content selection interface, the additional selection element being for a search result from the multimedia content application, the search result from the multimedia content application corresponding to the entity. ### Claim 5 (depends on 4) The method according to claim 4, wherein replacing the placeholder element is further in response to an interaction with the placeholder element. ### Claim 6 (depends on 4) The method according to claim 4, wherein the search result from the multimedia content application comprises a reference address provided by the multimedia content application. ### Claim 7 (depends on 1) The method according to claim 1, wherein the display device includes a smart television. ### Claim 8 A computer program product comprising one or more non-transitory computer-readable storage media having program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to: receive, from a remote data processing system, a query command that includes an entity; in response to receiving the query command, cause a multimedia content application, executing on a display device, to perform a search for the entity; receive, from the remote data processing system, a search result corresponding to the entity, wherein the search result corresponding to the entity is identified by the remote data processing system; and in response to receiving the search result from the remote data processing system, present, on a display of the display device and prior to completion of the search for the entity performed by the multimedia content application, a content selection interface, the content selection interface including a selection element for the search result from the remote data processing system and a placeholder element for a search result from the multimedia content application. ### Claim 9 (depends on 8) The computer program product according to claim 8, wherein the search result from the remote data processing system comprises a reference address identified by the remote data processing system using an address database. ### Claim 10 (depends on 8) The computer program product according to claim 8, wherein the search result from the remote data processing system comprises a reference address identified by the remote data processing system using a multimedia content provider. ### Claim 11 (depends on 8) The computer program product according to claim 8, wherein the program instructions are further executable to, in response to the completion of the search for the entity performed by the multimedia content application, replace the placeholder element with an additional selection element on the content selection interface, the additional selection element being for a search result from the multimedia content application, the search result from the multimedia content application corresponding to the entity. ### Claim 12 (depends on 11) The computer program product according to claim 11, wherein replacing the placeholder element is further in response to an interaction with the placeholder element. ### Claim 13 (depends on 11) The computer program product according to claim 11, wherein the search result from the multimedia content application comprises a reference address provided by the multimedia content application. ### Claim 14 (depends on 8) The computer program product according to claim 8, wherein the display device includes a smart television. ### Claim 15 A system comprising: a processor, a computer-readable memory, one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to: receive, from a remote data processing system, a query command that includes an entity; in response to receiving the query command, cause a multimedia content application, executing on a display device, to perform a search for the entity; receive, from the remote data processing system, a search result corresponding to the entity, wherein the search result corresponding to the entity is identified by the remote data processing system; and in response to receiving the search result from the remote data processing system, present, on a display of the display device and prior to completion of the search for the entity performed by the multimedia content application, a content selection interface, the content selection interface including a selection element for the search result from the remote data processing system and a placeholder element for a search result from the multimedia content application. ### Claim 16 (depends on 15) The system according to claim 15, wherein the search result from the remote data processing system comprises a reference address identified by the remote data processing system using an address database. ### Claim 17 (depends on 15) The system according to claim 15, wherein the search result from the remote data processing system comprises a reference address identified by the remote data processing system using a multimedia content provider. ### Claim 18 (depends on 15) The system according to claim 15, wherein the program instructions are further executable to, in response to the completion of the search for the entity performed by the multimedia content application, replace the placeholder element with an additional selection element on the content selection interface, the additional selection element being for a search result from the multimedia content application, the search result from the multimedia content application corresponding to the entity. ### Claim 19 (depends on 18) The system according to claim 18, wherein replacing the placeholder element is further in response to an interaction with the placeholder element. ### Claim 20 (depends on 18) The system according to claim 18, wherein the search result from the multimedia content application comprises a reference address provided by the multimedia content application.