# US 2021/0240756 A1 · Methods, systems, and media for processing queries relating to presented media content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240756 A1
- Document ID: US20210240756A1
- Status: Published application
- Kind: A1
- Application number: 17/234,084
- Filed: 2021-04-19
- Published: 2021-08-05
- Earliest claimed: 2015-04-14
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Alexander Taboriskiy; Emmanuel Mogenet; Oliver Heckmann; Matsvei Zhdanovich; Gokhan Hasan Bakir; Behshad Behzadi; Karoly Csalogany
- Web page: https://patentgrep.com/patent/US20210240756A1

## CPC

- G06F 16/438: Presentation of query results
- G06F 16/433: {using audio data}
- G06F 16/48: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/4722: for requesting additional data associated with the content
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods, systems, and media for processing queries relating to presented media content are provided. In some implementations, a method comprises: receiving a request to associate with a media playback device that is presenting media content to a user of the mobile device, wherein a mobile application executing on the mobile device and a media application executing on the media playback device exchange media playback information; activating a microphone associated with the mobile device to receive ambient sounds in response to associating with the media playback device; converting the received ambient sounds to one or more text inputs; determining whether the text inputs include a trigger term that corresponds to a request to initiate a query relating to the presented media content and the query; in response to determining that the trigger term has been included in the text inputs, determining the media playback information from the media application that includes timing information corresponding to when during the presentation of the media content the query was received and media content identification information; causing a search to be performed that includes the query, the timing information, and the media content identification information; obtaining a search result that is responsive to the query; and presenting at least a portion of the search result to the query on a mobile display associated with the mobile device.

## Claims

### Claim 1

A method for providing media guidance, the method comprising:
receiving, using a computing device, a plurality of input terms;
determining whether the plurality of input terms include a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented;
in response to determining that the trigger term has been included in the plurality of input terms, determining media playback information that includes timing information corresponding to a time at which the query was received during the presentation of the media content and media content identification information, wherein the timing information includes a time at which a last input term in the query was received;
causing a search to be performed that includes the query, the timing information, and the media content identification information, wherein the query includes at least a portion of the plurality of input terms other than the trigger term;
obtaining a search result that is responsive to the query; and
presenting at least a portion of the search result that is responsive to the query on a display associated with the computing device.

### Claim 2 (depends on 1)

The method of claim 1, wherein the at least a portion of the search result is determined to be presented on the display of the computing device based on search popularity information associated with the search result.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
retrieving the media playback information prior to receiving the query; and
associating the media playback information with the query in response to receiving the trigger term.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
receiving a request to associate with a media playback device, wherein the media playback device is presenting media content and wherein the media playback device is executing a media application that exchanges the media playback information with a mobile application executing on the computing device;
associating the computing device with the media playback device; and
activating an audio input device associated with the computing device to receive ambient sounds in response to associating with the media playback device, wherein the received ambient sounds are converted to the plurality of input terms.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
determining whether the audio input device associated with the computing device is capable of receiving the ambient sounds; and
activating an audio capturing device associated with the media playback device in response to determining that the audio input device associated with the computing device is not capable of receiving the ambient sounds.

### Claim 6 (depends on 4)

The method of claim 4, further comprising extracting the trigger term and the query from the plurality of input terms comprises processing the ambient sounds using a speech recognition system.

### Claim 7 (depends on 4)

The method of claim 4, further comprising:
requesting additional media playback information from the media application executing on the media playback device; and
transmitting the query, the media playback information, and the additional media playback information to a search system to receive the search result.

### Claim 8 (depends on 4)

The method of claim 4, further comprising:
determining whether the search result is to be presented on the display associated with the computing device; and
presenting at least a second portion of the search result on a display associated with the media playback device in response to determining that the at least the second portion of the search result is not to be presented on the display associated with the computing device.

### Claim 9 (depends on 4)

The method of claim 4, further comprising:
receiving requests from a plurality of computing devices to associate with the media playback device that is presenting the media content;
activating audio capturing devices associated with at least a portion of the plurality of computing devices and the media playback device to receive ambient sounds from users proximate to the plurality of computing devices and the media playback device;
determining that a plurality of queries have been received based on the received ambient sounds, wherein each of the plurality of queries includes the trigger term and the query relating to the presented media content;
obtaining a plurality of search results, wherein each of the plurality of search results is responsive to one of the plurality of queries; and
causing the plurality of search results to be presented.

### Claim 10 (depends on 9)

The method of claim 9, further comprising:
associating at least one search result of the plurality of search results with the respective computing device from which the ambient sounds including the query was received; and
causing at least a portion of the search result to be presented on the respective computing device.

### Claim 11

A system for providing media guidance, the system comprising:
a hardware processor that is configured to:
receive a plurality of input terms;
determine whether the plurality of input terms include a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented;
in response to determining that the trigger term has been included in the plurality of input terms, determine media playback information that includes timing information corresponding to a time at which the query was received during the presentation of the media content and media content identification information, wherein the timing information includes a time at which a last input term in the query was received;
cause a search to be performed that includes the query, the timing information, and the media content identification information, wherein the query includes at least a portion of the plurality of input terms other than the trigger term;
obtain a search result that is responsive to the query; and
present at least a portion of the search result to the query on a display associated with the computing device.

### Claim 12 (depends on 11)

The system of claim 11, wherein the at least a portion of the search result is determined to be presented on the display of the computing device based on search popularity information associated with the search result.

### Claim 13 (depends on 11)

The system of claim 11, wherein the hardware processor is further configured to:
retrieve the media playback information prior to receiving the query; and
associate the media playback information with the query in response to receiving the trigger term.

### Claim 14 (depends on 11)

The system of claim 11, wherein the hardware processor is further configured to:
receive a request to associate with a media playback device, wherein the media playback device is presenting media content and wherein the media playback device is executing a media application that exchanges the media playback information with a mobile application executing on the computing device;
associate the computing device with the media playback device; and
activate an audio input device associated with the computing device to receive ambient sounds in response to associating with the media playback device, wherein the received ambient sounds are converted to the plurality of input terms.

### Claim 15 (depends on 14)

The system of claim 14, wherein the hardware processor is further configured to:
determine whether the audio input device associated with the computing device is capable of receiving the ambient sounds; and
activate an audio capturing device associated with the media playback device in response to determining that the audio input device associated with the computing device is not capable of receiving the ambient sounds.

### Claim 16 (depends on 14)

The system of claim 14, wherein the hardware processor is further configured to extract the trigger term and the query from the plurality of input terms comprises processing the ambient sounds using a speech recognition system.

### Claim 17 (depends on 14)

The system of claim 14, wherein the hardware processor is further configured to:
request additional media playback information from the media application executing on the media playback device; and
transmit the query, the media playback information, and the additional media playback information to a search system to receive the search result.

### Claim 18 (depends on 14)

The system of claim 14, wherein the hardware processor is further configured to:
determine whether the search result is to be presented on the display associated with the computing device; and
present at least a second portion of the search result on a display associated with the media playback device in response to determining that the at least the second portion of the search result is not to be presented on the display associated with the computing device.

### Claim 19 (depends on 14)

The system of claim 14, wherein the hardware processor is further configured to:
receive requests from a plurality of computing devices to associate with the media playback device that is presenting the media content;
activate audio capturing devices associated with at least a portion of the plurality of computing devices and the media playback device to receive ambient sounds from users proximate to the plurality of computing devices and the media playback device;
determine that a plurality of queries have been received based on the received ambient sounds, wherein each of the plurality of queries includes the trigger term and the query relating to the presented media content;
obtain a plurality of search results, wherein each of the plurality of search results is responsive to one of the plurality of queries; and
cause the plurality of search results to be presented.

### Claim 20 (depends on 19)

The system of claim 19, wherein the hardware processor is further configured to:
associate at least one search result of the plurality of search results with the respective computing device from which the ambient sounds including the query was received; and
cause at least a portion of the search result to be presented on the respective computing device.

### Claim 21

A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform method for providing media guidance, the method comprising:
receiving a plurality of input terms;
determining whether the plurality of input terms include a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented;
in response to determining that the trigger term has been included in the plurality of input terms, determining media playback information that includes timing information corresponding to a time at which the query was received during the presentation of the media content and media content identification information, wherein the timing information includes a time at which a last input term in the query was received;
causing a search to be performed that includes the query, the timing information, and the media content identification information, wherein the query includes at least a portion of the plurality of input terms other than the trigger term;
obtaining a search result that is responsive to the query; and
presenting at least a portion of the search result to the query on a display associated with the computing device.
