# US 12,010,404 B2 · Dynamic user engagement and auxiliary search on streaming media content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,404 B2 - Document ID: US12010404B2 - Status: Granted - Kind: B2 - Application number: 17/955,419 - Filed: 2022-09-28 - Published: 2023-03-30 - Granted: 2024-06-11 - Earliest claimed: 2021-09-28 - Assignees: ARRIS Enterprises LLC - Applicants: ARRIS Enterprises LLC - Inventors: Kanakendiran Ramachandiran; Vinod Jatti - Web page: https://patentgrep.com/patent/US12010404B2 ## CPC - H04N 21/8545: for generating interactive applications - H04N 21/4722: for requesting additional data associated with the content - H04N 21/8133: {specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme} - H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} - H04N 21/858: Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The present disclosure describes techniques for performing auxiliary search on streaming media content. The techniques according to the present disclosure may comprise streaming media content to a display of a user and, receiving at least one query associated with a frame of the streamed media content. The techniques further comprise comparing the at least one query with metadata associated with the streamed media content to generate one or more results, and providing the one or more generated results to the display. ## Claims ### Claim 1 A method comprising: streaming media content to a display of a user; receiving at least one query associated with a frame of the streamed media content; comparing the at least one query with metadata associated with the streamed media content to generate one or more results; providing the one or more generated results to the display; wherein comparing the at least one query with metadata associated with the streamed media content comprises: applying natural language processing (NLP) technique and text analytics technique to the at least one query to determine one or more keywords present in the at least one query; comparing the one or more keywords with the metadata chunked with the media content; and generating the one or more results if the one or more keywords match the metadata chunked with the media content; and wherein in response to the one or more keywords not matching the metadata chunked with the media content, identifying at least one object in a frame associated with the at least one query using a computer vision technique; searching at least one database based on the at least one query and the at least one identified object; and generating the one or more results based on the search. ### Claim 2 (depends on 1) The method of claim 1, wherein the metadata associated with the streamed media content comprises at least one of: geographical coordinates associated with a location present in the frame of the streamed media content, name of the location present in the frame of the streamed media content, date and time of recording the streamed media content, details of one or more artists present or associated with the streamed media content, and information associated with one or more objects present in the streamed media content or any combinations thereof. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more results are generated based on a context of the streamed media content, and wherein the one or more results comprises at least one of one or more navigation links to different media contents associated with the context of the streamed media content, maps for navigating to a location present in the streamed media content, details of objects present in the frame, recommendations based the context of the streamed media content. ### Claim 4 (depends on 1) The method of claim 1, wherein the metadata is chunked along with the media content before transmitting to a media gateway for streaming the media content on the display of the user. ### Claim 5 (depends on 1) The method of claim 1, further comprising: in response to the one or more keywords not matching the metadata chunked with the media content, retrieving metadata that matches the one or more keywords from a metadata server; and generating the one or more results based on the retrieved metadata. ### Claim 6 (depends on 1) The method of claim 1, further comprising: in response to the one or more keywords not matching the metadata chunked with the media content, transmitting the at least one query to a metadata server; processing the at least one query by a metadata service administrator; updating existing repository of the metadata server to include details associated with the at least one query; and notifying the user with the details associated with the at least one query. ### Claim 7 (depends on 1) The method of claim 1, further comprising: receiving a selection of at least one result among the one or more generated results; and navigating the user to another media content based on the selected at least one result. ### Claim 8 A system comprising: a memory; a display unit in communication with the memory and configured to stream media content to a display; and a user interface in communication with the display unit and the memory, and configured to receive at least one query associated with a frame of the streamed media content; and one or more processors in communication with the display unit, the user interface, and memory, wherein the one or more processors are configured to provide one or more generated results on the display of the user, the one or more generated results produced by comparing the at least one query with metadata associated with the streamed media content; wherein comparing the at least one query with metadata associated with the streamed media content comprises: applying natural language processing (NLP) technique and text analytics technique to the at least one query to determine one or more keywords present in the at least one query; comparing the one or more keywords with the metadata chunked with the media content; and generating the one or more results if the one or more keywords match the metadata chunked with the media content; and wherein in response to the one or more keywords not matching the metadata chunked with the media content, identifying at least one object in a frame associated with the at least one query using a computer vision technique; searching at least one database based on the at least one query and the at least one identified object; and generating the one or more results based on the search. ### Claim 9 (depends on 8) The system of claim 8, wherein the metadata associated with the streamed media content comprises at least one of: geographical coordinates associated with a location present in the frame of the streamed media content, name of the location present in the frame of the streamed media content, date and time of recording the streamed media content, details of one or more artists present or associated with the streamed media content, and information associated with one or more objects present in the streamed media content or any combinations thereof. ### Claim 10 (depends on 8) The system of claim 8, wherein the one or more results are generated based on a context of the streamed media content, and wherein the one or more results comprises at least one of one or more navigation links to different media contents associated with the context of the streamed media content, maps for navigating to a location present in the streamed media content, details of objects present in the frame, recommendations based the context of the streamed media content. ### Claim 11 (depends on 8) The system of claim 8, wherein the metadata is chunked along with the media content before transmitting to a media gateway for streaming the media content on the display of the user. ### Claim 12 (depends on 8) The system of claim 8, wherein the one or more processors are further configured to: retrieve metadata that matches the one or more keywords from a metadata server in response to the one or more keywords not matching the metadata chunked with the media content; and generate the one or more results based on the retrieved metadata. ### Claim 13 (depends on 8) The system of claim 8, further comprising: a transceiver in communication with the one or more processors, wherein the transceiver is configured to: transmit the at least one query to a metadata server; and wherein: the metadata server comprises a metadata service administrator for processing the at least one query, updating existing repository of the metadata server to include details associated with the at least one query, and notifying the user with the details associated with the at least one query. ### Claim 14 (depends on 8) The system of claim 8, wherein the one or more processors are further configured to: receive, via the user interface, a selection of at least one result among the one or more generated results; and navigate the user to another media content based on the selected at least one result.