# US 2014/0193027 A1 · Search and identification of video content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0193027 A1 - Document ID: US20140193027A1 - Status: Published application - Kind: A1 - Application number: 13/839,782 - Filed: 2013-03-15 - Published: 2014-07-10 - Earliest claimed: 2013-01-07 - Assignees: Gracenote, Inc. - Inventors: Steven D. Scherf; Wilson Harron - Web page: https://patentgrep.com/patent/US20140193027A1 ## CPC - G06F 16/783: using metadata automatically derived from the content - G06F 16/71: Indexing; Data structures therefor; Storage structures - G06F 16/951: Indexing; Web crawling techniques - G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods and systems for searching for and retrieving video content via fingerprint matching are described. In some example embodiments, the methods and systems access an index of reference fingerprints, such as quantized values associated with multiple values calculated from patches of a frame, that are associated with sequential frames of known video content. The methods and systems may receive a search query that includes a group of query fingerprints associated with sequential frames of video content presented at a client device, access the index, compare the group of query fingerprints to reference fingerprints of the index, determine that the group of query fingerprints match the reference fingerprints of the index, and identify the video content presented at the client device based on the determined match. ## Claims ### Claim 1 A method, comprising: receiving a search query that includes a group of query fingerprints associated with sequential frames of video content presented at a client device; accessing an index of reference fingerprints that are associated with sequential frames of known video content; comparing the group of query fingerprints to reference fingerprints of the index; determining that the group of query fingerprints match the reference fingerprints of the index; and identifying the video content presented at the client device based on the determined match. ### Claim 2 (depends on 1) The method of claim 1, wherein comparing the group of query fingerprints to the reference fingerprints of the index includes: identifying a best match reference fingerprint that best matches a first query fingerprint from the received search query; comparing a next query fingerprint after the first query fingerprint of the group of query fingerprints from the received search query to a next reference fingerprint after the best match reference fingerprint; determining that the group of query fingerprints match the reference fingerprints of the generated index based on the comparison between the next query fingerprint and the next reference fingerprint. ### Claim 3 (depends on 1) The method of claim 1, wherein determining that the group of query fingerprints match the reference fingerprints of the index includes: identifying two or more match candidates that satisfy the received search query based on the comparison; and applying a selection heuristic to the identified two or more match candidates; and selecting one or more of the identified match candidates based on the applied selection heuristic. ### Claim 4 (depends on 1) The method of claim 1, wherein determining that the group of query fingerprints match the reference fingerprints of the index includes: identifying two or more match candidates based on a comparison of the group of query fingerprints to values associated with the reference fingerprints that are calculated from one or more large regions of the frames of known video content; and selecting one or more of the identified match candidates based on a comparison of the group of query fingerprints to values associated with the identified match candidates that are calculated from one or more small regions of the frames of known video content. ### Claim 5 (depends on 1) The method of claim 1, wherein accessing an index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of quantized values for each of the sequential frames of known video content. ### Claim 6 (depends on 1) The method of claim 1, wherein accessing an index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of 32 bit vectors associated with one or more different patches of a frame of video content. ### Claim 7 (depends on 1) The method of claim 1, wherein accessing an index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of integers associated with frames of video content, an integer being calculated by hashing three to five different Haar-like features of the frame of video content. ### Claim 8 (depends on 1) The method of claim 1, wherein accessing an index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of integers associated with a frame of video content, an integer being calculated by hashing a first four Haar-like features of the frame of video content. ### Claim 9 (depends on 1) The method of claim 1, wherein accessing an index of reference fingerprints includes accessing an index of reference fingerprints associated with frames of video content at a first frame rate; wherein receiving a search query includes a receiving a group of query fingerprints presented at a client device having a second frame rate faster than the first frame rate; and wherein comparing the group of query fingerprints to the reference fingerprints of the index includes comparing a portion of query fingerprints having the second frame rate to the reference fingerprints associated with frames of video content at the first frame rate. ### Claim 10 A system, comprising: an index module that is configured to receive a search query that includes a group of query fingerprints associated with sequential frames of video content presented at a client device, and query an index of reference fingerprints that are associated with sequential frames of known video content; a fingerprint match module that is configured to determine that the group of query fingerprints match the reference fingerprints of the index; and an identification module that is configured to identify the video content presented at the client device based on the determined match. ### Claim 11 (depends on 10) The system of claim 10, wherein the index module is configured to query an index of quantized values for each of the sequential frames of known video content. ### Claim 12 (depends on 10) The system of claim 10, wherein the index module is configured to query an index of 32 bit vectors associated with 32 different patches of a frame of video content. ### Claim 13 (depends on 10) The system of claim 10, wherein the index module is configured to query an index of integers associated with frames of video content, an integer being calculated by hashing a first four Haar-like features of the frame of video content. ### Claim 14 (depends on 10) The system of claim 10, wherein the index module is configured to query an index of integers associated with frames of video content, an integer being calculated by hashing three to five different Haar-like features of the frame of video content. ### Claim 15 A computer-readable storage medium whose contents, when executed by a computing system, cause the computing system to perform operations, comprising: receiving a search query that includes a group of query fingerprints associated with sequential frames of video content presented at a client device; accessing an index of reference fingerprints that are associated with sequential frames of known video content; comparing the group of query fingerprints to reference fingerprints of the index; determining that the group of query fingerprints match the reference fingerprints of the index; and identifying the video content presented at the client device based on the determined match. ### Claim 16 (depends on 15) The computer-readable storage medium of claim 15, wherein determining that the group of query fingerprints match the reference fingerprints of the index includes: identifying two or more match candidates that satisfy the received search query based on the comparison; and applying a selection heuristic to the identified two or more match candidates; and selecting one or more of the identified match candidates based on the applied selection heuristic. ### Claim 17 (depends on 15) The computer-readable storage medium of claim 15, wherein determining that the group of query fingerprints match the reference fingerprints of the index includes: identifying two or more match candidates that satisfy the received search query based on the comparison; and determining a match quality between the identified two or more match candidates and the group of query fingerprints; and selecting one or more of the identified match candidates based on the determined match quality. ### Claim 18 (depends on 15) The computer-readable storage medium of claim 15, wherein determining that the group of query fingerprints match the reference fingerprints of the index includes: identifying two or more match candidates that satisfy the received search query based on the comparison; and determining a number of matches between the identified two or more match candidates and the group of query fingerprints; and selecting one or more of the identified match candidates based on the determined number of matches. ### Claim 19 (depends on 15) The computer-readable storage medium of claim 15, wherein determining that the group of query fingerprints match the reference fingerprints of the index includes: identifying two or more match candidates that satisfy the received search query based on the comparison; and identifying a time constraint associated with a day or time at which each of the match candidates was presented; and selecting one or more of the identified match candidates based on the identified time constraint. ### Claim 20 (depends on 15) The computer-readable storage medium of claim 15, wherein determining that the group of query fingerprints match the reference fingerprints of the index includes: identifying two or more match candidates that satisfy the received search query based on the comparison; and identifying known video content associated with a previous received search query; and selecting one or more of the identified match candidates based on the identified known video content associated with a previous received search query.