# US 12,222,980 B2 · Generating congruous metadata for multimedia > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,222,980 B2 - Document ID: US12222980B2 - Status: Granted - Kind: B2 - Application number: 18/362,482 - Filed: 2023-07-31 - Published: 2023-11-23 - Granted: 2025-02-11 - Earliest claimed: 2008-03-04 - Assignees: Yahoo Assets LLC - Applicants: Yahoo Assets LLC - Inventors: Malcolm Slaney; Kilian Weinberger - Web page: https://patentgrep.com/patent/US12222980B2 ## CPC - G06F 16/5838: {using colour} - G06F 16/58: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually - G06F 16/587: using geographical or spatial information, e.g. location - G06F 16/68: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually - G06F 16/683: using metadata automatically derived from the content - G06F 16/687: using geographical or spatial information, e.g. location - G06F 16/78: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually - G06F 16/7847: {using low-level visual features of the video content} - G06F 16/787: using geographical or spatial information, e.g. location - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method of generating congruous metadata is provided. The method includes receiving a similarity measure between at least two multimedia objects. Each multimedia object has associated metadata. If the at least two multimedia objects are similar based on the similarity measure and a similarity threshold, the associated metadata of each of the multimedia objects are compared. Then, based on the comparison of the associated metadata of each of the at least two multimedia objects, the method further includes generating congruous metadata. Metadata may be tags, for example. ## Claims ### Claim 1 A method comprising: using, by a computing device, information indicating an absence of user verification of computer-generated congruous metadata to dynamically adjust an adjustable similarity threshold from less restrictive to more restrictive; analyzing, by the computing device, digital data of first and second content items and determining a similarity between the first and second content items based on the digital data analysis; determining, by the computing device, that the similarity determined for the first and second content items satisfies the more restrictive adjustable similarity threshold that is set based on the absence of user verification; responsive to determining that the similarity satisfies the more restrictive adjustable similarity threshold, analyzing, by the computing device, a first set of metadata associated with the first content item and a second set of metadata associated with the second content item, and generating, absent user verification, a third set of metadata found in both the first and second sets of metadata based on the analysis; storing, by the computing device, the third set of metadata generated absent user verification in a data store, the storing comprising associating the third set of metadata with the first content item and the second content item; and identifying, by the computing device, one or both of the first and second content items using the third set of metadata. ### Claim 2 (depends on 1) The method of claim 1, the storing further comprising labeling the third set of metadata as computer-generated congruous metadata indicating that the third set of metadata accurately describes the first content item and the second content item. ### Claim 3 (depends on 1) The method of claim 1, further comprising using the third set of metadata in a content item search and retrieval operation. ### Claim 4 (depends on 1) The method of claim 1, wherein the first and second content items are images. ### Claim 5 (depends on 1) The method of claim 1, wherein the first and second content items are audio objects. ### Claim 6 (depends on 1) The method of claim 1, wherein the first and second content items are videos. ### Claim 7 (depends on 1) The method of claim 1, wherein the first, second and third sets of metadata are tag annotations. ### Claim 8 (depends on 7) The method of claim 7, wherein the first and second sets of metadata are user-supplied tag annotations, and the third set of metadata comprises those user-supplied tag annotations from the first and second sets of metadata determined, based on the analysis of the first and second sets of metadata, to be accurately descriptive of the first and second content items. ### Claim 9 (depends on 1) The method of claim 1, wherein the first and second content items belong to a set of content items comprising at least one other content item whose similarity with the first and second content items satisfies the more restrictive adjustable similarity threshold, and the third set of metadata is associated with each content item in the set of content items. ### Claim 10 (depends on 1) The method of claim 1, wherein the first and second sets of metadata each comprise time and location indications that are analyzed and used in generating the third set of metadata based on the analysis. ### Claim 11 A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device perform a method comprising: using information indicating an absence of user verification of computer-generated congruous metadata to dynamically adjust an adjustable similarity threshold from less restrictive to more restrictive; analyzing digital data of first and second content items and determining a similarity between the first and second content items based on the digital data analysis; determining that the similarity determined for the first and second content items satisfies the more restrictive adjustable similarity threshold that is set based on the absence of user verification; responsive to determining that the similarity satisfies the more restrictive adjustable similarity threshold, analyzing a first set of metadata associated with the first content item and a second set of metadata associated with the second content item and generating, absent user verification, a third set of metadata found in both the first and second sets of metadata based on the analysis; storing the third set of metadata in a data store, the storing comprising associating the third set of metadata generated absent user verification with the first content item and the second content item; and identifying one or both of the first and second content items using the third set of metadata. ### Claim 12 (depends on 11) The non-transitory computer-readable storage medium of claim 11, the storing further comprising labeling the third set of metadata as computer-generated congruous metadata indicating that the third set of metadata accurately describes the first content item and the second content item. ### Claim 13 (depends on 11) The non-transitory computer-readable storage medium of claim 11, the method further comprising using the third set of metadata in a content item search and retrieval operation. ### Claim 14 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the first and second content items are images. ### Claim 15 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the first and second content items are audio objects. ### Claim 16 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the first and second content items are videos. ### Claim 17 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the first, second and third sets of metadata are user-supplied tag annotations, and wherein the third set of metadata comprises those user-supplied tag annotations from the first and second sets of metadata determined, based on the analysis of the first and second sets of metadata, to be accurately descriptive of the first and second content items. ### Claim 18 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the first and second content items belong to a set of content items comprising at least one other content item whose similarity with the first and second content items satisfies the more restrictive adjustable similarity threshold, and the third set of metadata is associated with each content item in the set of content items. ### Claim 19 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the first and second sets of metadata each comprise time and location indications that are analyzed and used in generating the third set of metadata based on the analysis. ### Claim 20 A computing device comprising: a processor; and a non-transitory storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising: using logic executed by the processor for using information indicating an absence of user verification of computer-generated congruous metadata to dynamically adjust an adjustable similarity threshold from less restrictive to more restrictive; analyzing logic executed by the processor for analyzing digital data of first and second content items and determining a similarity between the first and second content items based on the digital data analysis; determining logic executed by the processor for determining that the similarity determined for the first and second content items satisfies the more restrictive adjustable similarity threshold that is set based on the absence of user verification; analyzing logic executed by the processor for, responsive to determining that the similarity satisfies the more restrictive adjustable similarity threshold, analyzing a first set of metadata associated with the first content item and a second set of metadata associated with the second content item and generating, absent user verification, a third set of metadata found in both the first and second sets of metadata based on the analysis; storing logic executed by the processor for storing the third set of metadata generated absent user verification in a data store, the storing comprising associating the third set of metadata with the first content item and the second content item; and identifying logic executed by the processor for identifying one or both of the first and second content items using the third set of metadata.