# US 12,118,752 B2 · Determining colors of objects in digital images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,752 B2 - Document ID: US12118752B2 - Status: Granted - Kind: B2 - Application number: 17/658,799 - Filed: 2022-04-11 - Published: 2022-07-28 - Granted: 2024-10-15 - Earliest claimed: 2019-07-22 - Assignees: Adobe Inc. - Applicants: Adobe Inc. - Inventors: Zhihong Ding; Scott Cohen; Zhe Lin; Mingyang Ling - Web page: https://patentgrep.com/patent/US12118752B2 ## CPC - G06T 7/90: Determination of colour characteristics - G06F 18/22: Matching criteria, e.g. proximity measures - G06F 18/24: Classification techniques - G06N 3/02: Neural networks - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06T 7/11: Region-based segmentation - G06V 10/56: relating to colour - G06V 20/20: in augmented reality scenes - G06T 2207/10024: Color image - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The present disclosure relates to a color classification system that accurately classifies objects in digital images based on color. In particular, in one or more embodiments, the color classification system utilizes a multidimensional color space and one or more color mappings to match objects to colors. Indeed, the color classification system can accurately and efficiently detect the color of an object utilizing one or more color similarity regions generated in the multidimensional color space. ## Claims ### Claim 1 A method comprising: receiving a search query that comprises an indication of a target color and a corresponding query object; and identifying the corresponding query object in a digital image based on the indication of the target color by: identifying an object from a plurality of objects in the digital image, the object being formed by a plurality of pixels; mapping pixels of the plurality of pixels to a multidimensional color space to determine a set of color correspondences to a set of color similarity regions of the multidimensional color space, each of the color similarity regions being associated with a color and one or more alternative versions of the color; generating a set of color-matching scores for the object based on the set of color correspondences between the plurality of pixels and the set of color similarity regions; classifying the object as a first color associated with a first color similarity region based on the set of color-matching scores; and determining, based on a match between the indication of the target color and the first color, that the object is the corresponding query object. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving a search request for the object having the first color among a dataset of digital images; detecting a plurality of digital images that include the object from the dataset of digital images, the plurality of digital images including the digital image; generating a color-matching score for the object detected within each of the plurality of digital images with respect to the first color; identifying a subset of digital images of the plurality of digital images that comprise the object with a color-matching score satisfying a minimum color-matching threshold for the first color, the subset of digital images including the digital image; and provide the subset of digital images in response to the search request. ### Claim 3 (depends on 1) The method of claim 1, further comprising: detecting the plurality of objects within the digital image, wherein the plurality of objects comprise a plurality of instances of the object; generating a color-matching score for each instance of the object; classifying one or more instances of the object as having the first color based on the set of color matching scores; and returning the one or more instances of the object as having the first color. ### Claim 4 (depends on 1) The method of claim 1, further comprising: receiving the search query request for the object that comprises the indication of the target color where the target color is further indicated by a color name; identifying, in response to the search query indicating the color name of the target color, the first color similarity region for the first color within the multidimensional color space that is pre-mapped to a first color name; and returning, in response to identifying the first color similarity region for the first color within the multidimensional color space, an indication that the object in the digital image matches the first color based on the object being classified as the first color. ### Claim 5 (depends on 1) The method of claim 1, further comprising identifying the object in the digital image utilizing an object detection neural network. ### Claim 6 A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: receiving a search query that comprises an indication of a target color and a corresponding query object; and identifying the corresponding query object in a digital image based on the indication of the target color by: identifying an object from a plurality of objects in the digital image, the object being formed by a plurality of pixels; mapping pixels of the plurality of pixels to a multidimensional color space to determine a set of color correspondences to a set of color similarity regions of the multidimensional color space, each of the color similarity regions being associated with a color and one or more alternative versions of the color; generating a set of color-matching scores for the object based on the set of color correspondences between the plurality of pixels and the set of color similarity regions; classifying the object as a first color associated with a first color similarity region based on the set of color-matching scores; and determining, based on a match between the indication of the target color and the first color, that the object is the corresponding query object. ### Claim 7 (depends on 6) The non-transitory computer-readable medium of claim 6, further comprising instructions that, when executed by the at least one processor, the at least one processor to perform operations comprising: generating a second color similarity region corresponding to a second color by grouping one or more additional alternative versions of the second color and the second color together within the multidimensional color space. ### Claim 8 (depends on 6) The non-transitory computer-readable medium of claim 6, further comprising instructions that, when executed by the at least one processor, the at least one processor to perform operations comprising: determining an alternative version of the first color by: converting the first color from a first color model corresponding to the multidimensional color space to a second color model; modifying one or more color attributes of the first color within the second color model; and converting the first color with the modified one or more color attributes from the second color model back to the first color model corresponding to the multidimensional color space. ### Claim 9 (depends on 8) The non-transitory computer-readable medium of claim 8, wherein the one or more color attributes of the first color modified within the second color model comprise a color brightness, a color hue, or a color saturation level. ### Claim 10 (depends on 8) The non-transitory computer-readable medium of claim 8, wherein modifying the one or more color attributes of the first color within the second color model comprises reducing a brightness level of a first color copy in the second color model. ### Claim 11 (depends on 6) The non-transitory computer-readable medium of claim 6, wherein the first color similarity region for the first color comprises a first color point for the first color and a plurality of mapped alternative color points for the first color mapped to the multidimensional color space. ### Claim 12 (depends on 11) The non-transitory computer-readable medium of claim 11, wherein generating the set of color-matching scores for the object based on the set of color correspondences between the plurality of pixels and the set of color similarity regions comprises generating a first color-matching score for the object based on determining distances in the multidimensional color space between each of the pixels of the plurality of pixels of the object and each of the plurality of mapped alternative color points and the first color point. ### Claim 13 (depends on 12) The non-transitory computer-readable medium of claim 12, wherein classifying the object as the first color associated with the first color similarity region based on the set of color-matching scores comprises: assigning a pixel as valid based on the pixel being within a minimum threshold distance to at least one of the plurality of mapped alternative color points corresponding to the first color or to the first color point; and determining that the object matches the first color based on identifying a minimum percentage of pixels of the object being assigned as valid. ### Claim 14 A system comprising: one or more memory devices comprising a multidimensional color space for a plurality of colors comprising a plurality of color similarity regions each associated with a color label, the multidimensional color space comprising a first color similarity region associated with a first color label that groups regions of one or more alternative versions of a first color and a region of the first color within the multidimensional color space; and at least one processor configured to cause the system to: identify a plurality of instances of an object in a digital image; map pixels associated with each instance of the object to the multidimensional color space to determine correspondences to the first color similarity region; generate a color-matching score for each instance of the object based on the correspondences between the pixels associated with the instance of the object and the first color similarity region; classify a first instance of the object as the first color based on a color-matching score for the first instance of the object; and classify a second instance of the object as not being the first color based on a color-matching score for the second instance of the object. ### Claim 15 (depends on 14) The system of claim 14, wherein the at least one processor is further configured to receive a request to select pixels of an instance of the object in the digital image having the first color. ### Claim 16 (depends on 15) The system of claim 15, wherein the at least one processor is further configured to cause the system to: identify the plurality of instances of the object in the digital image in response to the request, wherein identifying the plurality of instances of the object comprises utilizing an object detection neural network to generate an approximate boundary for each instance of the object. ### Claim 17 (depends on 16) The system of claim 16, wherein the at least one processor is further configured to cause the system to: determine the pixels associated with each instance of the object by generating an object mask for each instance of the object from the approximate boundary for each instance of the object utilizing an object mask neural network. ### Claim 18 (depends on 14) The system of claim 14, wherein the first color similarity region comprises a complex-shaped color similarity region comprising a union of multidimensional spheres. ### Claim 19 (depends on 16) The system of claim 16, wherein the at least one processor is further configured to cause the system to generate the color-matching score for each instance of the object by determining a percentage of pixels of each instance of the object located within the first color similarity region. ### Claim 20 (depends on 19) The system of claim 19, wherein determining a percentage of pixels of each instance of the object located within the first color similarity region comprises determining a percentage of a subset of pixels of each instance of the object located within the first color similarity region.