# US 11,308,646 B2 · Extracting color from item images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,646 B2
- Document ID: US11308646B2
- Status: Granted
- Kind: B2
- Application number: 15/931,105
- Filed: 2020-05-13
- Published: 2021-11-18
- Granted: 2022-04-19
- Earliest claimed: 2020-05-13
- Assignees: Walmart Apollo, LLC
- Applicants: Walmart Apollo, LLC
- Inventors: Haider Syed
- Web page: https://patentgrep.com/patent/US11308646B2

## CPC

- G06Q 30/0603: {Catalogue creation or management}
- G06T 7/11: Region-based segmentation
- G06T 7/136: involving thresholding
- G06T 7/194: involving foreground-background segmentation
- G06T 7/90: Determination of colour characteristics
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 10/56: relating to colour
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06T 2207/10024: Color image
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system including one or more processors and one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform: obtaining an image of an item; removing background pixels from the image by removing white pixels from the image up to a first threshold; determining an item outline of the item in the image, wherein the item outline comprises aliased pixels along a periphery of the item in the image; removing grey pixels from the item outline in the image up to a second threshold to create a first updated image; removing shadows from the first updated image to create a second updated image based on a saliency map and a third threshold for shadow-like grey pixels; mapping each pixel in the second updated image to a respective mapped color in a predetermined color palette; and determining one or more dominant colors of the respective mapped colors based on one or more highest respective percentages of the respective mapped colors. Other embodiments are disclosed.

## Claims

### Claim 1

A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform:
obtaining an image of an item;
removing background pixels from the image by removing white pixels from the image based on a first criteria that depends on a first threshold;
determining an item outline of the item in the image, wherein the item outline comprises aliased pixels along a periphery of the item in the image;
removing grey pixels from the item outline in the image based on a second criteria that depends on a second threshold and a third threshold to create a first updated image, wherein the second threshold is greater than the first threshold;
removing shadows from the first updated image to create a second updated image based on a saliency map and a third criteria for shadow-like grey pixels; mapping each pixel in the second updated image to a respective mapped color in a predetermined color palette; and
determining one or more dominant colors of the respective mapped colors based on one or more highest respective percentages of the respective mapped colors.

### Claim 2 (depends on 1)

The system of claim 1, wherein the first criteria comprises:
|r−1|+|g−1|+|b−1|<0.05,
where r represents a scaled red value of an RGB color value of a respective pixel; g represents a scaled green value of the RGB color value of the respective pixel, b represents a scaled blue value of the RGB color value of the respective pixel, and the first threshold is 0.05.

### Claim 3 (depends on 1)

The system of claim 1, wherein the second criteria threshold comprises:
|r−1|+|g−1|+|b−1|<0.1 and m−s<10,
where r represents a scaled red value of an RGB color value of a respective pixel; g represents a scaled green value of the RGB color value of the respective pixel, b represents a scaled blue value of the RGB color value of the respective pixel, m represents a middle value of R, G, and B color values of the RGB color value of the respective pixel on a [0, 255] scale, s represents a minimum of the R, G, and B color values of the RGB color value of the respective pixel, the second threshold is 0.1, and the third threshold is 10.

### Claim 4 (depends on 1)

The system of claim 1, wherein determining the item outline of the item in the image further comprises:
generating an outer outline mask and an inner outline mask of the item in the image;
generating an item outline mask by subtracting the inner outline mask from the outer outline mask; and
determining the item outline based on the item outline mask.

### Claim 5 (depends on 1)

The system of claim 1, wherein the third criteria threshold for shadow-like grey pixels comprises:
a saturation level below a predetermined saturation level on a color saturation scale; and
a light value below a predetermined light value.

### Claim 6 (depends on 1)

The system of claim 1, wherein the third criteria threshold for shadow-like grey pixels comprises:
(R−G)<2 and (G−B)<2 and (R−B)<2,
where R represents a red value of an RGB color value of a respective pixel; G represents a green value of the RGB color value of the respective pixel, and B represents a blue value of the RGB color value of the respective pixel.

### Claim 7 (depends on 1)

The system of claim 1, wherein mapping the each pixel in the second updated image to the respective mapped color in the predetermined color palette further comprises:
precomputing a mapping from each possible RGB color value to a respective color in the predetermined color palette based on a distance between the each possible RGB color value and the respective color in the predetermined color palette;
storing respective color labels in a linear index array that is indexed based on the each possible RGB color value; and
using the linear index array to map the each pixel in the second updated image to the respective mapped color.

### Claim 8 (depends on 1)

The system of claim 1, wherein mapping the each pixel in the second updated image further comprises:
verifying that the respective mapped color matches a color in a set of predetermined colors for a hue slice in an HSV space.

### Claim 9 (depends on 1)

The system of claim 1, wherein mapping the each pixel in the second updated image further comprises:
verifying that the respective mapped color matches a color in a respective set of predetermined colors in one of a black-grey-white region or a color region of a hue slice in an HSV space.

### Claim 10 (depends on 1)

The system of claim 1, wherein computing instructions are further configured to perform:
updating a color attribute value for the item in an item catalog based on the one or more dominant colors.

### Claim 11

A method being implemented via execution of computing instructions configured to run at one or more processors and stored at one or more non-transitory computer-readable media, the method comprising:
obtaining an image of an item;
removing background pixels from the image by removing white pixels from the image based on a first criteria that depends on a first threshold;
determining an item outline of the item in the image, wherein the item outline comprises aliased pixels along a periphery of the item in the image;
removing grey pixels from the item outline in the image based on a second criteria that depends on a second threshold and a third threshold to create a first updated image, wherein the second threshold is greater than the first threshold;
removing shadows from the first updated image to create a second updated image based on a saliency map and a third criteria for shadow-like grey pixels;
mapping each pixel in the second updated image to a respective mapped color in a predetermined color palette; and
determining one or more dominant colors of the respective mapped colors based on one or more highest respective percentages of the respective mapped colors.

### Claim 12 (depends on 11)

The method of claim 11, wherein the first criteria comprises:
|r−1|+|g−1|+|b−1|<0.05,
where r represents a scaled red value of an RGB color value of a respective pixel; g represents a scaled green value of the RGB color value of the respective pixel, b represents a scaled blue value of the RGB color value of the respective pixel, and the first threshold is 0.05.

### Claim 13 (depends on 11)

The method of claim 11, wherein the second criteria threshold comprises:
|r−1|+|g−1|+|b−1|<0.1 and m−s<10,
where r represents a scaled red value of an RGB color value of a respective pixel; g represents a scaled green value of the RGB color value of the respective pixel, b represents a scaled blue value of the RGB color value of the respective pixel, m represents a middle second highest value of the R, G, and B color values of the RGB color value of the respective pixel on a [0, 255] scale, s represents a minimum of the R, G, and B color values of the RGB color value of the respective pixel, the second threshold is 0.1, and the third threshold is 10.

### Claim 14 (depends on 11)

The method of claim 11, wherein determining the item outline of the item in the image further comprises:
generating an outer outline mask and an inner outline mask of the item in the image;
generating an item outline mask by subtracting the inner outline mask from the outer outline mask; and
determining the item outline based on the item outline mask.

### Claim 15 (depends on 11)

The method of claim 11, wherein the third criteria threshold for shadow-like grey pixels comprises:
a saturation level below a predetermined saturation level on a color saturation scale; and
a light value below a predetermined light value.

### Claim 16 (depends on 11)

The method of claim 11, wherein the third criteria threshold for shadow-like grey pixels comprises:
(R−G)<2 and (G−B)<2 and (R−B)<2,
where R represents a red value of an RGB color value of a respective pixel; G represents a green value of the RGB color value of the respective pixel, and B represents a blue value of the RGB color value of the respective pixel.

### Claim 17 (depends on 11)

The method of claim 11, wherein mapping the each pixel in the second updated image to the respective mapped color in the predetermined color palette further comprises:
precomputing a mapping from each possible RGB color value to a respective color in the predetermined color palette based on a distance between the each possible RGB color value and the respective color in the predetermined color palette;
storing respective color labels in a linear index array that is indexed based on the each possible RGB color value; and
using the linear index array to map the each pixel in the second updated image to the respective mapped color.

### Claim 18 (depends on 11)

The method of claim 11, wherein mapping the each pixel in the second updated image further comprises:
verifying that the respective mapped color matches a color in a set of predetermined colors for a hue slice in an HSV space.

### Claim 19 (depends on 11)

The method of claim 11, wherein mapping the each pixel in the second updated image further comprises:
verifying that the respective mapped color matches a color in a respective set of predetermined colors in one of a black-grey-white region or a color region of a hue slice in an HSV space.

### Claim 20 (depends on 11)

The method of claim 11, further comprising:
updating a color attribute value for the item in an item catalog based on the one or more dominant colors.