# US 2020/0401838 A1 · Color extraction of a video stream > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401838 A1 - Document ID: US20200401838A1 - Status: Published application - Kind: A1 - Application number: 17/014,682 - Filed: 2020-09-08 - Published: 2020-12-24 - Earliest claimed: 2016-01-13 - Assignees: Snap Inc. - Applicants: Snap Inc. - Inventors: Maksim Igorevich Gusarov - Web page: https://patentgrep.com/patent/US20200401838A1 ## CPC - G06T 3/20: Linear translation of whole images or parts thereof, e.g. panning - G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215) - G06V 10/56: relating to colour - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - G06T 2207/10016: Video; Image sequence - G06T 2207/10024: Color image - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Systems, devices, media, and methods are presented for segmenting an image of a video stream with a client device, extracting one or more color from the image and modifying the video stream. The systems, devices, and method convert images of the set of images to a coordinate representation, perform a histogram equalization, identifies one or more colors of the coordinate representation based on an area of interest, determines a prevailing color, and applies the prevailing color to pixels of the video stream. ## Claims ### Claim 1 A computer-implemented method comprising: accessing, by one or more processors, a first image of a video stream depicting an object; and generating a uniform color range using a first range of color values of a color channel of the first image, the generating the uniform color range comprising translating the first range of color values of the color channel of the first image to the uniform color range, the translating comprising: computing a difference value between an old color value of a color in the color channel and a minimum of old color values of the color in the color channel; computing a ratio of the difference value to a difference between a maximum of the old color values and the minimum of the old color values; and computing a new color value for the first range of color values by applying a multiplication factor to the ratio. ### Claim 2 (depends on 1) The method of claim 1, further comprising determining one or more gray pixels in the first image by: computing a gray value as a function of: a first absolute value of a first difference between a red color and a green color; a second absolute value of a second difference between the green color and a blue color; and a third absolute value of a third difference between the blue color and the red color. ### Claim 3 (depends on 2) The method of claim 2, further comprising accumulating the first, second and third absolute values to compute the gray value. ### Claim 4 (depends on 3) The method of claim 3, further comprising comparing the gray value to a threshold to identify one or more grey pixels. ### Claim 5 (depends on 1) The method of claim 1, further comprising repeating the translating operations for translating another range of colors of another color channel to the uniform color range. ### Claim 6 (depends on 1) The method of claim 1, further comprising converting the first image to a hue, saturation, and value (HSV) coordinate representation, the HSV coordinate representation comprising a plurality of segments, each of the segments including a hue sum array, a saturation sum array, a value sum array and a count array. ### Claim 7 (depends on 6) The method of claim 6, wherein the hue sum array stores a sum of hues, wherein the saturation sum array stores a sum of saturations, the value sum array stores a sum of values, and the count array stores a total number of pixels mapped to the respective segment. ### Claim 8 (depends on 1) The method of claim 1, further comprising: detecting an area of interest for the first image positioned on a portion of the object depicted in the video stream; modifying, during capture of a video clip, pixel values of first image in the video clip to be within a set of predetermined pixel value ranges; computing a prevailing color of a portion of the video clip; generating a message for transmission to a second user device comprising the video clip; and automatically deleting the message transmitted to the second user device in response to a deletion trigger event that is a function of viewing time or viewing completion at the second user device. ### Claim 9 (depends on 1) The method of claim 1 further comprising: identifying an upper threshold for a lightness value within a set of cylindrical coordinate representations of the first image; determining one or more pixel values exceeding the upper threshold; and removing the one or more pixel values exceeding the upper threshold. ### Claim 10 (depends on 1) The method of claim 1 further comprising: identifying a lower threshold for a lightness value within a set of cylindrical coordinate representations of the first image; determining one or more pixel values which are below the lower threshold; and removing the one or more pixel values below the lower threshold. ### Claim 11 (depends on 1) The method of claim 1 further comprising: converting the first image to a hue, saturation and lightness (HSL) coordinate representation. ### Claim 12 (depends on 1) The method of claim 1 further comprising: converting the first image to a hue, saturation and value (HSV) coordinate representation, wherein the HSV coordinate representation is divided into a plurality of segments, a first segment of the plurality of segments includes a set of similar colors for a set of pixels of the first image, the first segment storing a sum of hues, a sum of saturations, and a sum of values for the set of pixels of the first image. ### Claim 13 A system, comprising: one or more processors; and a non-transitory machine-readable storage medium storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: accessing a first image of a video stream depicting an object; and generating a uniform color range using a first range of color values of a color channel of the first image, generating the uniform color range comprises translating the first range of color values of the color channel of the first image to the uniform color range, the translating comprising: computing a difference value between an old color value of a color in the color channel and a minimum of old color values of the color in the color channel; computing a ratio of the difference value to a difference between a maximum of the old color values and the minimum of the old color values; and computing a new color value for the first range of color values by applying a multiplication factor to the ratio. ### Claim 14 (depends on 13) The system of claim 13, wherein the operations further comprise determining one or more gray pixels in the first image by: computing a gray value as a function of: a first absolute value of a first difference between a red color and a green color; a second absolute value of a second difference between the green color and a blue color; and a third absolute value of a third difference between the blue color and the red color. ### Claim 15 (depends on 14) The system of claim 14, wherein the operations further comprise accumulating the first, second and third absolute values to compute the gray value. ### Claim 16 (depends on 13) The system of claim 13, wherein the operations further comprise: detecting an area of interest for the first image positioned on a portion of the object depicted in the video stream; modifying, during capture of a video clip, pixel values of first image in the video clip to be within a set of predetermined pixel value ranges; computing a prevailing color of a portion of the video clip; generating a message for transmission to a second user device comprising the video clip; and automatically deleting the message transmitted to the second user device in response to a deletion trigger event that is a function of viewing time or viewing completion at the second user device. ### Claim 17 A non-transitory machine-readable storage medium storing processor-executable instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising: accessing a first image of a video stream depicting an object; and generating a uniform color range using a first range of color values of a color channel of the first image, generating the uniform color range comprises translating the first range of color values of the color channel of the first image to the uniform color range, the translating comprising: computing a difference value between an old color value of a color in the color channel and a minimum of old color values of the color in the color channel; computing a ratio of the difference value to a difference between a maximum of the old color values and the minimum of the old color values; and computing a new color value for the first range of color values by applying a multiplication factor to the ratio. ### Claim 18 (depends on 17) The non-transitory machine-readable storage medium of claim 17, wherein the operations further comprise determining one or more gray pixels in the first image by: computing a gray value as a function of: a first absolute value of a first difference between a red color and a green color; a second absolute value of a second difference between the green color and a blue color; and a third absolute value of a third difference between the blue color and the red color. ### Claim 19 (depends on 18) The non-transitory machine-readable storage medium of claim 18, wherein the operations further comprise accumulating the first, second and third absolute values to compute the gray value. ### Claim 20 (depends on 17) The non-transitory machine-readable storage medium of claim 17, wherein the operations further comprise: detecting an area of interest for the first image positioned on a portion of the object depicted in the video stream; modifying, during capture of a video clip, pixel values of first image in the video clip to be within a set of predetermined pixel value ranges; computing a prevailing color of a portion of the video clip; generating a message for transmission to a second user device comprising the video clip; and automatically deleting the message transmitted to the second user device in response to a deletion trigger event that is a function of viewing time or viewing completion at the second user device.