# US 2016/0366388 A1 · Methods and devices for gray point estimation in digital images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2016/0366388 A1 - Document ID: US20160366388A1 - Status: Published application - Kind: A1 - Application number: 14/735,699 - Filed: 2015-06-10 - Published: 2016-12-15 - Earliest claimed: 2015-06-10 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Euan Barron - Web page: https://patentgrep.com/patent/US20160366388A1 ## CPC - H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control - G06T 5/40: using histogram techniques - G06T 7/90: Determination of colour characteristics - H04N 1/6027: {Correction or control of colour gradation or colour contrast (H04N1/6058 takes precedence)} - H04N 1/6077: {Colour balance, e.g. colour cast correction} - H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices - H04N 23/84: for processing colour signals - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A device and a method for estimating gray point in digital image frames are disclosed. The method includes obtaining a digital image frame and determining red-green-blue (RGB) values for each pixel in at least a part of the digital image frame. The method further includes calculating a first component value and a second component value in a pre-determined color space for said each pixel, where the first component value and the second component value are calculated from the RGB values for said each pixel. The method further includes determining a two-dimensional (2-D) distribution based on the first component value and the second component value for said each pixel. Thereafter, the method includes identifying one or more saturated color clusters in the 2-D distribution, and analyzing the one or more saturated color clusters to estimate a gray point for at least the part of the digital image frame. ## Claims ### Claim 1 A method, comprising: obtaining a digital image frame; determining red-green-blue (RGB) values for each pixel in at least a part of the digital image frame; calculating a first component value and a second component value in a pre-determined color space for said each pixel, the first component value and the second component value calculated from the RGB values for said each pixel; determining a two-dimensional (2-D) distribution based on the first component value and the second component value for said each pixel; identifying one or more saturated color clusters in the 2-D distribution; and analyzing the one or more saturated color clusters to estimate a gray point for at least the part of the digital image frame. ### Claim 2 (depends on 1) The method of claim 1, wherein the first component value and the second component value correspond to a R/G (red/green) value and a B/G (blue/green) value in an RGB color space, respectively. ### Claim 3 (depends on 2) The method of claim 2, wherein identifying the one or more saturated color clusters in the 2-D distribution comprises: identifying one or more peak values distally located from a substantially central area in the 2-D distribution; and selecting localized regions associated with the one or more peak values as the one or more saturated color clusters. ### Claim 4 (depends on 3) The method of claim 3, wherein analyzing the one or more saturated color clusters comprises: determining principal component axes for the one or more saturated color clusters, wherein a principal component axis is determined for each saturated color cluster from among the one or more saturated color clusters; projecting the principal component axes to identify a point of intersection of the projected principal component axes; and comparing the point of intersection with gray points on a gray point curve for different lighting conditions for an image capture module from which the digital image frame is originated, wherein the gray point of at least the part of the digital image frame is estimated based on the comparison. ### Claim 5 (depends on 1) The method of claim 1, wherein the first component value and the second component value correspond to a saturation value and a hue value, respectively. ### Claim 6 (depends on 5) The method of claim 5, wherein identifying the one or more saturated color clusters in the 2-D distribution comprises: identifying one or more peak values in the 2-D distribution; and selecting localized regions associated with the one or more peak values as the one or more saturated color clusters. ### Claim 7 (depends on 6) The method of claim 6, wherein analyzing the one or more saturated color clusters comprises: determining if each saturated color cluster of the one more saturated color clusters is associated with a symmetrical distribution on either side of a substantially central constant hue axis associated with said each saturated color cluster; and estimating at least one gray point shift value required for obtaining the symmetrical distribution if at least one saturated color cluster from among the one or more saturated color clusters is associated with an asymmetrical distribution, wherein the gray point of at least the part of the digital image frame is estimated based on the at least one gray point shift value. ### Claim 8 (depends on 1) The method of claim 1, further performing, if the digital image frame is obtained in a raw format, a white balancing of at least the part of the digital image frame based on the estimated gray point. ### Claim 9 (depends on 1) The method of claim 1, further performing, if the digital image frame is obtained as a processed image format or obtained post white balancing of the digital image frame: determining an accuracy of the white balancing of at least the part of the digital image frame based on the estimated gray point; and correcting the white balancing of at least the part of the digital image based on the estimated gray point if the white balancing is determined to be inaccurate. ### Claim 10 (depends on 9) The method of claim 9, wherein the first component value and the second component value correspond to an ‘A’ color channel co-ordinate and a ‘B’ color channel co-ordinate in a LAB color space, respectively. ### Claim 11 A device, comprising: at least one memory comprising image processing instructions, the at least one memory configured to receive and store a digital image frame; and at least one processor communicably coupled with the at least one memory, the at least one processor is configured to execute the image processing instructions to at least perform: determining red-green-blue (RGB) values for each pixel in at least a part of the digital image frame; calculating a first component value and a second component value in a pre-determined color space for the said each pixel, the first component value and the second component value calculated from the RGB values for the said each pixel; determining a two-dimensional (2-D) distribution based on the first component value and the second component value for the said each pixel; identifying one or more saturated color clusters in the 2-D distribution; and analyzing the one or more saturated color clusters to estimate a gray point for at least the part of the digital image frame. ### Claim 12 (depends on 11) The device of claim 11, wherein the first component value and the second component value correspond to a R/G (red/green) value and a B/G (blue/green) value in a RGB color space, respectively. ### Claim 13 (depends on 12) The device of claim 12, wherein the at least one processor is configured to identify the one or more saturated color clusters in the 2-D distribution by: identifying one or more peak values distally located from a substantially central area in the 2-D distribution; and selecting localized regions associated with the one or more peak values as the one or more saturated color clusters. ### Claim 14 (depends on 13) The device of claim 13, wherein the at least one processor is configured to analyze the one or more saturated color clusters by: determining principal component axes for the one or more saturated color clusters, wherein a principal component axis is determined for each saturated color cluster from among the one or more saturated color clusters; projecting the principal component axes to identify a point of intersection of the projected principal component axes; and comparing the point of intersection with gray points on a gray point curve for different lighting conditions for an image capture module from which the digital image frame is originated, wherein the gray point of at least the part of the digital image frame is estimated based on the comparison. ### Claim 15 (depends on 11) The device of claim 11, wherein the first component value and the second component value correspond to a saturation value and a hue value, respectively, and wherein the at least one processor is configured to identify the one or more saturated color clusters in the 2-D distribution by: identifying one or more peak values in the 2-D distribution; and selecting localized regions associated with the one or more peak values as the one or more saturated color clusters. ### Claim 16 (depends on 15) The device of claim 15, wherein the at least one processor is configured to analyze the one or more saturated color clusters by: determining if each saturated color cluster of the one more saturated color clusters is associated with a symmetrical distribution on either side of a substantially central constant hue axis associated with said each saturated color cluster; and estimating at least one gray point shift value required for obtaining the symmetrical distribution if at least one saturated color cluster from among the one or more saturated color clusters is associated with an asymmetrical distribution, wherein the gray point of at least the part of the digital image frame is estimated based on the at least one gray point shift value. ### Claim 17 (depends on 11) The device of claim 11, wherein the at least one processor is configured to further perform: a white balancing of at least the part of the digital image frame based on the estimated gray point if the digital image frame is obtained in a raw format, and a determination of an accuracy of the white balancing of at least the part of the digital image frame if the digital image frame is obtained as a processed image format or obtained post white balancing of the digital image frame, and, correct the white balancing of at least the part of the digital image based on the estimated gray point if the white balancing is determined to be inaccurate. ### Claim 18 (depends on 11) The device of claim 11, wherein the device is implemented in at least one of a mobile device, an image-processing module in an image capture device or a remote web-based server. ### Claim 19 A method, comprising: obtaining a digital image frame; partitioning the digital image frame into a plurality of parts based on a pre-determined criterion; and processing each part from among the plurality of parts by: determining red-green-blue (RGB) values for each pixel in said each part; calculating a first component value and a second component value in a pre-determined color space for said each pixel, the first component value and the second component value calculated from the RGB values for said each pixel; determining a two-dimensional (2-D) distribution based on the first component value and the second component value for said each pixel; identifying one or more saturated color clusters in the 2-D distribution; and analyzing the one or more saturated color clusters to estimate a gray point for said each part; and, performing white balancing of said each part based on the estimated gray point for said each part. ### Claim 20 (depends on 19) The method of claim 19, wherein the digital image frame is partitioned into the plurality of parts based on average color hue criterion.