# US 12,015,835 B2 · Multi-sensor imaging color correction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,015,835 B2 - Document ID: US12015835B2 - Status: Granted - Kind: B2 - Application number: 17/584,119 - Filed: 2022-01-25 - Published: 2023-07-27 - Granted: 2024-06-18 - Earliest claimed: 2022-01-25 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Micha Galor Gluskin; Jing Wang; Jiafu Luo - Web page: https://patentgrep.com/patent/US12015835B2 ## CPC - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N 23/16: Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction - G06T 7/90: Determination of colour characteristics - H04N 9/646: {for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters} - H04N 9/67: for matrixing (camera processing pipelines for matrixing of colour signals H04N23/85) - H04N 23/13: with multiple sensors - G06T 2207/10024: Color image - G06T 2207/10048: Infrared image - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems and techniques are provided for processing images. For example, a process can include obtaining a first color image including first one or more pixels from a first image sensor and obtaining a second color image including second one or more pixels from a second sensor, the second color image including infrared (IR) information from a second image sensor. The process can include determining a transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels based on a comparison associated with the first one or more pixels and the second one or more pixels. The process can include generating a color corrected image at least in part by transforming the second color image including IR information to a color corrected image based on the determined transformation. ## Claims ### Claim 1 A method for processing images, comprising: obtaining a first color image including first one or more pixels from a first image sensor, wherein an infrared (IR) cut filter is disposed between a lens of the first image sensor and the first one or more pixels from the first image sensor; obtaining a second color image including second one or more pixels from a second image sensor, the second color image including IR information from the second image sensor; determining a transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels based on a comparison associated with the first one or more pixels and the second one or more pixels; and generating a color corrected image at least in part by transforming the second color image including IR information to a color corrected image based on the determined transformation. ### Claim 2 (depends on 1) The method of claim 1, wherein a field of view associated with the first one or more pixels at least partially overlaps with a field of view associated with the second one or more pixels. ### Claim 3 (depends on 1) The method of claim 1, wherein the first image sensor is associated with a first optical axis, and the second image sensor is associated with a second optical axis, different from the first optical axis. ### Claim 4 (depends on 1) The method of claim 1, wherein the first image sensor has a first resolution, and the second image sensor has a second resolution, different from the first resolution. ### Claim 5 (depends on 1) The method of claim 1, wherein the first image sensor comprises a single pixel. ### Claim 6 (depends on 1) The method of claim 1, wherein the first image sensor comprises a plurality of pixels. ### Claim 7 (depends on 1) The method of claim 1, wherein each of the first one or more pixels includes a corresponding red component, green component, and blue component and each of the second one or more pixels includes a corresponding red component, green component, and blue component, and wherein determining the transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels comprises: determining a first green statistical measure of green components of the first one or more pixels and a second green statistical measure of green components of the second one or more pixels; determining a first red statistical measure of red components of the first one or more pixels and a second red statistical measure of red components of the second one or more pixels; determining a first blue statistical measure of blue components of the first one or more pixels and a second blue statistical measure of blue components of the second one or more pixels; determining a green color correction based on a numerical correspondence of the first green statistical measure and the second green statistical measure; determining a red color correction based on a numerical correspondence of the first red statistical measure and the second red statistical measure; and determining a blue color correction based on a numerical correspondence of the first blue statistical measure and the second blue statistical measure. ### Claim 8 (depends on 7) The method of claim 7, wherein generating the color corrected image comprises applying the red color correction to the red components of each pixel of the second one or more pixels, applying the green color correction to the green components of each pixel of the second one or more pixels, and applying the blue color correction to the blue components of each pixel of the second one or more pixels. ### Claim 9 (depends on 1) The method of claim 1, wherein the transformation between colors of the first one or more pixels and colors of the second one or more pixels comprises a color correction matrix. ### Claim 10 (depends on 1) The method of claim 1, wherein generating the color corrected image comprises applying the determined transformation to the second one or more pixels to produce first color corrected pixels. ### Claim 11 (depends on 1) The method of claim 1, wherein determining the transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels comprises: dividing one or more of the first color image and the second color image into a plurality of sections; determining a first section transformation associated with a first subset of the second one or more pixels associated; and determining a second section transformation associated with a second subset of the second one or more pixels, wherein the second subset of the second one or more pixels differs from the first subset by at least one pixel. ### Claim 12 (depends on 11) The method of claim 11, further comprising determining a global transformation for generating the color corrected image based on the first section transformation and the second section transformation. ### Claim 13 (depends on 11) The method of claim 11, further comprising generating the color corrected image at least in part by transforming the first subset of the second one or more pixels by the first section transformation and transforming the second subset of the second one or more pixels by the second section transformation. ### Claim 14 (depends on 1) The method of claim 1, wherein the first one or more pixels do not include IR information. ### Claim 15 An apparatus for processing images, comprising: a memory; and one or more processors coupled to the memory and configured to: obtain a first color image including first one or more pixels from a first image sensor, wherein an infrared (IR) cut filter is disposed between a lens of the first image sensor and the first one or more pixels from the first image sensor; obtain a second color image including second one or more pixels from a second image sensor, the second color image including IR information from the second image sensor; determine a transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels based on a comparison associated with the first one or more pixels and the second one or more pixels; and generate a color corrected image at least in part by transform the second color image including IR information to a color corrected image based on the determined transformation. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein a field of view associated with the first one or more pixels at least partially overlaps with a field of view associated with the second one or more pixels. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the first image sensor is associated with a first optical axis, and the second image sensor is associated with a second optical axis, different from the first optical axis. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein the first image sensor has a first resolution, and the second image sensor has a second resolution, different from the first resolution. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the first image sensor comprises a single pixel. ### Claim 20 (depends on 15) The apparatus of claim 15, wherein the first image sensor comprises a plurality of pixels. ### Claim 21 (depends on 15) The apparatus of claim 15, wherein: each of the first one or more pixels includes a corresponding red component, green component, and blue component and each of the second one or more pixels includes a corresponding red component, green component, and blue component; and to determine the transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels, the one or more processors are configured to: determine a first green statistical measure of green components of the first one or more pixels and a second green statistical measure of green components of the second one or more pixels; determine a first red statistical measure of red components of the first one or more pixels and a second red statistical measure of red components of the second one or more pixels; determine a first blue statistical measure of blue components of the first one or more pixels and a second blue statistical measure of blue components of the second one or more pixels; determine a green color correction based on a numerical correspondence of the first green statistical measure and the second green statistical measure; determine a red color correction based on a numerical correspondence of the first red statistical measure and the second red statistical measure; and determine a blue color correction based on a numerical correspondence of the first blue statistical measure and the second blue statistical measure. ### Claim 22 (depends on 21) The apparatus of claim 21, wherein, to generate the color corrected image, the one or more processors are configured to apply the red color correction to the red components of each pixel of the second one or more pixels, apply the green color correction to the green components of each pixel of the second one or more pixels, and apply the blue color correction to the blue components of each pixel of the second one or more pixels. ### Claim 23 (depends on 15) The apparatus of claim 15, wherein the transformation between colors of the first one or more pixels and colors of the second one or more pixels comprises a color correction matrix. ### Claim 24 (depends on 15) The apparatus of claim 15, wherein, to generate the color corrected image, the one or more processors are configured to apply the determined transformation to the second one or more pixels to produce first color corrected pixels. ### Claim 25 (depends on 15) The apparatus of claim 15, wherein the one or more processors are configured to: divide one or more of the first color image and the second color image into a plurality of sections; determine a first section transformation associated with a first subset of the second one or more pixels associated; and determine a second section transformation associated with a second subset of the second one or more pixels, wherein the second subset of the second one or more pixels differs from the first subset by at least one pixel. ### Claim 26 (depends on 25) The apparatus of claim 25, wherein the one or more processors are configured to: determine a global transformation for generate the color corrected image based on the first section transformation and the second section transformation. ### Claim 27 (depends on 25) The apparatus of claim 25, wherein the one or more processors are configured to: generate the color corrected image at least in part by transform the first subset of the second one or more pixels by the first section transformation and transform the second subset of the second one or more pixels by the second section transformation. ### Claim 28 (depends on 15) The apparatus of claim 15, wherein the first one or more pixels do not include IR information.