# US 2021/0243335 A1 · Color correction of imagery having different levels-of-detail > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243335 A1 - Document ID: US20210243335A1 - Status: Published application - Kind: A1 - Application number: 16/777,627 - Filed: 2020-01-30 - Published: 2021-08-05 - Earliest claimed: 2020-01-30 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Sebastian WLOCH - Web page: https://patentgrep.com/patent/US20210243335A1 ## CPC - H04N 1/6027: {Correction or control of colour gradation or colour contrast (H04N1/6058 takes precedence)} - H04N 1/6072: {adapting to different types of images, e.g. characters, graphs, black and white image portions} - G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) - G06T 2210/36: Level of detail - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method for color correcting imagery is disclosed. A plurality of images of a scene having different levels of detail is received. A first color correction is determined based on a first reference image and a first target image. The first reference image has a first level of detail and the first target image has a second level of detail greater than the first level of detail. The first color correction is applied to the first target image to generate a first color-corrected image having the second level of detail. A second color correction is determined based on the first color-corrected image and a second target image. The second target image has a third level of detail greater than the second level of detail. The second color correction is applied to the second target image to generate a second color-corrected image. ## Claims ### Claim 1 A method for color correcting imagery performed by a computing system, the method comprising: receiving a plurality of images of a scene having different levels of detail; determining a first color correction based on a first reference image and a first target image of the plurality of images, where the first reference image has a first level of detail and the first target image has a second level of detail greater than the first level of detail of the first reference image; applying the first color correction to the first target image to generate a first color-corrected image having the second level of detail; determining a second color correction based on the first color-corrected image and a second target image of the plurality of images, where the second target image has a third level of detail greater than the second level of detail of the first color-corrected image; and applying the second color correction to the second target image to generate a second color-corrected image having the third level of detail. ### Claim 2 (depends on 1) The method of claim 1, where determining the first color correction includes determining a contrast correction transformation matrix based on the first reference image and the first target image, and determining a brightness correction transformation matrix based on the first reference image and the first target image, and wherein applying the first color correction includes applying the contrast correction transformation matrix and/or the brightness correction transformation matrix to the first target image. ### Claim 3 (depends on 1) The method of claim 1, further comprising: assessing a first error between the first reference image and the first color-corrected image; responsive to the first error exceeding a first threshold, applying a second-pass color correction to the first target image, where the second-pass color correction includes 1) identifying one or more regions of interest in either the first reference image or the first target image, the one or more regions of interest having a color difference greater than a color difference threshold, 2) re-determining the first color correction based on the first reference image and the first target image while either modifying or ignoring the one or more regions of interest, and 3) applying the re-determined first color correction to the first target image to re-determine the first color-corrected image based on the second-pass color correction. ### Claim 4 (depends on 3) The method of claim 3, where modifying the one or more regions of interest includes copying pixels in the one or more regions of interest between the reference image and the target image. ### Claim 5 (depends on 3) The method of claim 3, further comprising: assessing a second error between the first reference image and the first color-corrected image as re-determined based on the second-pass color correction; and responsive to the second error exceeding a second threshold, applying a third-pass color correction to the first target image. ### Claim 6 (depends on 5) The method of claim 5, where the third-pass color correction includes 1) determining local matrices of contrast correction transformations and brightness correction transformations based on regions of one or more pixels of the first reference image and the first target image, and 2) applying the local matrices of contrast correction transformations and brightness correction transformations to the first target image. ### Claim 7 (depends on 6) The method of claim 6, where the third-pass color correction further includes 1) calculating derivatives of the contrast correction transformations and brightness correction transformations of the local matrices to generate masked local matrices, and 2) applying the masked local matrices to the first target image. ### Claim 8 (depends on 7) The method of claim 7, where the third-pass color correction further includes 1) performing a blurring operation on the masked local matrices to generate blurred masked local matrices and 2) applying the blurred masked local matrices to the first target image. ### Claim 9 (depends on 1) The method of claim 1, further comprising: sending, via a computer network, the first color correction and the second color correction to a remote computing system configured to store the first target image and the second target image, and where the remote computing system is configured to apply the first color correction to the first target image to generate the first color-corrected image and apply the second color correction to the second target image to generate the second color-corrected image. ### Claim 10 (depends on 9) The method of claim 9, where coefficients of a contrast correction transformation matrix and/or coefficients of a brightness correction transformation matrix corresponding to each of the first color correction and the second color correction are sent to the remote computing system. ### Claim 11 A computing system comprising: a logic subsystem; a storage subsystem holding instructions executable by the logic subsystem to: receive a plurality of images of a scene having different levels of detail; determine a first color correction based on a first reference image and a first target image of the plurality of images, where the first reference image has a first level of detail and the first target image has a second level of detail greater that the first level of detail of the first reference image; apply the first color correction to the first target image to generate a first color-corrected image having the second level of detail; determine a second color correction based on the first color-corrected image and a second target image of the plurality of images, where the second target image has a third level of detail greater than the second level of detail of the first color-corrected image; and apply the second color correction to the second target image to generate a second color-corrected image having the third level of detail. ### Claim 12 (depends on 11) The computing system of claim 11, where determining the first color correction includes determining a contrast correction transformation matrix between the first reference image and the first target image, determining a brightness correction transformation matrix between the first reference image and the first target image and derived from the contrast correction transformation matrix, and wherein applying the first color correction includes applying the contrast correction transformation matrix and/or the brightness correction transformation matrix to the first target image. ### Claim 13 (depends on 12) The computing system of claim 12, where the contrast correction transformation matrix is based on a quotient of a standard deviation of pixel values of the reference image and a standard deviation of pixel values of the target image. ### Claim 14 (depends on 12) The computing system of claim 12, where the brightness correction transformation matrix is based on a difference of an average pixel value of the reference image and an average pixel value of the target image multiplied by the contrast correction transformation matrix. ### Claim 15 (depends on 11) The computing system of claim 11, where the storage subsystem further holds instructions executable by the logic subsystem to: assess a first error between the first reference image and the first color-corrected image; responsive to the first error exceeding a first threshold, apply a second-pass color correction to the first target image, where the second-pass color correction includes 1) identifying one or more regions of interest in either the first reference image or the first target image, the one or more regions of interest having a color difference greater than a color difference threshold, 2) re-determining the first color correction based on the first reference image and the first target image while either modifying or ignoring the one or more regions of interest, and 3) applying the re-determined first color correction to the first target image to generate the first color-corrected image based on the second-pass color correction. ### Claim 16 (depends on 15) The computing system of claim 15, where modifying the one or more regions of interest includes copying pixels in the one or more regions of interest between the reference image and the target image. ### Claim 17 (depends on 15) The computing system of claim 15, where the storage subsystem further holds instructions executable by the logic subsystem to: assess a second error between the first reference image and the first color-corrected image generated based on the second-pass color correction; and responsive to the second error exceeding a second threshold, apply a third-pass color correction to the first target image, where the third-pass color correction includes 1) determining local matrices of contrast correction transformations and brightness correction transformations based on regions of one or more pixels of the first reference image and the first target image, and 2) applying the local matrices of contrast correction transformations and brightness correction transformations to the first target image. ### Claim 18 (depends on 11) The computing system of claim 11, where the storage subsystem further holds instructions executable by the logic subsystem to: send, via a computer network, the first color correction and the second color correction to a remote computing system configured to store the first target image and the second target image, and where the remote computing system is configured to apply the first color correction to the first target image to generate the first color-corrected image and apply the second color correction to the second target image to generate the second color-corrected image. ### Claim 19 (depends on 18) The computing system of claim 18, where coefficients of a contrast correction transformation matrix and/or coefficients of a brightness correction transformation matrix corresponding to each of the first color correction and the second color correction are sent to the remote computing system. ### Claim 20 A method for color correcting imagery performed by a computing system, the method comprising: receiving a plurality of images of a scene having different levels of detail; determining a first color correction based on a first reference image and a first target image of the plurality of images, where the first reference image has a first level of detail and the first target image has a second level of detail greater that the first level of detail of the first reference image, and where the first color correction includes a first contrast correction transformation matrix of coefficients and a first brightness correction transformation matrix of coefficients; applying the first color correction to the first target image to generate a first color-corrected image having the second level of detail; determining a second color correction based on the first color-corrected image and a second target image of the plurality of images, where the second target image has a third level of detail greater than the second level of detail of the first color-corrected image and where the second color correction includes a second contrast correction transformation matrix of coefficients and a second brightness correction transformation matrix of coefficients; sending, via a computer network, coefficients of the first contrast correction transformation matrix and/or coefficients of the first brightness correction transformation matrix of the first color correction and coefficients of the second contrast correction transformation matrix and/or coefficients of the second brightness correction transformation matrix of the second color correction to a remote computing system configured to store the first target image and the second target image, and where the remote computing system is configured to apply the first color correction to the first target image to generate the first color-corrected image and apply the second color correction to the second target image to generate the second color-corrected image.