# US 2008/0304087 A1 · Method of compensating the color tone differences between two images of the same scene

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0304087 A1
- Document ID: US20080304087A1
- Status: Published application
- Kind: A1
- Application number: 11/811,717
- Filed: 2007-06-11
- Published: 2008-12-11
- Earliest claimed: 2007-06-11
- Assignees: Sony Corporation and Sony Electronics Inc.
- Inventors: Jaewon Shin; Mohammad Gharavi-Alkhansari; Ali Tabatabai
- Web page: https://patentgrep.com/patent/US20080304087A1

## CPC

- G06T 5/40: using histogram techniques
- G06T 5/92: based on global image properties
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The color tone compensation method provides a simple and efficient method to compensate the color tone differences between two different sources of images. A first image sample, such as a still image, from a first image capturing source and a second image sample, such as a video frame, from a second image capturing source are aligned, and a tone-mapping estimation routine is applied to the two aligned images. The tone-mapping estimation routine uses the pixel intensity value histograms associated with the two aligned images and generates a tone mapping table. The tone mapping table includes a conversion intensity value for each intensity value in the second image. The conversion intensity value is a statistical measure, such as the mean, calculated according to the data in the corresponding pixel intensity value histogram. The tone-mapping table is applied to any image generated by the second image capturing source, thereby generating a new image with similar color tone as the first image generated by the first image capturing source.

## Claims

### Claim 1

A method of compensating for color tone differences between two images, the method comprising:
a. receiving a first image of a scene and a second image of the scene, wherein the first image is generated by a first source and the second image is generated by a second source;
b. determining an intensity value for each pixel in the first image and for each pixel in the second image;
c. aligning each pixel in the second image to a pixel in the first image;
d. for each specific intensity value in the second image, establishing a relationship between the specific intensity value in the second image and a corresponding range of intensity values in the first image;
e. determining a conversion intensity value for each specific intensity value in the second image according to a statistical measure of the corresponding range of intensity values in the first image;
f. generating a tone-mapping table according to the conversion intensity values such that each intensity value in the second image is associated with a specific conversion intensity value; and
g. applying the tone-mapping table to one or more images generated by the second source, thereby forming one or more third images.

### Claim 2 (depends on 1)

The method of claim 1 wherein applying the tone-mapping table comprises:
a. determining an intensity value for each pixel in the one or more images;
b. determining the conversion intensity value for each intensity value in the one or more images according to the tone-mapping table; and
c. applying the conversion intensity value to each pixel in the one or more images.

### Claim 3 (depends on 1)

The method of claim 1 wherein establishing the relationship between the specific intensity value in the second image and the corresponding range of intensity values in the first image includes determining a specific set of pixels in the second image that correspond to the specific intensity level, and determining a specific set of pixels in the first image that match the specific set of pixels in the second image, wherein the matching specific set of pixels in the first image include the range of intensity values corresponding to the specific set of pixels in the first image.

### Claim 4 (depends on 1)

The method of claim 1 wherein the first source and the second source are configured within an integrated device.

### Claim 5 (depends on 1)

The method of claim 1 wherein the first image is generated in a first color space and the second image is generated in a second color space different than the first color space.

### Claim 6 (depends on 1)

The method of claim 1 wherein the first image and the second image are generated at the same time.

### Claim 7 (depends on 1)

The method of claim 1 wherein the first image and the second image are generated at different times.

### Claim 8 (depends on 1)

The method of claim 1 wherein the first image comprises a still image and the second image comprises a video frame.

### Claim 9 (depends on 8)

The method of claim 8 wherein aligning each pixel comprises up-sampling the video frame such that a number of pixels in the video frame matches a number of pixels in the still image.

### Claim 10 (depends on 1)

The method of claim 1 wherein the first image comprises a video frame and the second image comprises a still image.

### Claim 11 (depends on 10)

The method of claim 10 wherein aligning each pixel comprises up-sampling the video frame such that a number of pixels in the video frame matches a number of pixels in the still image.

### Claim 12 (depends on 1)

The method of claim 1 wherein the first image comprises a first still image and the second image comprises a second still image.

### Claim 13 (depends on 1)

The method of claim 1 wherein the first image comprises a first video frame and the second image comprises a second video frame.

### Claim 14 (depends on 1)

The method of claim 1 wherein the statistical measure is the mean of the range of intensity values in the first image.

### Claim 15 (depends on 1)

The method of claim 1 wherein the statistical measure is the average of the range of intensity values in the first image.

### Claim 16 (depends on 1)

The method of claim 1 further comprising generating the tone-mapping table for each color component in the second image.

### Claim 17

A method of compensating for color tone differences between two images, the method comprising:
a. receiving a first image of a scene and a second image of the scene, wherein the first image is generated by a first source and the second image is generated by a second source;
b. determining an intensity value for each pixel in the first image and for each pixel in the second image;
c. aligning each pixel in the second image to a pixel in the first image;
d. for each specific intensity value in the second image, determining a specific set of pixels in the second image that correspond to the specific intensity level, and determining a specific set of pixels in the first image that match the specific set of pixels in the second image, wherein the matching specific set of pixels in the first image include a corresponding range of intensity values, thereby establishing a relationship between the specific intensity value in the second image and the corresponding range of intensity values in the first image;
e. determining a conversion intensity value for each specific intensity value in the second image according to a statistical measure of the corresponding range of intensity values in the first image;
f. generating a tone-mapping table according to the conversion intensity values such that each intensity value in the second image is associated with a specific conversion intensity value; and
g. applying the tone-mapping table to one or more images generated by the second source, thereby forming one or more third images.

### Claim 18 (depends on 17)

The method of claim 17 wherein applying the tone-mapping table comprises:
a. determining an intensity value for each pixel in the one or more images;
b. determining the conversion intensity value for each intensity value in the one or more images according to the tone-mapping table; and
c. applying the conversion intensity value to each pixel in the one or more images.

### Claim 19 (depends on 17)

The method of claim 17 wherein the first source and the second source are configured within an integrated device.

### Claim 20 (depends on 17)

The method of claim 17 wherein the first image is generated in a first color space and the second image is generated in a second color space different than the first color space.

### Claim 21 (depends on 17)

The method of claim 17 wherein the first image and the second image are generated at the same time.

### Claim 22 (depends on 17)

The method of claim 17 wherein the first image and the second image are generated at different times.

### Claim 23 (depends on 17)

The method of claim 17 wherein the first image comprises a still image and the second image comprises a video frame.

### Claim 24 (depends on 23)

The method of claim 23 wherein aligning each pixel comprises up-sampling the video frame such that a number of pixels in the video frame matches a number of pixels in the still image.

### Claim 25 (depends on 17)

The method of claim 17 wherein the first image comprises a video frame and the second image comprises a still image.

### Claim 26 (depends on 25)

The method of claim 25 wherein aligning each pixel comprises up-sampling the video frame such that a number of pixels in the video frame matches a number of pixels in the still image.

### Claim 27 (depends on 17)

The method of claim 17 wherein the first image comprises a first still image and the second image comprises a second still image.

### Claim 28 (depends on 17)

The method of claim 17 wherein the first image comprises a first video frame and the second image comprises a second video frame.

### Claim 29 (depends on 17)

The method of claim 17 wherein the statistical measure is the mean of the range of intensity values in the first image.

### Claim 30 (depends on 17)

The method of claim 17 wherein the statistical measure is the average of the range of intensity values in the first image.

### Claim 31 (depends on 17)

The method of claim 17 further comprising generating the tone-mapping table for each color component in the second image.

### Claim 32

A method of compensating for color tone differences between two images, the method comprising:
a. receiving a first image of a scene and a second image of the scene, wherein the first image is generated by a first source and the second image is generated by a second source;
b. aligning groups of pixels in the second image to groups of pixels in the first image;
c. determining an intensity value for each pixel group in the first image and for each pixel group in the second image;
d. for each specific intensity value in the second image, establishing a relationship between the specific intensity value in the second image and a corresponding range of intensity values in the first image;
e. determining a conversion intensity value for each specific intensity value in the second image according to a statistical measure of the corresponding range of intensity values in the first image;
f. generating a tone-mapping table according to the conversion intensity values such that each intensity value in the second image is associated with a specific conversion intensity value; and
g. applying the tone-mapping table to one or more images generated by the second source, thereby forming one or more third images.

### Claim 33 (depends on 32)

The method of claim 32 wherein applying the tone-mapping table comprises:
a. determining an intensity value for each pixel group in the one or more images;
b. determining the conversion intensity value for each intensity value in the one or more images according to the tone-mapping table; and
c. applying the conversion intensity value to each pixel group in the one or more images.

### Claim 34 (depends on 32)

The method of claim 32 wherein establishing the relationship between the specific intensity value in the second image and the corresponding range of intensity values in the first image includes determining a specific set of pixel groups in the second image that correspond to the specific intensity level, and determining a specific set of pixel groups in the first image that match the specific set of pixel groups in the second image, wherein the matching specific set of pixel groups in the first image include the range of intensity values corresponding to the specific set of pixel groups in the first image.

### Claim 35 (depends on 32)

The method of claim 32 wherein the first source and the second source are configured within an integrated device.

### Claim 36 (depends on 32)

The method of claim 32 wherein the first image is generated in a first color space and the second image is generated in a second color space different than the first color space.

### Claim 37 (depends on 32)

The method of claim 32 wherein the first image and the second image are generated at the same time.

### Claim 38 (depends on 32)

The method of claim 32 wherein the first image and the second image are generated at different times.

### Claim 39 (depends on 32)

The method of claim 32 wherein the first image comprises a still image and the second image comprises a video frame.

### Claim 40 (depends on 39)

The method of claim 39 wherein aligning each pixel comprises up-sampling the video frame such that a number of pixel groups in the video frame matches a number of pixel groups in the still image.

### Claim 41 (depends on 32)

The method of claim 32 wherein the first image comprises a video frame and the second image comprises a still image.

### Claim 42 (depends on 41)

The method of claim 41 wherein aligning each pixel comprises up-sampling the video frame such that a number of pixel groups in the video frame matches a number of pixel groups in the still image.

### Claim 43 (depends on 32)

The method of claim 32 wherein the first image comprises a first still image and the second image comprises a second still image.

### Claim 44 (depends on 32)

The method of claim 32 wherein the first image comprises a first video frame and the second image comprises a second video frame.

### Claim 45 (depends on 32)

The method of claim 32 wherein the statistical measure is the mean of the range of intensity values in the first image.

### Claim 46 (depends on 32)

The method of claim 32 wherein the statistical measure is the average of the range of intensity values in the first image.

### Claim 47 (depends on 32)

The method of claim 32 further comprising generating the tone-mapping table for each color component in the second image.

### Claim 48

An apparatus for compensating for color tone differences between two images, the apparatus comprising:
a. an application for:
i. receiving a first image of a scene and a second image of the scene, wherein the first image is generated by a first source and the second image is generated by a second source;
ii. determining an intensity value for each pixel in the first image and for each pixel in the second image;
iii. aligning each pixel in the second image to a pixel in the first image;
iv. for each specific intensity value in the second image, establishing a relationship between the specific intensity value in the second image and a corresponding range of intensity values in the first image;
v. determining a conversion intensity value for each specific intensity value in the second image according to a statistical measure of the corresponding range of intensity values in the first image;
vi. generating a tone-mapping table according to the conversion intensity values such that each intensity value in the second image is associated with a specific conversion intensity value; and
vii. applying the tone-mapping table to one or more images generated by the second source, thereby forming one or more third images; and
b. a processor configured for executing the application; and
c. a memory coupled to the processor, the memory configured for temporarily storing data for execution by the processor.
