# US 11,445,127 B2 · Leveraging HDR sensors for handling mixed illumination auto white balance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,445,127 B2
- Document ID: US11445127B2
- Status: Granted
- Kind: B2
- Application number: 16/901,152
- Filed: 2020-06-15
- Published: 2021-12-16
- Granted: 2022-09-13
- Earliest claimed: 2020-06-15
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Omry Sendik; Roee Sfaradi; Doron Sabo
- Web page: https://patentgrep.com/patent/US11445127B2

## CPC

- H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
- H04N 23/76: by influencing the image signals
- G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215)
- G06T 7/90: Determination of colour characteristics
- H04N 23/71: Circuitry for evaluating the brightness variation
- H04N 23/73: by influencing the exposure time
- H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control
- G06T 2207/10024: Color image
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods for setting the white balance of an image are described. Embodiments of the systems and methods may receive image data comprising a plurality of exposures, generate a plurality of white balance values based on merge information from a high dynamic range (HDR) merge of the exposures, and adjust a white balance of each pixel of the image data based on the white balance values.

## Claims

### Claim 1

A method for image processing, comprising:
receiving image data comprising a plurality of exposures;
identifying exposure information for each pixel of the image data based on a high dynamic range (HDR) merge of the exposures,
wherein the exposure information comprises a correspondence between the pixel and one or more of the exposures, and
wherein the image data is divided into illumination segments based on the exposure information;
identifying a plurality of white balance regions; and
generating a plurality of white balance values based on the exposure information;
adjusting a white balance of each pixel of the image data based on the white balance values,
wherein each pixel of the image data corresponds to one of the white balance regions, each of the white balance regions corresponds to one of the white balance values, and the white balance of each pixel is adjusted based on the corresponding white balance region or a combination of corresponding white balance regions.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
performing the HDR merge on the exposures; and
dividing the image data into the illumination segments based on the HDR merge, wherein each of the white balance values corresponds to one of the illumination segments.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
exposing a camera sensor to light for a first length of time to produce a first exposure; and
exposing the camera sensor to the light for a second length of time different from the first length of time to produce a second exposure, wherein the exposures include at least the first exposure and the second exposure.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the exposure information indicates a single exposure for each of the pixels.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
the exposure information indicates a linear combination of the exposures for each of the pixels.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
identifying a factor comprising a linear combination of the exposures for each of the pixels;
selecting an exposure having a highest weight for each of the pixels; and
determining whether the highest weight for each of the pixels is above a threshold value, wherein the image data is divided into the illumination segments based on the determination.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
identifying an average RGB value for pixels in each of the illumination segments; and
applying one or more gains to an RGB value for each of the pixels in each of the illumination segments based on the corresponding average RGB value, wherein the white balance of each pixel in the image data is adjusted based on the applied one or more gains.

### Claim 8 (depends on 1)

The method of claim 1, wherein:
the image data is based on a plurality of illumination sources, and each of a plurality of illumination segments corresponds to one or more of the illumination sources.

### Claim 9 (depends on 1)

The method of claim 1, wherein:
the plurality of exposures comprise more than two exposures.

### Claim 10

An apparatus for image processing, comprising:
a camera;
a processor; and
a memory storing instructions and in electronic communication with the processor, the processor being configured to execute the instructions to:
receive image data comprising a plurality of exposures;
perform an HDR merge on the exposures;
divide the image data into a plurality of illumination segments based on the HDR merge;
generate a plurality of white balance values based on the plurality of illumination segments; and
identify an average RGB value for pixels in each of the illumination segments;
apply one or more gains to an RGB value for each of the pixels in each of the illumination segments based on the corresponding average RGB value;
adjust a white balance of each pixel of the image data based on the white balance values and based on the applied one or more gains.

### Claim 11 (depends on 10)

The apparatus of claim 10, the processor being further configured to execute the instructions to:
identify a factor comprising a linear combination of the exposures for each of the pixels;
select an exposure having a highest weight for each of the pixels; and
determine whether the highest weight for each of the pixels is above a threshold value, wherein the image data is divided into the illumination segments based on the determination.

### Claim 12 (depends on 10)

The apparatus of claim 10, the processor being further configured to execute the instructions to:
identify a plurality of white balance regions, wherein each pixel of the image data corresponds to one of the white balance regions, each of the white balance regions corresponds to one of the white balance values, and the white balance of each pixel is adjusted based on the corresponding white balance region.

### Claim 13

A non-transitory computer readable medium storing code for image processing, the code comprising instructions executable by a processor to:
receive image data comprising a plurality of exposures;
generate a plurality of white balance values based on merge information from a high dynamic range (HDR) merge of the exposures;
identify a plurality of white balance regions, wherein each pixel of the image data corresponds to one of the white balance regions, and each of the white balance regions corresponds to one of the white balance values; and
adjust a white balance of each pixel of the image data based on the white balance values and based on the corresponding white balance region.

### Claim 14 (depends on 13)

The non-transitory computer readable medium of claim 13, the code further comprising instructions executable by the processor to:
perform the HDR merge on the exposures; and
divide the image data into a plurality of illumination segments based on the HDR merge, wherein each of the white balance values corresponds to one of the illumination segments.

### Claim 15 (depends on 13)

The non-transitory computer readable medium of claim 13, the code further comprising instructions executable by the processor to:
identify a factor comprising a linear combination of the exposures for each of the pixels;
select an exposure having a highest weight for each of the pixels; and
determine whether the highest weight for each of the pixels is above a threshold value, wherein the image data is divided into the illumination segments based on the determination.

### Claim 16 (depends on 13)

The non-transitory computer readable medium of claim 13, the code further comprising instructions executable by the processor to:
identify an average RGB value for pixels in each of the illumination segments; and
apply one or more gains to an RGB value for each of the pixels in each of the illumination segments based on the corresponding average RGB value, wherein the white balance of each pixel in the image data is adjusted based on the applied one or more gains.