# US 2019/0246025 A1 · Adaptive Infrared Illumination for Exposure Correction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0246025 A1
- Document ID: US20190246025A1
- Status: Published application
- Kind: A1
- Application number: 15/890,254
- Filed: 2018-02-06
- Published: 2019-08-08
- Earliest claimed: 2018-02-06
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Bill Duran; Adrian Mircea Proca; Nikhil Sathe
- Web page: https://patentgrep.com/patent/US20190246025A1

## CPC

- H04N 23/74: by influencing the scene brightness using illuminating means
- G03B 15/03: Combinations of cameras with lighting apparatus; Flash units
- G08B 13/19604: {involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change}
- H04N 23/10: for generating image signals from different wavelengths
- H04N 23/20: for generating image signals from infrared radiation only
- H04N 23/56: provided with illuminating means
- H04N 23/71: Circuitry for evaluating the brightness variation
- H04N 23/73: by influencing the exposure time
- H04N 23/76: by influencing the image signals
- H04N 25/61: the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

A system and method that provides adaptive IR illumination for a camera. In some implementations, IR illumination adapts to the objects in a scene being monitored and IR illumination adaptation may be implemented in conjunction with automatic exposure and other exposure adjustment methods. Some implementations include: illuminating a scene using a first IR illumination power; capturing a first image of the scene with a camera; and determining whether a first region of the first image is underexposed or overexposed. When the first region is underexposed, the scene is illuminated at a second power level greater than the first illumination power and a second image is captured. When the first region is overexposed, the scene is illuminated at a third power level that is less than the first illumination power and a third image is captured. The operations are repeated until scene illumination is satisfactory.

## Claims

### Claim 1

A method implemented in a camera system, comprising:
when the camera system is in a night mode, wherein the camera system includes a lens assembly, an image sensor, a memory for storing image data from the image sensor, and an IR illuminator:
illuminating by the IR illuminator a scene using a first IR light at a first illumination power;
capturing by the image sensor a first image of the scene within a field of view of the lens assembly and storing in the memory first image data corresponding to the first image;
determining whether a first region of the first image has an illumination level below a lower illumination threshold, resulting in loss of image detail in the first region due to underexposure, or an illumination level above an upper illumination threshold, resulting in loss of image detail in the first region due to overexposure;
in response to the determining,
(A) when the first region has the illumination level below the lower illumination threshold:
illuminating by the IR illuminator the scene using a second IR light at a second power level that is greater than the first illumination power;
capturing by the image sensor a second image of the scene; and
storing in the memory second image data corresponding to the second image; and
(B) when the first region has the illumination level above the upper illumination threshold:
illuminating by the IR illuminator the scene using a third IR light at a third power level that is less than the first illumination power;
capturing by the image sensor a third image of the scene; and
storing in the memory third image data corresponding to the third image.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
when the first region has the illumination level below the lower illumination threshold, increasing a sensitivity of the image sensor; and
when the first region has the illumination level above the upper illumination threshold, decreasing the sensitivity of the image sensor.

### Claim 3 (depends on 1)

The method of claim 1, wherein the lens assembly comprises a shutter system, the method further comprising:
when the first region has the illumination level below the lower illumination threshold, increasing a shutter speed of the shutter system; and
when the first region has the illumination level above the upper illumination threshold, decreasing the shutter speed of the shutter system.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
after storing in the memory the second image data corresponding to the second image:
repeating the illuminating, capturing, and determining operations at the second power level; and
after storing in the memory the third image data corresponding to the third image:
repeating the illuminating, capturing, and determining operations at the third power level.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining whether the first region of the second image still has the illumination level below the lower illumination threshold;
in accordance with a determination that the first region still has the illumination level below the lower illumination threshold, increasing the second power level, and repeating the illuminating, capturing, and determining operations at the increased second power level.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
determining whether the first region of the third image still has the illumination level above the upper illumination threshold;
in accordance with a determination that the first region still has the illumination level above the upper illumination threshold, decreasing the third power level, and repeating the illuminating, capturing, and determining operations at the decreased third power level.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining whether a first region of the first image has an illumination level below a lower illumination threshold or an illumination level above an upper illumination threshold comprises obtaining a histogram of the first image.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining whether a first region of the first image has an illumination level below a lower illumination threshold or an illumination level above an upper illumination threshold comprises dividing the first image into a plurality of regions, wherein the first region is one of the plurality of regions.

### Claim 9 (depends on 8)

The method of claim 8, wherein determining whether a first region of the first image has an illumination level below a lower illumination threshold or an illumination level above an upper illumination threshold comprises obtaining lux level values for the plurality of regions of the first image.

### Claim 10

A camera system, comprising:
a lens assembly;
an image sensor;
a memory for storing image data from the image sensor;
an IR illuminator;
one or more processors; and
memory storing one or more programs for execution by the one or more processors, the one or more programs comprising instructions for:
when the camera system is in a night mode:
illuminating by the IR illuminator a scene using a first IR light at a first illumination power;
capturing by the image sensor a first image of the scene within a field of view of the lens assembly and storing in the memory first image data corresponding to the first image;
determining whether a first region of the first image has an illumination level below a lower illumination threshold, resulting in loss of image detail in the first region due to underexposure, or an illumination level above an upper illumination threshold, resulting in loss of image detail in the first region due to overexposure;
in response to the determining,
(A) when the first region has the illumination level below the lower illumination threshold:
illuminating by the IR illuminator the scene using a second IR light at a second power level that is greater than the first illumination power;
capturing by the image sensor a second image of the scene; and
storing in the memory second image data corresponding to the second image; and
(B) when the first region has the illumination level above the upper illumination threshold:
illuminating by the IR illuminator the scene using a third IR light at a third power level that is less than the first illumination power;
capturing by the image sensor a third image of the scene; and
storing in the memory third image data corresponding to the third image.

### Claim 11 (depends on 10)

The camera system of claim 10, wherein the IR illuminator comprises one or more illuminator elements, each of the illuminator elements providing substantially uniform illumination to the scene, wherein the illuminating by the IR illuminator of the scene using a second IR light at a second power level that is greater than the first illumination power comprises uniformly increasing input power to the one or more illuminator elements, and illuminating by the IR illuminator of the scene using a third IR light at a third power level that is less than the first illumination power comprises uniformly decreasing the input power to the one or more illuminator elements.

### Claim 12 (depends on 10)

The camera system of claim 10, wherein the IR illuminator includes multiple IR illuminator elements, each of which provides substantially uniform illumination to the scene and each of which can be individually turned on or off, wherein the illuminating by the IR illuminator of the scene using a second IR light at a second power level that is greater than the first illumination power comprises increasing power to a first subset of the illuminator elements, and illuminating by the IR illuminator of the scene using a third IR light at a third power level that is less than the first illumination power comprises decreasing power to and/or turning off a second subset of the illuminator elements.

### Claim 13 (depends on 10)

The camera system of claim 10, wherein the IR illuminator includes a single IR illuminator element that has an adjustable spatial illumination distribution with respect to the scene, wherein the illuminating by the IR illuminator of the scene using a second IR light at a second power level that is greater than the first illumination power comprises configuring the illuminator element to produce more illumination power directed substantially to the first region, and illuminating by the IR illuminator of the scene using a third IR light at a third power level that is less than the first illumination power comprises configuring the illuminator element to produce less illumination directed substantially to the first region.

### Claim 14 (depends on 10)

The camera system of claim 10, where the IR illuminator includes multiple IR illuminator elements each of which has a different spatial illumination direction with respect to the scene being monitored and each of which can be individually turned on or off, wherein the illuminating by the IR illuminator of the scene using a second IR light at a second power level that is greater than the first illumination power comprises increasing power to a fourth subset of the illuminator elements that produce illumination directed substantially to the first region, and illuminating by the IR illuminator the scene using a third IR light at a third power level that is less than the first illumination power comprises decreasing input power to and/or turning off at least the fourth subset of the illuminator elements.

### Claim 15 (depends on 10)

The camera system of claim 10, further comprising:
tiling the first image data to produce a plurality of tiles each corresponding to the first image date from a distinct predefined portion of the field of view and determining whether each of the tiles includes an overexposure condition or an underexposure condition; and
defining as the first region a contiguous region of the tiles including the overexposure or the underexposure condition.

### Claim 16

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by a camera system with a lens assembly, an image sensor, a memory for storing image data from the image sensor, an IR illuminator, and one or more processors, cause the camera system to:
when the camera system is in a night mode:
illuminate by the IR illuminator a scene using a first IR light at a first illumination power;
capture by the image sensor a first image of the scene within a field of view of the lens assembly and store in the memory first image data corresponding to the first image;
determine whether a first region of the first image has an illumination level below a lower illumination threshold, resulting in loss of image detail in the first region due to underexposure, or an illumination level above an upper illumination threshold, resulting in loss of image detail in the first region due to overexposure;
in response to the determining,
(A) when the first region has the illumination level below the lower illumination threshold:
illuminate by the IR illuminator the scene using a second IR light at a second power level that is greater than the first illumination power;
capture by the image sensor a second image of the scene; and
store in the memory second image data corresponding to the second image; and
(B) when the first region has the illumination level above the upper illumination threshold:
illuminate by the IR illuminator the scene using a third IR light at a third power level that is less than the first illumination power;
capture by the image sensor a third image of the scene; and
store in the memory third image data corresponding to the third image.

### Claim 17 (depends on 16)

The computer readable storage medium of claim 16, the one or more instructions further comprising instructions for:
tiling the first image data to produce a plurality of tiles each corresponding to the first image date from a distinct predefined portion of the field of view;
determining using an automatic exposure module of the camera system whether each of the tiles includes an overexposure condition or an underexposure condition; and
defining as the first region a contiguous region of the tiles including the overexposure or the underexposure condition.

### Claim 18 (depends on 16)

The computer readable storage medium of claim 16, the one or more instructions further comprising instructions for:
in combination with adjustment of the power of the IR illuminator, adjusting one or more of the camera's imaging settings, including frame rate, sensor gain, aperture, and exposure time, to compensate for the overexposure or the underexposure.

### Claim 19 (depends on 16)

The computer readable storage medium of claim 16, the one or more instructions further comprising instructions for:
detecting an object of interest in the scene; and
identifying a position of the object of interest in the field of view as the region so as to provide correct exposure of the object of interest.

### Claim 20 (depends on 19)

The computer readable storage medium of claim 19, the one or more instructions further comprising instructions for:
tracking the object of interest as it moves in the scene; and
repeating the identifying operation so as to provide correct exposure of the object of interest as it moves in the scene.

### Claim 21 (depends on 16)

The computer readable storage medium of claim 16, wherein the first image data comprises a frame of video data.

### Claim 22 (depends on 19)

The computer readable storage medium of claim 19, the one or more instructions further comprising instructions for: transmitting the first image data to a server, wherein the server performs the detecting of the object of interest and the identifying of the position of the object of interest in the field of view.

### Claim 23 (depends on 19)

The computer readable storage medium of claim 19, wherein the object of interest is the face of a person in the field of view.
