# US 10,872,430 B2 · Active illumination 3D zonal imaging system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,872,430 B2
- Document ID: US10872430B2
- Status: Granted
- Kind: B2
- Application number: 16/558,027
- Filed: 2019-08-30
- Published: 2019-12-19
- Granted: 2020-12-22
- Earliest claimed: 2017-07-11
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Onur Can Akkaya; Cyrus Soli Bamji
- Web page: https://patentgrep.com/patent/US10872430B2

## CPC

- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G01S 7/4817: {relating to scanning}
- G01S 7/4863: Detector arrays, e.g. charge-transfer gates
- G01S 17/10: using transmission of interrupted, pulse-modulated waves (determination of distance by phase measurements G01S17/32)
- G01S 17/18: wherein range gates are used
- G01S 17/32: using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
- G01S 17/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46)
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- H04N 5/44504: {Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits}
- H04N 23/56: provided with illuminating means
- H04N 23/74: by influencing the scene brightness using illuminating means
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An active illumination range camera comprising illumination and imaging systems that is operable to provide a range image of a scene in the imaging system's field of view (FOV) by partitioning the range camera FOV into sub-FOVs, and controlling the illumination and imaging systems to sequentially illuminate and image portions of the scene located in the respective sub-FOVs.

## Claims

### Claim 1

An active illumination range camera operable to determine distances to features in a scene, the range camera comprising:
an imaging system characterized by a field of view (FOV) and comprising a photosensor having light sensitive pixels, and an optical system configured to collect light from a scene in the FOV and image the collected light onto pixels of the photosensor;
an illumination system controllable to generate and direct a field of illumination (FOI) to illuminate at least a portion of the FOV; and
a controller operable to:
partition the at least a portion of the FOV into a plurality of zones;
control the illumination system to generate and direct a FOI to sequentially illuminate the zones in turn with transmitted light through selectively activating and deactivating portions of the photosensor correlated to pixels of the zones;
accumulate photocharge reflected by features within the zones from the transmitted light; and
increment a counter value upon illumination of each of the zones;
halt illumination of the zones once the counter value reaches a maximum count; and
determine distances to features in the scene using the accumulated photocharge reflected by the features within the zones from the transmitted light.

### Claim 2 (depends on 1)

The active illumination range camera according to claim 1 wherein sequentially illuminating the zones in turn comprises illuminating each zone a plurality of times to acquire data sufficient to determine distances to the features of the scene in the zone before illuminating a next zone.

### Claim 3 (depends on 2)

The active illumination range camera according to claim 2 wherein the controller is operable following illumination of each zone the plurality of times to read out the pixels in the corresponding region.

### Claim 4 (depends on 3)

The active illumination range camera according to claim 3 further comprising providing a range image of the features in each zone and combining the provided range images to provide a range image of the scene.

### Claim 5 (depends on 1)

The active illumination range camera according to claim 1 wherein sequentially illuminating the zones in turn comprises:
(a) illuminating each zone once in sequence to accumulate the photocharge to acquire a portion of the data for providing a range image of the scene; and
(b) repeating (a) a number of times until the counter value reaches the maximum count.

### Claim 6 (depends on 5)

The active illumination range camera according to claim 5 wherein the controller is operable following illumination of a zone to store the photocharge accumulated by pixels in photosensor region corresponding to the zone in the pixels.

### Claim 7 (depends on 6)

The active illumination range camera according to claim 6 wherein the controller is operable upon completion of each repetition, to read the photosensor to acquire a frame of the photosensor.

### Claim 8 (depends on 7)

The active illumination range camera according to claim 7 wherein the controller is operable to using data in all the frames to provide a range image of the scene following a last acquisition of a frame.

### Claim 9 (depends on 6)

The active illumination range camera according to claim 6 and comprising reading the pixels in the region to acquire a frame of the region prior to illuminating a next region of the photosensor.

### Claim 10 (depends on 1)

The active illumination range camera according to claim 1 wherein partitioning the FOV into zones comprises determining a number of zones.

### Claim 11 (depends on 10)

The active illumination range camera according to claim 10 wherein determining a number of the zones comprises determining the number based on reflectance of features in the FOV.

### Claim 12 (depends on 1)

The active illumination range camera according to claim 1 wherein the controller is operable to control intensity of light in the FOI illuminating a zone and/or duration of exposure of the zone to the light based on reflectance of features in the zone.

### Claim 13 (depends on 1)

The active illumination range camera according to claim 1 and comprising at least one or any combination of more than one of a continuous wave time of flight (CW-TOF) camera, a gated time of flight (GT-TOF) camera, a structured light range camera, and/or a stereo range camera.

### Claim 14 (depends on 1)

The active illumination range camera according to claim 1 and comprising:
controlling the illumination system to generate and direct a FOI that illuminates a zone with time modulated light characterized by a modulation frequency; and
differentially time modulating sensitivity to light of pixels in the corresponding region of the photosensor at the modulation frequency of the modulated light but phase shifted from a phase of modulation of the light by an offset phase.

### Claim 15

A method for determining distances to features in a scene, the method comprising:
partitioning a field of view (FOV) of a camera in which the scene is located into a plurality of zones;
generating and directing a field of illumination of a light source (FOI) to sequentially illuminate the zones in turn with transmitted light through selectively activating and deactivating portions of a photosensor correlated to pixels of the zones;
accumulating photocharge reflected by features within zones from the transmitted light;
incrementing a counter value upon illumination of each of the zones;
halt illumination of the zones once the counter value reaches a maximum count; and
determining distances to features in the scene using the accumulated photocharge reflected by the features within the zones from the transmitted light.

### Claim 16 (depends on 15)

The method according to claim 15 wherein sequentially illuminating the zones in turn comprises illuminating each zone a plurality of times to acquire data sufficient to determine distances to features of the scene in the zone before illuminating a next zone.

### Claim 17 (depends on 16)

The method according to claim 16 and following illumination each zone the plurality of times reading out the pixels in the corresponding region.

### Claim 18 (depends on 15)

The method according to claim 15 wherein sequentially illuminating the zones in turn comprises:
(a) illuminating each zone once in sequence to accumulate photocharge to acquire a portion of the data for providing a range image of the scene;
(b) repeating (a) a number of times sufficient to acquire all the data for providing the range image.

### Claim 19 (depends on 18)

The method according to claim 18 and comprising:
following illumination of a zone storing the photocharge accumulated by pixels in the photosensor region corresponding to the zone in the pixels; and
upon completion of each repetition, reading the photosensor to acquire a frame of the photosensor; and
following completion of repetition last repetition using data in all the frames to provide a range image of the scene.

### Claim 20

One more computer-storage memory devices embodied with instructions that are executable for determining distances to features in a scene, comprising:
partitioning a field of view (FOV) of a camera in which the scene is located into a plurality of zones;
generating and directing a field of illumination of a light source (FOI) to sequentially illuminate the zones in turn with transmitted light through selectively activating and deactivating portions of a photosensor correlated to pixels of the zones;
accumulating photocharge reflected by features within zones from the transmitted light;
incrementing a counter value upon illumination of each of the zones;
halt illumination of the zones once the counter value reaches a maximum count; and
determining distances to features in the scene using the accumulated photocharge reflected by the features within the zones from the transmitted light.