# US 2021/0067719 A1 · Dynamic Range for Depth Sensing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0067719 A1
- Document ID: US20210067719A1
- Status: Published application
- Kind: A1
- Application number: 17/096,472
- Filed: 2020-11-12
- Published: 2021-03-04
- Earliest claimed: 2016-11-29
- Assignees: X Development LLC
- Applicants: X Development LLC
- Inventors: Emily Cooper; Fengqiang Li
- Web page: https://patentgrep.com/patent/US20210067719A1

## CPC

- H04N 13/243: using three or more two-dimensional [2D] image sensors
- G01B 11/2513: {with several lines being projected in more than one direction, e.g. grids, patterns}
- G01S 1/00: Beacons or beacon systems transmitting signals having a characteristic or characteristics capable of being detected by non-directional receivers and defining directions, positions, or position lines fixed relatively to the beacon transmitters; Receivers co-operating therewith (position fixing by co-ordinating a plurality of determinations of direction or position lines G01S5/00)
- G01S 11/12: using electromagnetic waves other than radio waves
- H04N 23/71: Circuitry for evaluating the brightness variation
- H04N 23/74: by influencing the scene brightness using illuminating means
- H04N 23/75: by influencing optical camera components
- H04N 25/58: involving two or more exposures
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

An example system includes a patterned light projector operable to direct first and second portions of patterned light toward first and second surfaces, respectively, in an environment. The first and second surfaces may be at first and second distances, respectively, from the structured light projector. A graduated optical filter may be situated along an optical path of the patterned light. The graduated optical filter includes first and second regions to attenuate an intensity of the first and second portions of the patterned light, respectively, by first and second amounts, respectively. The first amount is greater than the second amount. The system additionally includes an image sensor operable to generate image data based on at least the first and second portions of the patterned light and a processor configured to determine first and second values indicative of an estimate of the first and second distances, respectively, based on the image data.

## Claims

### Claim 1

A system comprising:
a patterned light projector comprising a light source and a mask situated between the light source and an environment, wherein the patterned light projector is operable to project (i) a first portion of patterned light onto a first surface in the environment and (ii) a second portion of the patterned light onto a second surface in the environment, wherein the first portion of the patterned light is generated by a first set of gaps in a first region of the mask, wherein the second portion of the patterned light is generated by a second set of gaps in a second region of the mask, and wherein a second size of the second set of gaps is greater than a first size of the first set of gaps;
at least one image sensor operable to generate image data representing the environment, wherein at least some of the image data represents the environment illuminated by the first portion of the patterned light and the second portion of the patterned light; and
a control system configured to:
identify, based on the image data, an object of interest within the environment; and
based on identifying the object, cause repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.

### Claim 2 (depends on 1)

The system of claim 1, wherein the second region of the mask surrounds and is coplanar with the first region of the mask.

### Claim 3 (depends on 1)

The system of claim 1, wherein the control system is further configured to:
determine, based on the image data, that the object is illuminated by the second portion of the patterned light; and
cause the repositioning of the patterned light projector further based on determining that the object is illuminated by the second portion of the patterned light.

### Claim 4 (depends on 1)

The system of claim 1, wherein the control system is further configured to:
determine an extent of exposure of a portion of the image data that represents the object; and
based additionally on the extent of exposure, cause the repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.

### Claim 5 (depends on 1)

The system of claim 1, wherein the control system is further configured to:
determine, based on the image data, a value indicative of a distance between the object and the patterned light projector; and
based additionally on the value indicative of the distance, cause the repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.

### Claim 6 (depends on 1)

The system of claim 1, further comprising:
a robotic device, wherein the control system is further configured to:
cause the at least one image sensor to generate additional image data representing the object illuminated by the first portion of the patterned light; and
cause the robotic device to navigate through the environment based on the additional image data.

### Claim 7 (depends on 1)

The system of claim 1, wherein the image data comprises (i) red-green-blue (RGB) image data and (ii) infrared (IR) image data, and wherein the patterned light comprises IR light.

### Claim 8 (depends on 7)

The system of claim 7, wherein the control system is configured to identify the object of interest based on the RGB image data.

### Claim 9 (depends on 1)

The system of claim 1, wherein the at least one image sensor comprises a first image sensor and a second image sensor, wherein the image data comprises first image data generated by the first image sensor and second image data generated by the second image sensor, wherein the first image data and the second image data each represent the environment illuminated by the first portion of the patterned light and the second portion of the patterned light, and wherein the control system is further configured to:
identify the object of interest based on corresponding features between the first image data and the second image data.

### Claim 10 (depends on 1)

The system of claim 1, wherein the mask comprises an opaque material, and wherein the first set of gaps and the second set of gaps each comprise a respective set of transparent regions arranged to define a pattern of the patterned light.

### Claim 11 (depends on 1)

The system of claim 1, wherein the first set of gaps comprise a first set of diffractive optical elements, and wherein the second set of gaps comprises a second set of diffractive optical elements.

### Claim 12

A method comprising:
projecting, by a patterned light projector, (i) a first portion of patterned light onto a first surface in an environment and (ii) a second portion of the patterned light onto a second surface in the environment, wherein the patterned light projector comprises a light source and a mask situated between the light source and the environment, wherein the first portion of the patterned light is generated by a first set of gaps in a first region of the mask, wherein the second portion of the patterned light is generated by a second set of gaps in a second region of the mask, and wherein a second size of the second set of gaps is greater than a first size of the first set of gaps;
generating, by at least one image sensor, image data representing the environment, wherein at least some of the image data represents the environment illuminated by the first portion of the patterned light and the second portion of the patterned light;
identifying, by a control system and based on the image data, an object of interest within the environment; and
causing, by the control system and based on identifying the object, repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.

### Claim 13 (depends on 12)

The method of claim 12, wherein the second region of the mask surrounds and is coplanar with the first region of the mask.

### Claim 14 (depends on 12)

The method of claim 12, further comprising:
determining, based on the image data, that the object is illuminated by the second portion of the patterned light; and
causing the repositioning of the patterned light projector further based on determining that the object is illuminated by the second portion of the patterned light.

### Claim 15 (depends on 12)

The method of claim 12, further comprising:
determining an extent of exposure of a portion of the image data that represents the object; and
based additionally on the extent of exposure, causing the repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
determining, based on the image data, a value indicative of a distance between the object and the patterned light projector; and
based additionally on the value indicative of the distance, causing the repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.

### Claim 17

A non-transitory computer readable storage medium having stored thereon instructions that, when executed by a computing system, cause the computing system to perform operations comprising:
causing a patterned light projector to project (i) a first portion of patterned light onto a first surface in an environment and (ii) a second portion of the patterned light onto a second surface in the environment, wherein the patterned light projector comprises a light source and a mask situated between the light source and the environment, wherein the first portion of the patterned light is generated by a first set of gaps in a first region of the mask, wherein the second portion of the patterned light is generated by a second set of gaps in a second region of the mask, and wherein a second size of the second set of gaps is greater than a first size of the first set of gaps;
obtaining, from at least one image sensor, image data representing the environment, wherein at least some of the image data represents the environment illuminated by the first portion of the patterned light and the second portion of the patterned light;
identifying, based on the image data, an object of interest within the environment; and
causing, based on identifying the object, repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the operations further comprise:
determining, based on the image data, that the object is illuminated by the second portion of the patterned light; and
causing the repositioning of the patterned light projector further based on determining that the object is illuminated by the second portion of the patterned light.

### Claim 19 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the operations further comprise:
determining an extent of exposure of a portion of the image data that represents the object; and
based additionally on the extent of exposure, causing the repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.

### Claim 20 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the operations further comprise:
determining, based on the image data, a value indicative of a distance between the object and the patterned light projector; and
based additionally on the value indicative of the distance, causing the repositioning of the patterned light projector such that the first portion of the patterned light is directed onto the object.
