# US 11,994,591 B2 · Determining depth using multiple modulation frequencies

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,994,591 B2
- Document ID: US11994591B2
- Status: Granted
- Kind: B2
- Application number: 17/115,180
- Filed: 2020-12-08
- Published: 2022-06-09
- Granted: 2024-05-28
- Earliest claimed: 2020-12-08
- Assignees: Zoox, Inc.
- Applicants: Zoox, Inc.
- Inventors: Subasingha Shaminda Subasingha; Yongzhe Chen; Mehran Ferdowsi; Samuel Holladay; Turhan Karadeniz; Robert Nicholas Moor; Joseph Patrick Warga; Harrison Thomas Waschura; Silas Kogure Wilkinson
- Web page: https://patentgrep.com/patent/US11994591B2

## CPC

- G01S 17/931: of land vehicles
- G01S 7/484: Transmitters
- G01S 13/931: of land vehicles
- G01S 15/931: of land vehicles
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 7/55: from multiple images
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- B60W 60/001: {Planning or execution of driving tasks}
- B60W 2554/402: Type
- B60W 2554/4041: Position
- G01S 17/14: wherein a voltage or current pulse is initiated and terminated in accordance with the pulse transmission and echo reception respectively, e.g. using counters
- G01S 17/89: for mapping or imaging
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30241: Trajectory
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T 2207/30261: Obstacle
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. The sensor may generate first image data at a first configuration and second image data at a second configuration. An estimated depth of an object may be determined from the first image data, and an actual depth of the object may be determined from the second image data, based on the estimated depth. In examples, the first and second configurations have different modulation frequencies such that a nominal maximum depth in the first configuration is greater than the nominal maximum depth in the second configuration.

## Claims

### Claim 1

A vehicle comprising:
a time-of-flight sensor;
one or more processors; and
memory storing processor-executable instructions that, when executed by the one or more processors, configure the vehicle to:
configure the time-of-flight sensor in a first configuration;
receive, with the time-of-flight sensor in the first configuration, first depth information generated by the time-of-flight sensor, the first depth information being based, at least in part, on a first modulation frequency;
determine, based on the first depth information, an estimated depth of an object from the time-of-flight sensor, the estimated depth not exceeding a first nominal maximum sensor depth associated with the first modulation frequency;
configure the time-of-flight sensor in a second configuration;
receive, with the time-of-flight sensor in the second configuration, second depth information generated by the time-of-flight sensor, the second depth information being based, at least in part, on a second modulation frequency higher than the first modulation frequency;
determine a plurality of candidate depth measurements based, at least in part, on the second depth information, a first candidate depth of the candidate depth measurements comprising a nominal depth not exceeding a second nominal maximum sensor depth associated with the second modulation frequency and a second candidate depth of the candidate depth measurements comprising a sum of the nominal depth and a multiple of the second nominal maximum sensor depth; and
determine a measured distance of the object from the time-of-flight sensor as one of the plurality of candidate depth measurements closest to the estimated depth.

### Claim 2 (depends on 1)

The vehicle of claim 1, wherein:
the second modulation frequency is selected to provide the second nominal maximum sensor depth less than the first nominal maximum sensor depth associated with the first modulation frequency.

### Claim 3 (depends on 2)

The vehicle of claim 2, wherein the first modulation frequency is equal to or less than 10 MHz and the second modulation frequency is equal to or greater than 15 MHz.

### Claim 4 (depends on 1)

The vehicle of claim 1, wherein the plurality of candidate depth measurements comprises a first depth measurement and a second depth measurement, the second depth measurement comprising a sum of the first depth measurement and the second nominal maximum sensor depth associated with the second modulation frequency.

### Claim 5 (depends on 1)

The vehicle of claim 1, wherein the instructions further configure the vehicle to:
determine, based at least in part on at least one of the measured distance of the object, a position of the object in an environment of the vehicle;
determine a travel path through the environment relative to the object; and
control the vehicle to travel along the travel path.

### Claim 6

A method comprising:
configuring a time-of-flight sensor in a first configuration, the first configuration having an associated first modulation frequency;
emitting, with the time-of-flight sensor in the first configuration, first emitted light at the first modulation frequency;
receiving, at the time-of-flight sensor, first return light associated with the first emitted light;
determining, based at least in part on the first return light, first depth information about a surface;
determining, based on the first depth information, an estimated depth of the surface from the time-of-flight sensor, the estimated depth not exceeding a first nominal maximum depth of the time-of-flight sensor associated with the first configuration;
configuring the time-of-flight sensor in a second configuration having an associated second modulation frequency higher than the first modulation frequency;
emitting, with the time-of-flight sensor in the second configuration, second emitted light at the second modulation frequency;
receiving, at the time-of-flight sensor, second return light associated with the second emitted light;
determining, based at least in part on the second return light, second data comprising second depth information about the surface;
determining, based at least in part on the second depth information, a plurality of candidate depths of the surface, the plurality of candidate depths comprising a nominal depth not exceeding a second nominal maximum depth of the time-of-flight sensor associated with the second configuration and a first candidate depth comprising a sum of the nominal depth and the second nominal maximum depth; and
determining a measured depth of the surface as one of the plurality of candidate depths closest to the estimated depth.

### Claim 7 (depends on 6)

The method of claim 6, wherein the estimated depth is a sensed depth of the surface determined from the first depth information.

### Claim 8 (depends on 6)

The method of claim 6, wherein the first modulation frequency is selected to provide the first nominal maximum depth of the time-of-flight sensor in the first configuration and the second modulation is selected to provide the second nominal maximum depth of the time-of-flight sensor in the second configuration, the second nominal maximum depth being less than the nominal maximum depth.

### Claim 9 (depends on 8)

The method of claim 8, wherein the determining the measured depth comprises:
determining, based on the second depth information, the plurality of candidate depths;
comparing the plurality of candidate depths to the estimated depth; and
determining the measured depth as the one of the plurality of candidate depths closest to the estimated depth.

### Claim 10 (depends on 9)

The method of claim 9, wherein the plurality of candidate depths comprise a measured depth of the surface determined from the second depth information and a second depth comprising a sum of the measured depth and the second nominal maximum depth of the time-of-flight sensor in the second configuration.

### Claim 11 (depends on 10)

The method of claim 10, wherein the measured depth and the second depth are shorter than the first nominal maximum depth associated with the time-of-flight sensor in the first configuration.

### Claim 12 (depends on 6)

The method of claim 6, wherein the first modulation frequency is equal to or less than 10 MHz and the second modulation frequency is equal to or greater than 15 MHz.

### Claim 13 (depends on 12)

The method of claim 12, wherein the first modulation frequency is from 3 MHz to 5 MHz and the second modulation frequency is from 21 MHz to 24 MHz.

### Claim 14 (depends on 6)

The method of claim 6, further comprising:
determining, based at least in part on the actual depth of the surface, a position of an object in an environment of a vehicle;
determining a trajectory of the vehicle through the environment relative to the object; and
controlling the vehicle to travel along the trajectory.

### Claim 15

A system comprising:
one or more processors; and
computer-readable storage media storing instructions executable by the one or more processors to perform acts comprising:
causing a time-of-flight sensor to be configured in a first configuration, the first configuration having an associated first modulation frequency;
causing the time-of-flight sensor to emit, with the time-of-flight sensor in the first configuration, first emitted light at the first modulation frequency;
determining, based at least in part on first return light associated with the first emitted light and received at the time-of-flight sensor, first time-of-flight sensor information associated with the first configuration, the first time-of-flight sensor information including first depth information;
determining an estimated distance of a surface based on the first depth information, the estimated distance not exceeding a first nominal maximum depth associated with the first configuration;
causing the time-of-flight sensor to be configured in a second configuration, the second configuration having an associated second modulation frequency higher than the first modulation frequency;
causing the time-of-flight sensor to emit, with the time-of-flight sensor in the second configuration, second emitted light at the second modulation frequency;
determining, based at least in part on second return light associated with the first emitted light and received at the time-of-flight sensor, second time-of-flight sensor information determined in the second configuration, the second time-of-flight sensor information including second depth information and the second configuration having an associated second nominal maximum depth different from the first nominal maximum depth; and
determining, based at least in part on the second depth information and the estimated distance, a measured depth of the surface, the measured depth corresponding to a candidate depth of a plurality of candidate depths, the plurality of candidate depths comprising a nominal depth not exceeding a second nominal maximum depth of the time-of-flight sensor associated with the second configuration and a first candidate depth comprising a sum of the nominal depth and the second nominal maximum depth.

### Claim 16 (depends on 15)

The system of claim 15, wherein the estimated depth is a sensed depth of the surface determined from the first depth information.

### Claim 17 (depends on 15)

The system of claim 15, wherein the determining the measured depth comprises:
determining, based on the second depth information, the plurality of candidate depths for the surface;
comparing the plurality of candidate depths to the estimated depth; and
determining the measured depth of the surface as the one of the plurality of candidate depths closest to the estimated depth.

### Claim 18 (depends on 17)

The system of claim 17, wherein the plurality of candidate depths comprise a measured depth of the surface determined from the second depth information and a second depth comprising a sum of the measured depth and the second nominal maximum depth associated with the second configuration.

### Claim 19 (depends on 18)

The system of claim 18, wherein the measured depth and the second depth are shorter than the first nominal maximum depth associated with the time-of-flight sensor in the first configuration.

### Claim 20 (depends on 15)

The system of claim 15, wherein the first modulation frequency is equal to or less than 10 MHz and the second modulation frequency is equal to or greater than 15 MHz.